oswbba

THE OSWATCHER ANALYZER USER'S GUIDE

Carl Davis
Center of Expertise
August 28 , 2017

To see how to use this tool and it's different features you can view a series of short "how to do" videos in document 301137.1

Contents

- Introduction

- Overview

- License

- Supported Platforms

- Installing oswbba

- Starting oswbba

- Java Heap Errors on Startup

- Stopping oswbba

- Using oswbba

  - Menu Option

  - Command Line Option

  - Understanding the (-D) and (-A) options

- Specifying the Begin/End Time of an Analysis (Recommended Method)

- Limiting the Analysis to a Subset of Files in Your Archive (Old method)

- Ignoring Specific Directories From Analysis

- Limiting the PS Analysis

- Generating a Dashboard

- Sample Charts

- Known Issues

- Reporting Feedback

Introduction

The OSWatcher Analyzer (oswbba) is a utility that allows the user to graph and analyze data collected from running OSWatcher. In addition to providing graphing and reporting capabilities, the analyzer will look for problems and provide recommendations on how to resolve the issues. Analyzing this data automatically avoids the time consuming task of manually inspecting all the files that OSWatcher collects..

NOTE: oswbba replaces the utility OSWg. This was done to eliminate the confusion caused by having multiple tools in Oracle support named OSWatcher. oswbba only analyzes data collected by OSWatcher (oswbb) and none of the other tools.

### Best Practices

#### Pro-Active Problem Avoidance and Diagnostic Collection

Although some problems may be unforeseen, in many cases problems may be avoidable if signs are detected early enough. Additionally, if an issue does occur, it is no use collecting information about that issue after the event. oswbb is one of the tools that support recommends for collecting such diagnostics. For information on suggested uses, other proactive preparations and diagnostics see:

<Document 1482811.1> Best Practices: Proactively Avoiding Database and Query Performance Issues
<Document 1477599.1> Best Practices Around Data Collection For Performance Issues

Back to Contents

Overview

oswbba is a graphing and analysis utility which comes bundled with OSWatcher. This utility is a stand-alone java program which requires java to be installed. Because oswbba is written in java, the OSWatcher files that have been collected on Unix/Linux, can be analyzed on different Unix/Linux/Windows platforms that have java installed. Please note oswbba does not run on any other platforms except Unix/Linux/Windows. Additionally, oswbba must be running in an x-windows environment in order to use all of its features.

oswbba analyzes the OSWatcher archive files (vmstat, iostat, top, ps, netstat, HP-UX sar and Linux meminfo) and automatically look for problems and helps to determine a root cause of the problem if possible. In addition to providing analysis, oswbba has an integrated dashboard which allows visibility into the different subsystems (cpu, memory, i/o, network and also nfs (Linux only). Problems can be quickly identified and displayed on the dashboard. The analysis text report is integrated with the browser dashboard so both textual data and graphics can be accessed inside the same browser window. See the OSW_Analyzer.pdf in the docs directory for more information.

Back to Contents

License

oswbba use is under Oracle's standard licensing terms and does not require additional licenses for use.

Back to Contents

Supported Platforms

oswbba is certified to run on the following platforms. It does not run on MacOS:

- AIX
- Solaris
- HP-UX
- Linux
- Windows

Back to Contents

Installing oswbba

oswbba is a Oracle support tool which requires no installation. It comes bundled with OSWatcher and is a standalone java jar file. Consult the README.txt for details about system requirements.

Back to Contents

Starting oswbba

Before starting the oswbba utility you must have java installed on your system. Java can be downloaded for free from https://java.com. Also if you are running Oracle you already have a version of java installed. To verify you have the correct version of java installed on your system issue the following command...

$ java -version

If java is not installed, contact your system administrator to get the current version of java installed. Alternatively, you can use the version of java that comes shipped with Oracle. Here is an example of using the version of java that comes shipped with the database...(depending upon the version of the database, the jre may be in a different location)

//Note: the location of the jre is in $ORACLE_HOME/jre/1.4.2/bin //now put this location in the Unix path: $ export PATH=$ORACLE_HOME/jre/1.4.2/bin:$PATH

Once the correct version of java has been verified, you can start oswbba. oswbba requires an input directory to run. To specify the input directory you must use the -i option. The input directory is the fully qualified path name of an archive directory location containing the oswbb logs. The archive directory must contain the respective subdirectories--oswvmstat, oswiostat, oswps, oswtop, oswnetstat, etc. If you are running Linux or HP-UX, then additional directories are also created. It is important to note the program requires an archive directory name not an individual log directory name or individual filename.

$java -jar oswbba.jar -i /u02/home/osw/archive Starting OSWbba V8.0.0 OSWatcher Analyzer Written by Oracle Center of Expertise Copyright (c) 2017 by Oracle Corporation Parsing Data. Please Wait... Enter 1 to Display CPU Process Queue Graphs Enter 2 to Display CPU Utilization Graphs Enter 3 to Display CPU Other Graphs Enter 4 to Display Memory Graphs Enter 5 to Display Disk IO Graphs Enter 5 to Display NFS Graphs Enter GC to Generate All CPU Gif Files Enter GM to Generate All Memory Gif Files Enter GD to Generate All Disk Gif Files Enter GN to Generate All Network Gif Files Enter GF to Generate All NFS Gif Files Enter A to Analyze Data Enter S to Analyze Subset of Data(Changes analysis dataset including graph time scale Enter D to Generate Dashboard Enter L to Specify Alternate Location of Gif Directory Enter T to Specify Different Time Scale Enter B to Return to Default Time scale Enter R to Remove Currently Displayed Graphs Enter X to Export Parsed Data to File Enter Q to Quit Program Please Select an Option:

Java Heap Errors on Startup

oswbba parses all the archive files in memory prior to generating graphs or performing an analysis. If you have a large amount of files to parse you may need to allocate more memory in the java heap. If you experience any error messages regarding out of memory such as java.lang.OutOfMemoryError, you may have to increase the size of the java heap. To increase the size of the java heap use the -Xmx flag.

$java -jar -Xmx512M oswbba.jar -i /u02/home/oswbb/archive Starting oswbba V7.0 OSWatcher Analyzer Written by Oracle Center of Expertise Copyright (c) 2012 by Oracle Corporation Parsing Data. Please Wait...

Back to Contents

Stopping oswbba

To stop the oswbba utility select option "Q" from the menu.

Back to Contents

Using oswbba

oswbba has multiple user interface options. If oswbba is started as above, the user will be able to choose from a list of options on a menu. In all cases oswbba must be supplied the archive directory location with the -i flag.

If the underlying data from the OSWatcher archive is not available, such as the case where the data collector is missing or cannot run due to permission issues, then the option to analyze that data will be missing from the menu of options.

Using oswbba: Menu Option

oswbba can be run with a menu driven user interface. This option gives the user the most flexibility and allows graphs to be displayed real-time. To start oswbba with the menu option issue the following on the command line...

java -jar oswbba.jar -i <fully qualified path name of an oswbb archive directory>

After starting oswbba, a set of options will display. Enter an option value and hit return.

OPTIONS The following options are supported:
1..3 | These options display graphs of specific CPU components of vmstat. Option 1 displays the process run, wait and block queues. Option 2 displays CPU utilization graphs for system, user and idle. Option 3 displays graphs for context switches and interrupts.
4 | This option displays memory graphs for free memory and available swap.
5 | This option displays graphs for disk i/o. The device name along with the average service time of each device is then listed. The user then selects one of the devices out of the list of devices. Graphs are available for reads/second, writes/second, service time and percent busy.
6 | This option display nfs statistics. This option is available only on Linux and only if nfsiostat is enabled in OSWatcher
GC | Generates image files of the graphs associated with OS CPU (Option 1, 2, 3 above). These files are by default written to the gif directory but can be written to any directory by the use of Option L below.
GM | Generates image files of the graphs associated with OS memory (Option 4 above). These files are by default written to the gif directory but can be written to any directory by the use of Option L below.
GD | Generates image files of i/o stats (Option 5 above). These files are by default written to the gif directory but can be written to any directory by the use of Option L below.
GN | Generates image files of the graphs associated with OS network (Option 5 above). These files are by default written to the gif directory but can be written to any directory by the use of Option L below.
GF | Generates image files of the graphs associated with NFSIOSTAT for Linux only (Option 6 above). These files are by default written to the gif directory but can be written to any directory by the use of Option L below.
L | This option allows the user to specify an alternative location to place the image and data (XLS) files.
T | By default OSWbba parses all the OSWbb log files contained in the input directory and the default graphs are based on the entire time span of all the logs. By default, OSWbb keeps the last 48 hours of logs in the archive. This means the default graph will graph all 48 hours of data. This option allows the user to specify a different subset of times within the entire collection. An example would be only to graph a 2 hour period out of the entire 48 hour collection.
B | This option resets the graphing timescale back to the time encompassing the entire log collection. Please note this resets the time for graphs but does not reset the time for analysis.
R | This option removes all graphs from the screen.
X | This option exports the analyzer parsed data to flat files that can be loaded into a spreadsheet. The exported data is written to the data subdirectory.
A | This option analyzes the files in the archive and produces a report.
S | This option analyzes a subset of the data in the oswbb directory and produces a report in the analysis directory.
D | Generates a responsive web page dashboard. The dashboard integrates the analyzer text base report (option A) with a menu driven interface that quickly helps identifies critical issues and provides guidance on what to look for. Selecting this option will first generate the analysis option (A) as this data is required for the dashboard option.
Q | Exits the program.

Back to Contents

Using oswbba: Command Line Option

All graphing, dashboard generation and analysis options are available to be passed into oswbba from the command line. Only the -i option is required. Use the table below to add additional options.

java -jar oswbba.jar -i <fully qualified path name of an oswbb archive directory> -P <name> -L <name> -6 -7 -8 Example: java -jar oswbba.jar -i archive -6 -7 -P tuesday_crash

OPTIONS The following options are supported:
-i | <archive dir> Required. This is the input archive directory location.
-GC | Same as option GC from the menu. Will generate all cpu gif files.
-GM | Same as option GM from the menu. Will generate all memory gif files.
-GD | Same as option GD from the menu. Will generate all disk gif files.
-GN | Same as option GN from the menu. Will generate all network gif files.
-GF | Same as option GF from the menu. Will generate all nfs gif files.
-S | This option used with other options to identify only a subset of the data in the oswbb directory will be used for analysis.
-L | <location name> User specified location of an existing directory to place any gif files or data files (xls files) generated by oswbba. This overrides the oswbba default location. This directory must pre-exist!
-A | <optional analysis directory name> Same as option A above. This option analyzes the files in the archive directory and produces a report in the analysis directory. If you do not specify an analysis directory, then the files will be placed in a system generated directory name under the analysis directory. If you do not want to put the optional name make sure this is the last option on the command line.
-D | <optional analysis directory name> Same as option D above. This option generates a dashboard (responsive web page). The dashboard integrates the analyzer text base report (option A) with a menu driven interface that quickly helps identifies critical issues and provides guidance on what to look for. Selecting this option will first also force the analysis (-A) to happen. If you do not specify an analysis directory, then the files will be placed in a system generated directory name under the analysis directory. If you do not want to put the optional name make sure this is the last option on the command line
-START | <filename>Used with the analysis option to specify the first file located in the oswvmstat directory to analyze.
-STOP | <filename>Used with the analysis option to specify the last file located in the oswvmstat directory to analyze.
-B | <begin time> Same as option T from the menu. The begin time will allow the user to select a begin time from within the archive of files to graph. This overrides the default start time which is the earliest time entry in the archive directory. The format of the start time is Mon DD HH:MM:SS YYYY. (Example :Jul 25 11:58:01 2007). An invalid begin time will result in using a default value of the first timestamp in your archive.
-E | <end time> Same as option T from the menu. The end time will allow the user to select an end time from within the archive of files to graph. This overrides the default end time which is the latest time entry in the archive directory. The format of the end time is Mon DD HH:MM:SS YYYY. (Example :Jul 25 11:58:01 2007).An invalid end time will result in using a default value of the last timestamp in your archive. .

Back to Contents

Understanding the (-D) and (-A) options

The (-D) and (-A) options specify a unique directory name under the oswbba/analysis directory where the analysis files get generated. Note this is a directory name and not a filename. This is a change from previous releases. If no directory name is specified then a default directory name will be generated. In this case make sure to place the (-D) or (-A) options at the end of the list of options. Note that the (-D) and (-A) options are mutually exclusive. Selecting the (-D) option will automatically trigger an analysis (-A) option. Therefore, it is not necessary to put a (-A) option alongside a (-D) option. Selecting the (-A) option however, will not trigger a (-D) dashboard. In both cases, the user specified directory name must be unique or the program will terminate with a warning.

Back to Contents

Specifying the Begin/End Time of an Analysis (Recommended Method)

Starting with release 7.0, the begin and end times could be specified to limit the analysis to a particular time of interest. This behavior changed in release 7.2. Begin and end times can be controlled by using the command line options -b and -e. These options are available if you run from the command line or can be chosen from the menu with the -s option. To use this option you must specify the begin time and the end time. If you specify either parameter without specifying the other parameter, or one or both of these times are not valid, the program will default to the following values. A warning will also be written to inform you that you have not specified valid inputs and default parameters will be used instead. Note: This is new behavior that differs from previous versions.

If -b is specified and no -e is specified then program warns the input is invalid and uses the last end date available in the archive.

If -e is specified and no -b is specified then program warns the input is invalid and uses the first begin date available in the archive.

If -b is specified and is before the first available date in the archive then the program uses the first available date in the archive.

If -e is specified and is after the last available date in the archive then the program uses the last available date in the archive.

Example. In this example your archive contains 17 hours worth of data in the oswvmstat directory. You want to only analyze the time beginning Jan 9 13:15:00 2013 and ending Jan 19 13:30:00 2013

$ls oswvmstat coesrv40.us.oracle.com_vmstat_13.01.09.1300.dat coesrv40.us.oracle.com_vmstat_13.01.09.1400.dat coesrv40.us.oracle.com_vmstat_13.01.09.1500.dat coesrv40.us.oracle.com_vmstat_13.01.09.1600.dat coesrv40.us.oracle.com_vmstat_13.01.09.1700.dat coesrv40.us.oracle.com_vmstat_13.01.09.1800.dat coesrv40.us.oracle.com_vmstat_13.01.09.1900.dat coesrv40.us.oracle.com_vmstat_13.01.09.2000.dat coesrv40.us.oracle.com_vmstat_13.01.09.2100.dat coesrv40.us.oracle.com_vmstat_13.01.09.2200.dat coesrv40.us.oracle.com_vmstat_13.01.09.2300.dat coesrv40.us.oracle.com_vmstat_13.01.10.0000.dat coesrv40.us.oracle.com_vmstat_13.01.10.0100.dat coesrv40.us.oracle.com_vmstat_13.01.10.0200.dat coesrv40.us.oracle.com_vmstat_13.01.10.0300.dat coesrv40.us.oracle.com_vmstat_13.01.10.0400.dat coesrv40.us.oracle.com_vmstat_13.01.10.0500.dat java -jar oswbba.jar -i archive -b Jan 9 13:15:00 2013 -e Jan 19 13:30:00 2013 -A

Back to Contents

Limiting the Analysis to a Subset of Files in Your Archive (Old method)

Previously if a user wanted to analyze only a specific time window all of the files in the archive would have to be parsed first. Starting in release 6.0, the analysis can be limited to a subset of files. This can be controlled by using the command line options -START and -STOP. These options are available if you run from the command line or if you want to use the menu. In either case, you must use the -START and -STOP option along with the -s option (to tell the analyzer you want to analyze only a subset of available files.

To use this option you must specify the starting filename in the oswvmstat directory. oswbba only analyzes files in chronological order. To specify the starting filename locate a filename in the oswvmstat directory and specify it using the -START option. The -STOP option specifies the last file in the oswvmstat directory that you want to analyze. If these filenames are misspelled or cannot be found the analyzer will ignore the -START and -STOP options and analyze the entire contents of the archive. If you specify either parameter without specifying the other parameter the following default values will be assumed:

If -START is specified and no -STOP is specified then the analysis continues through the last file in the oswvmstat directory.

If -STOP is specified and no -START is specified then the analysis starts at the oldest file in the oswvmstat directory and continues through the file specified in the -STOP option.

You must also specify the -s option to tell the analyzer you want to analyze only a subset of available files. If the -s flag is NOT specified the analyzer will analyze the entire contents of the archive.

oswbba uses the times associated with these oswvmstat files for the other directories so that all iostat, top and netstat files will be analyzed during this time frame.

Example. In this example your archive contains 17 hours worth of data in the oswvmstat directory. You want to only analyze files starting at 13.01.09.1300 through 13.01.09.2000

$ls oswvmstat coesrv40.us.oracle.com_vmstat_13.01.09.1300.dat coesrv40.us.oracle.com_vmstat_13.01.09.1400.dat coesrv40.us.oracle.com_vmstat_13.01.09.1500.dat coesrv40.us.oracle.com_vmstat_13.01.09.1600.dat coesrv40.us.oracle.com_vmstat_13.01.09.1700.dat coesrv40.us.oracle.com_vmstat_13.01.09.1800.dat coesrv40.us.oracle.com_vmstat_13.01.09.1900.dat coesrv40.us.oracle.com_vmstat_13.01.09.2000.dat coesrv40.us.oracle.com_vmstat_13.01.09.2100.dat coesrv40.us.oracle.com_vmstat_13.01.09.2200.dat coesrv40.us.oracle.com_vmstat_13.01.09.2300.dat coesrv40.us.oracle.com_vmstat_13.01.10.0000.dat coesrv40.us.oracle.com_vmstat_13.01.10.0100.dat coesrv40.us.oracle.com_vmstat_13.01.10.0200.dat coesrv40.us.oracle.com_vmstat_13.01.10.0300.dat coesrv40.us.oracle.com_vmstat_13.01.10.0400.dat coesrv40.us.oracle.com_vmstat_13.01.10.0500.dat java -jar oswbba.jar -i archive -START coesrv40.us.oracle.com_vmstat_13.01.09.1300.dat -STOP coesrv40.us.oracle.com_vmstat_13.01.09.2000.dat -s

Back to Contents

Ignoring Specific Directories From Analysis

Because the archive directory can contain gigabytes worth of data, the analyzer can take some time to analyze extremely large files. The analyzer parses these files and stores the data into memory structures that grow in size as more files are parsed. This parsing will consume more and more memory potentially causing the analysis to take a longer time and may cause the maximum size specified by the jvm to be reached. In this case you will need to restart using a larger value for jvm (see the section named "Java Heap Errors on Startup" located in this document. This is normally only a problem if you have many days worth of data or you have thousands of processes on the machine making for large ps files. If you find this to be the case you can eliminate certain directories from being analyzed. You cannot however, eliminate vmstat data because this data is critical to any analysis performed by oswbba. There also may be times when resources on the machine may not be available to allow this kind of heavier processing. By eliminating the oswps directory, this more intensive data parsing and analysis can be eliminated or postponed until a later time. To eliminate specific directories from the analysis you must specify the appropriate flag when running oswbba. Flags are as follows:

OPTIONS The following options are supported:
-NO_IOSTAT | <filename> Ignores files in the oswiostat directory from analysis.
-NO_TOP | <filename> Ignores files in the oswtop directory from analysis.
-NO_PS | <filename> Ignores files in the oswps directory from analysis.
-NO_NETSTAT | <filename> Ignores files in the oswnetstat directory from analysis.
-NO_LINUX | <filename> Ignores files in the oswmeminfo directory from analysis.
-MEM_ALL | <filename>Analyzes virtual and resident memory allocations for all processes. This is very resource intensive.

An example of ignoring netstat and iostat data from analysis:

java -jar oswbba.jar -i archive -NO_NETSTAT -NO_IOSTAT

Back to Contents

Limiting the PS Analysis

You can control the amount of analysis done on the ps data with respect to process memory. The default now is to show only the top 10 processes which are increasing in memory consumption. In the case you want to see all memory allocations over time you can do this by including the -MEM_ALL parameter when analyzing the data from the command line option. This would be useful for example in the case you were experiencing a memory leak. Enabling this parameter however, will result in long analysis times as the ps files are normally large and all the data related to all processes is parsed, stored and analyzed. There may be times when resources on the machine may not be available to allow this kind of heavier processing. By limiting the memory analysis, this more intensive data parsing and analysis can be eliminated or postponed until a later time.

An example of analyzing all process information contained in the ps directory :

java -jar oswbba.jar -i archive -MEM_ALL

Back to Contents

Generating a Dashboard

Generating a responsive web page dashboard is a useful feature of oswbba. The dashboard allows for a view into the different subsystems (cpu, memory, i/o, network and also nfs (Linux only). Any problems can be quickly identified and displayed on the dashboard. The analysis based text report is integrated with the dashboard so both textual data and graphics can be accessed inside the same browser window.

1. Select option (D) from them menu. This generates the dashboard 2. cd to the analysis directory 3. Locate the analysis directory generated in step 1 4. cd into the specific analysis directory 5. Inside the specific analysis directory there will be a. single analysis.txt file b. directory named dashboard 6. cd into the dashboard directory 7. Inside the dashboard directory there will be a. index.html (This is the dashboard). Open this file with a browser b. css directory containing cascading style sheets used by the dashboard c. fonts directory containing fonts used by the dashboard d. generated_files directory containing gifs that were generated and used by the dashboard e. images directory containing misc images used by the dashboard f. js directory containing javascript used by the dashboard

https://mosemp.us.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=206049425975944&parent=DOCUMENT&sourceId=301137.1&id=461053.1&_afrWindowMode=0&_adf.ctrl-state=baoj3tz0s_167

Known Issues

Because oswbba builds graphs based on the Unix operating system date function, the time stamp must be in Standard English LANG format. The time stamp is formatted automatically by default (setting the parameter oswgCompliance = 1) in the OSWatcher.sh file.

Back to Contents

Reporting Feedback

If you encounter problems running oswbba which are not listed under the Known Issue section or would like to provide comments/feedback about oswbba (including enhancement requests) please send email to carl.davis@oracle.com

Legal Notices and Terms of Use