nmon a fundamental tool for the Performance Engineer

To check and track the system usage resources in terms of CPU, memory, I/O, kernel, Disk, Network, it is fundamental to have installed on all servers the nmon tool.  It was an internal project at IBM for many years. On AIX is nmon is a native command. For Linux has been released to open source under GPL.

For Linux you can download here (and untar) from or directly with this command:

#sudo apt-get install nmon 

Approach:
The approach of a performance engineer is to track the resource consumption with nmon for the entire time window of the execution of the performance test . So start the nmon on all servers involved on the test. If the test runs for one hour I usually run the command with these options:

Linux:
#nmon -lfT -s 10 -c 360           <collects for 1 hour>

AIX:
#nmon -lMPT -fT -s 10 -c 360      <collects for 1 hour>

Linux options:

-l <dpl>  disks/line default 150 to avoid spreadsheet issues. EMC=64.
-T           as -t plus saves command line arguments in UARG section
-f            spreadsheet output format

-s <seconds>  between snap shots
-c <number>   of refreshes

AIX options :
-l Specifies the number of disks to be listed on each line. By default, 150 disks are listed per line. For EMC disks, specify a value of 64.
-M Includes the MEMPAGES section in the recording file. The MEMPAGES section displays detailed memory statistics per page size.
-P Includes the Paging Space section in the recording file.
-T Includes the top processes in the output and saves the command-line arguments into the UARG section. You cannot specify the -t, -T, or -Y flags with each other.

-f Specifies that the output is in spreadsheet format. By default, the command takes 288 snapshots of system data with an interval of 300 seconds between each snapshot. The name of the output file is in the format of hostname_YYMMDD_HHMM.nmon.

-s Specifies the interval in seconds between 2 consecutive recording snapshots.
-c Specifies the number snapshots that must be taken by the command. The default value is 10000000.

Analyze the output:
After recording the nmon samples you can display using:

• nmon analyser excel spreadsheet download here

• java application nmonvisualizer download here 

Documentation:

• Linux nmon doc
• AIX nmon doc

Some Useful articles:

• https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/nmon

• https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/AIX%20performance%20analysis%20and%20tuning