In my opinion one of the hardest thing to do by any system admin is to pin point the exact cause of performance bottleneck. Many of us struggle to do this. This post will help to get started with some basic performance monitoring and troubleshooting. The post contains some of the excellent videos by Brenden Gregg. Gregg is one of the leading experts on DTrace and creator of the DTraceToolkit.
1. uptime : Load averages
The uptime command gives us the load averages along with the system uptime information. The uptime command gives the 1 minute, 5 minute and 15 minute load averages which signifies how much is the overall CPU utilization.
Example:
# uptime 11:57pm up 44 mins(s), 2 users, load average: 0.08, 0.13, 0.14
2. mpstat : Key fields
The mpstat command is used to check how the load is balanced across CPUs and what is the load on each CPU. The video explains about the key fields in the output of mpstat and how to interpret them to analyze a performance issue.
3. mpstat : All the fields
The 2nd video in the mpstat series goes through the remaining fields in the mpstat command output.
4. mpstat : Digging deeper
The 3rd part in the mpstat series goes deeper in the analysis of the mpstat command output and how to use it with dtrace to dig deeper into a performance issue.
5. vmstat : Key fields
The vmstat command is used to report virtual memory statistics and getting information about CPU load, paging and system interrupts. The video discusses the key fields in the about of vmstat command and how to interpret them.
6. vmstat : All the fields
The 2nd part in the vmstat series goes through all the fields in the vmstat command output and how to analyze them to troubleshoot a performance bottleneck.
7. vmstat : Scope
The 3rd part in the vmstat series diggs deeper into the virtual memory statistics and troubleshooting virtual memory bottlenecks along with some other tools like prstat.
8. sar and prstat
Unfortunately, Brenden does not have a video explicitly on the usage of sar and prstat for performance monitoring and troubleshooting. But he has well explained it, how to use them along with the other tools discussed above.
Here is an another video by Gabriel Smith that I would like to share on how to do basic performance troubleshooting in Solaris.
I hope the post will help you guys with getting started with performance troubleshooting on a solaris machine. Do comment me back if you have any other good references which I can include in the post.