Iostat is one of the most useful command when troubleshooting a disk I/O performance issue. It can also be used to troubleshoot a CPU performance issue as well. The iostat command can be run in various ways.
Summary since boot
By default if you run iostat without any options it will show the summary since boot which is very useful to compare the system performance in a bottleneck vs average system performance.
Example 1
# iostat tty sd0 sd1 sd2 sd3 cpu tin tout kps tps serv kps tps serv kps tps serv kps tps serv us sy wt id 0 5 18 1 19 0 0 0 0 0 0 0 0 0 0 1 0 99
The output shows all the device instances as well as terminal and cpu statistics. Below are the parameter being displayed :
kps - kilobytes per second. tps - transactions per second. serv - average service time.
tin - terminal in chars tout - terminal out chars
us - % user time sy - % system time wt - % I/O wait time (ignore this as it is no longer calculated & will always return 0) id - % Idle time
Current statistics
The current disk I/O statistics need to be analyzed, which can be done by running iostat over some period of time. The syntax to run iostat at some interval with some optional count is :
# iostat [interval] [count]
Example 2
To run iostat at 1 seconds interval with a count of 5 :
# iostat 1 5 tty sd0 sd1 sd2 sd3 cpu tin tout kps tps serv kps tps serv kps tps serv kps tps serv us sy wt id 0 6 17 1 18 0 0 0 0 0 0 0 0 0 0 1 0 99 0 235 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 99 0 80 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 100 0 81 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 99 0 80 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 99
Display Only disk statistics
Classic style
The default iostat also shows the CPU statistics. To display only the disk statistics use the -d otion. This would print disk statistics in a classic way with kps, tps and serv values.
Example 3
# iostat -d 1 3
sd0 sd1 sd2 sd3
kps tps serv kps tps serv kps tps serv kps tps serv
16 0 18 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
I actually have 5 disks in my system, but the above output only shows disk statistics for 4 disk only. That’s because the default iostat command shows disk statistics only for first 4 disks. To print disk statistics for more than 4 disk we can use -l option ( we can provide number of disks for which we want disk statistics with l )
Example 4
# iostat -dl 5 1 3
sd0 sd1 sd2 sd3 sd4
kps tps serv kps tps serv kps tps serv kps tps serv kps tps serv
16 0 18 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
In the example above 5 is number of disk to print the statistics for and (1 3) is the interval and count respectively.
New style
In the latest style of iostat command with option -D, we get rps (reads per second), wps (write per second), and util (% utilization).
Example 5
# iostat -D 1 3
sd0 sd1 sd2 sd3
rps wps util rps wps util rps wps util rps wps util
0 0 0.3 0 0 0.0 0 0 0.0 0 0 0.0
0 0 0.0 0 0 0.0 0 0 0.0 0 0 0.0
0 0 0.0 0 0 0.0 0 0 0.0 0 0 0.0
Statistics for a specific disk(s)
We can also get disk I/O statistics only for a specific disk or set of disk using iostat :
Example 6
# iostat -dl 2 sd0 sd4 1 3
sd0 sd4
kps tps serv kps tps serv
16 0 18 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0
Remember to limit the disks shown in output with l, otherwise it would print all disk anyways.
To print logical disk names
Most of the times we understand logical disk names. So to print disk I/O statistics using the logical disk names use the option -n :
Example 7
# iostat -n tty c8t0d0 c8t1d0 c7t0d0 c8t2d0 cpu tin tout kps tps serv kps tps serv kps tps serv kps tps serv us sy wt id 0 5 16 0 18 0 0 0 0 0 0 0 0 0 0 1 0 99
Extended Device statistics
The option -x displays the extended device statistics.
Example 8
# iostat -x
extended device statistics
device r/s w/s kr/s kw/s wait actv svc_t %w %b
sd0 0.2 0.3 6.9 9.1 0.0 0.0 18.5 0 0
sd1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
sd2 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0 0
sd3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
sd4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
sd5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
The extended statistics shows some extra info such as reads/sec (r/s), writes/sec (w/s), kilobytes read and kilobytes write per sec (kr/s and kw/s), avg. active transactions (actv), average service time (svc_t) etc.
Per partition statistics
with the -p option we can also print the per partition io statistics if we want.
Example 9
# iostat -xpn 1 10
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.2 0.3 6.8 9.0 0.0 0.0 0.0 18.5 0 0 c8t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c8t0d0s0
0.2 0.3 6.8 9.0 0.0 0.0 0.0 19.9 0 0 c8t0d0s1
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c8t0d0s8
Error Statistics
To check for hard and soft errors on disk use the -e option with iostat command :
Example 10
# iostat -en
---- errors ---
s/w h/w trn tot device
0 0 0 0 c8t0d0
0 0 0 0 c8t1d0
0 0 0 0 c7t0d0
0 0 0 0 c8t2d0
0 0 0 0 c8t3d0
0 0 0 0 c8t4d0
The various errors are :
Hard error : Re-read fails several times for CRC check
Transport error : Errors reported by I/O bus
Total errors : Soft error + Hard error + Transport errors
The various errors statistics can also be displayed with -E option :
Example 11
# iostat -E sd0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 Vendor: VMware, Product: VMware Virtual S Revision: 1.0 Serial No: Size: 10.74GB [10737418240 bytes] Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 Illegal Request: 9 Predictive Failure Analysis: 0 sd1 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0 Vendor: VMware, Product: VMware Virtual S Revision: 1.0 Serial No: Size: 24.70GB [24696061952 bytes] Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0 Illegal Request: 6 Predictive Failure Analysis: 0
Combining multiple options
Now when using iostat, we usually combine 3 or 4 options to give you only the output in which we are interested. As show in below example we would only get non-zero lines, with logical disk name, extended statistics etc by combining various options in iostat example.
Example 12
# iostat -xntcz 1 3
tty cpu
tin tout us sy wt id
0 5 0 1 0 99
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.1 0.3 6.1 17.3 0.0 0.0 0.0 20.8 0 0 c8t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0 0 c7t0d0
tty cpu
tin tout us sy wt id
0 313 0 62 0 38
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 54.7 0.0 50579.8 0.0 5.7 0.2 104.5 1 68 c8t0d0
tty cpu
tin tout us sy wt id
0 288 0 35 0 65
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
0.0 164.4 0.0 36788.5 0.0 5.1 0.0 31.0 0 68 c8t0d0