The Geek Diary

How to troubleshoot NFS transfer latency issues using “nfsiostat” in CentOS / RHEL

by

The nfsiostat command works like the iostat command except only for the NFS mount points. The nfsiostat gets input from /proc/self/mountstats and provides information about the input/output performance of NFS shares mounted in the system. The nfsiostat command is provided by the nfs-utils package.

Understanding the nfsiostat command output (Examples Included)

Below is a sample output from the nfsiostat command.

nfsiostat command output linux

Understanding the different fields of the nfsiostat command output

Below are few of the fields you can look into to investigate NFS congestion issues.

  • The op/s value is the total number of operations (RPCs) of all types sent per second. This not only includes I/O RPCs such as reads and writes, but all NFS RPCs such as open, lookup, access, getattr, etc.
  • NFS doesn’t have a blocksize as its I/O is not based around sectors or blocks like a standard disk filesystem. All the I/O information is being displayed in kilobytes. e.g. kB/op shows the average size of each RPC command of the given type in kilobytes. The size mount options can influence the size of certain I/O operations, but do not function as a “blocksize” which I/O transactions must be a multiple of.
  • RTT: This is the time from when the kernel RPC client sends the RPC request to the time it receives the reply. RTT includes network transit time and server execution time.
  • exe: This is the time from when the NFS client dispatches the RPC request to the kernel RPC client to the time the RPC request is complete. “exe” includes RTT, and RPC client queuing and processing time. Average execution time “exe” goes up if the requests have to wait in the backlog queue.

Using avg RTT (Round Trip Time) field to determine NFS latency

The average Round Trip Time (avg RTT) in milliseconds is a good measurement for NFS latency. In below example, The avg RTT (Round Trip Time) column is the average latency of the connection in ms. 

# nfsiostat 5 10 /data

From the above command output, the average round trip time is ###. Higher the value of avg RTT value higher is the latency. You can confirm the latency from this field value being very high.

You May Also Like