What is Fiber Channel HBA Queue Depth
The queue depth indicates the number of I/O requests that are “in flight”, that have been requested but not yet been acknowledged to have been completed when communicating with a SAN storage. These requests can be configured per single Logical Unit Number (LUN) that is accessed or based on the HBA. The maximum queue depth describes the value of requests which is allowed to be in flight at maximum. The maximum queue depth setting can significantly influence the storage performance.
The maximum queue depth should be chosen carefully. Low values can lead to bad I/O performance. High values can also lead to bad performance, in having the SAN target not using caches/scheduling in an optimal way. Not only the queue depth of a single HBA, but the queue depth of all HBAs connected to a storage port on the SAN target influence each others performance. The vendor of the SAN target might have recommendations for the maximum queue depth to be used. Further relevant factors are:
- the number of HBA ports connected to the target
- the I/O pattern generated by applications running on the system
Changing the queue depth of Qlogic HBA
To know the current queue depth value qdepth, run the following command:
# cat /sys/module/qla2xxx/parameters/ql2xmaxqdepth 32
The above example returns “32”.
To set/change the qdepth value, sysfs can be used, it is not required to unload/load the module. The following changes the ql2xmaxqdepth:
# echo 16 > /sys/module/qla2xxx/parameters/ql2xmaxqdepth
To set the qdepth value in a way so they are configured also after reboots of the system change the value of the ql2xmaxqdepth option in the qla2xxx driver:
# modinfo qla2xxx | grep ql2xmaxqdepth parm: ql2xmaxqdepth:Maximum queue depth to set for each LUN. Default is 32. (int)
The following entry need to be added to /etc/modprobe.conf or a file in directory /etc/modprobe.d and the initrd image rebuilt to make the change permanent:
# vi /etc/modprobe.conf alias scsi_hostadapter1 qla2xxx options qla2xxx ql2xmaxqdepth=16
Changing the queue depth of Emulex HBA
For Emulex HBA no dynamic modification of the maximum queue depth is currently possible, it is required to unload/load the module with new parameters, or reboot the whole system. Emulex HBA support the following options to influence the queue depth:
# modinfo lpfc|grep queue_depth parm: lpfc_lun_queue_depth:Max number of FCP commands we can queue to a specific LUN (uint) parm: lpfc_hba_queue_depth:Max number of FCP commands we can queue to a lpfc HBA (uint)
These options can be used in /etc/modprobe.conf or a file in directory /etc/modprobe.d and the initrd image rebuilt.
The default value of lpfc could also be checked from /sys/module/lpfc/parameters/. But note that those values could not be modified (no “w” permission)
# ls -lrt /sys/module/lpfc/parameters/ -r--r--r--. 1 root root 4096 May 19 13:55 lpfc_hba_queue_depth -r--r--r--. 1 root root 4096 May 19 13:55 lpfc_lun_queue_depth -r--r--r--. 1 root root 4096 May 19 13:55 lpfc_tgt_queue_depth