A system administrator must know some of the basic concepts of storage systems. In this post I will try to explain some of the concepts and useful commands related to identifying HBA cards, WWN numbers and fiber channel paths. Before we start with the commands let us get acquainted with some of the storage related terms.
Initiator : Initiator is typically a server with HBA card installed. The initiator initiates the connection over the fiber channel to the storage system.
Target : Target is typically the storage system itself. Target servers the requests from the initiators for the data.
WWN : WWN stands for the world wide name. Similar to a network card having a unique MAC address, every HBA port will have a unique WWN number.
Solaris 10 and Solaris 11
We can get a list of HBA ports using the luxadm command. Remember that luxadm command only work on hba’s that are bound to the Oracle drivers. So if native (non oracle) Emulex driver is in use then the luxadm command will not work. In that case use the fcinfo command.
# luxadm -e port /devices/ssm@0,0/pci@18,600000/SUNW,emlxs@1/fp@0,0:devctl CONNECTED /devices/ssm@0,0/pci@18,600000/SUNW,emlxs@1,1/fp@0,0:devctl CONNECTED /devices/ssm@0,0/pci@1c,600000/SUNW,emlxs@1/fp@0,0:devctl CONNECTED /devices/ssm@0,0/pci@1c,600000/SUNW,emlxs@1,1/fp@0,0:devctl CONNECTED
"CONNECTED" --> HBA has established connection with either target or initiator. "NOT CONNECTED" --> HBA has not established a connection with some other node or it is connected to a switch that has no target.
The fcinfo command give us more info on the HBA ports (model, manufactures etc) and the WWN numbers of the HBA ports.
# fcinfo hba-port HBA Port WWN: 23000800xxxxxxx OS Device Name: /dev/cfg/c1 Manufacturer: QLogic Corp. Model: 2200 Firmware Version: 2.01.145 FCode/BIOS Version: ISP2200 FC-AL Host Adapter Driver: 1.14 01/11/20 Serial Number: not available Driver Name: qlc Driver Version: 20090415-2.30 Type: L-port State: online Supported Speeds: 1Gb Current Speed: 1Gb Node WWN: 22000800xxxxxxx HBA Port WWN: 10000000cxxxxxxx OS Device Name: /dev/cfg/c5 Manufacturer: Emulex Model: LP10000DC-S Firmware Version: 1.92a1 (T2D1.92A1) FCode/BIOS Version: Boot:5.01a4 Fcode:1.50a4 Serial Number: BG52130223 Driver Name: emlxs Driver Version: 2.31t (2009.04.05.14.10) Type: N-port State: online Supported Speeds: 1Gb 2Gb Current Speed: 2Gb Node WWN: 20000000cxxxxxxx HBA Port WWN: 10000000cxxxxxxx OS Device Name: /dev/cfg/c6 Manufacturer: Emulex Model: LP10000DC-S Firmware Version: 1.92a1 (T2D1.92A1) FCode/BIOS Version: Boot:5.01a4 Fcode:1.50a4 Serial Number: BG52130223 Driver Name: emlxs Driver Version: 2.31t (2009.04.05.14.10) Type: N-port State: online Supported Speeds: 1Gb 2Gb Current Speed: 2Gb Node WWN: 20000000cxxxxxxx
Check the fields “Model:” and “Manufacturer:” in the output above. You can identify various manufactures of the HBA cards from these fields.
Node WWN - WWN of HBA itself Port WWN - WWN on a specific port of the HBA
LPxxxxxx-E are NON-Oracle HBAs usually resold by EMC.
LPxxxxxx-H are NON-Oracle HBAs usually resold by Hitachi.
LPxxxxxx-M are NON-Oracle HBAs usually resold by
375-xxxx-xx are Oracle Qlogic HBA Cards.
QLAxxxx may or may not be Oracle Qlogic HBA Cards so you would need to use “prtpicl Command:” to determine.
Model: 2200 with Manufacturer: QLogic Corp is a old Oracle Fibre Channel Arbitrated Loop (FC-AL) HBA Cards.
For Solaris 8 and 9
The fcinfo command may not work in Solaris 8 and 9. So we’ll have to use the below command. but the command would also show other devices such as network cards. look for the driver-name to identify the HBA cards.
# prtpicl -v | egrep "subsystem-id|model|devfs-path|driver-name|binding-name" | awk '/subsystem-id/,/binding-name/' | sed '/subsystem-id/{x;p;x;}' | nawk -F: '{ print $2 }' subsystem-id 0x13a1 model LSI,1064 devfs-path /pci@0,600000/pci@0/pci@8/pci@0/scsi@1 driver-name mpt binding-name pci1000,50 subsystem-id 0x1648 devfs-path /pci@0,600000/pci@0/pci@8/pci@0/network@2 driver-name bge binding-name pci14e4,1648 subsystem-id 0x1648 devfs-path /pci@0,600000/pci@0/pci@8/pci@0/network@2,1 driver-name bge binding-name pci14e4,1648 subsystem-id 0xfe00 model LPe11000-M4 devfs-path /pci@0,600000/pci@0/pci@9/fibre-channel@0 driver-name emlxs binding-name pciex10df,fe00 .........
Some of the old HBAs may not display the “subsystem-id” in the above comman output. In that case use the below commands :
# prtpicl -v | egrep -i "qla|lpfc" # prtpicl -v | grep -i fibre-channel
For Example
# prtpicl -v | grep -i fibre-channel fibre-channel (obp-device, 4b300000160e) fibre-channel (obp-device, 4b30000016be) fibre-channel (obp-device, 4b3000001ca2) fibre-channel (obp-device, 4b3000001d52)
Each listing in the above output is a HBA port. To list the paths grep for the “devfs-path” string.
# prtpicl -v | grep -i devfs-path | grep -i fibre-channel | grep -v sd | nawk -F: '{ print $2 } devfs-path /pci@8,700000/fibre-channel@2 devfs-path /pci@8,700000/fibre-channel@3 devfs-path /pci@9,700000/fibre-channel@2 devfs-path /pci@9,700000/fibre-channel@3
Using the prtdiag command
In case if the prtpicl command also doee not work, use the prtdiag command to identify the HBA cards. prtdiag may be very helpful in case of a Solaris x86/x64 system.
# prtdiag -v ...... ========================= IO Cards ========================= Bus Max IO Port Bus Freq Bus Dev, Brd Type ID Side Slot MHz Freq Func State Name Model ---- ---- ---- ---- ---- ---- ---- ---- ----- -------------------------------- ---------------------- I/O PCI 8 B 4 33 33 1,0 ok pci-pci1011,26.5/SUNW,qlc (scsi-+ PCI-BRIDGE I/O PCI 8 B 4 33 33 4,0 ok SUNW,qlc-pci1077,2200.1077.4083.+ device on pci-bridge I/O PCI 8 A 1 66 66 1,0 ok SUNW,emlxs-pci10df,fc00/fp (fp) LP10000DC-S I/O PCI 8 A 1 66 66 1,1 ok SUNW,emlxs-pci10df,fc00/fp (fp) LP10000DC-S .........
Gathering information on target port WWN
To get information on target port WWN use the “remote-port -p” option with fcinfo. The syntax for the same is :
# fcinfo remote-port -p [HBA Port WWN at server end (initiator)]
For example
# fcinfo remote-port -p 10000000c993b528 Remote Port WWN: 5006016239a01db6 Active FC4 Types: SCSI SCSI Target: yes Node WWN: 50060160b9a01db6 Remote Port WWN: 5006016b3b205594 Active FC4 Types: SCSI SCSI Target: yes Node WWN: 50060160bb205594
Another way to get the target port WWNs is using the luxadm command with the dump_map option :
# luxadm -e dump_map /devices/pci@8,700000/SUNW,qlc@3/fp@0,0:devctl Pos Port_ID Hard_Addr Port WWN Node WWN Type 0 664f13 0 50060e800428a551 50060e800428a551 0x0 (Disk device) 1 610b13 0 210000e08b1c4dd6 200000e08b1c4dd6 0x0 (Disk device)
Port_ID ending with “00” or “13” indicates a FC switch attached storage.
Getting LUN list from a specific storage port
We can also get a list LUNs coming from a specific storage port (i.e. target port). The syntax for getting this details :
# fcinfo remote-port -s -p 10000000c993b528 Remote Port WWN: 5006016239a01db6 Active FC4 Types: SCSI SCSI Target: yes Node WWN: 50060160b9a01db6 LUN: 0 Vendor: DGC Product: LUNZ OS Device Name: /dev/rdsk/c2t5006016239A01DB6d0s2 LUN: 1 Vendor: DGC Product: RAID 5 OS Device Name: /dev/rdsk/c3t60060160545D1C00C0369BCD3361DC11d0s2 LUN: 33 Vendor: DGC Product: RAID 5 OS Device Name: /dev/rdsk/c3t60060160545D1C00B87CDAF33B44DC11d0s2