• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer navigation

The Geek Diary

  • OS
    • Linux
    • CentOS/RHEL
    • Solaris
    • Oracle Linux
    • VCS
  • Interview Questions
  • Database
    • oracle
    • oracle 12c
    • ASM
    • mysql
    • MariaDB
  • DevOps
    • Docker
    • Shell Scripting
  • Big Data
    • Hadoop
    • Cloudera
    • Hortonworks HDP

How to Configure a Fiber-Channel Over Ethernet (FCoE) Interface in CentOS/RHEL

by admin

The Setup

This configuration uses below OS, Storage and Server Hardware –

OS release – “RHEL 7.1”
Kernel – 3.8.13-55.1.6.el7.x86_64
Hardware: HP ProLiant BL460c Gen8
Ethernet Interfaces with FCoE capability –

04:00.0 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.1 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.2 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.3 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.4 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.5 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)

Storage – HP 3PAR StoreServ Storage server (FCoE target).
Switch – FCoE-enabled switch.

The Steps

Follow the steps outlined below to configure the FCoE interface in a CentOS/RHEL system.

1. Configure FCoE using system BIOS – Consult your Server HW documentation for instructions of how to enable FCoE on available interfaces.

2. Install the required packages:

# yum install fcoe-utils lldpad

3. Find out the interfaces configured for FCoE – In this example its eno3 (6C:C2:17:2B:4D:11) and eno4 (6C:C2:17:2B:4D:19).

4. Configure Ethernet interface to use FCoE – bring down the interfaces and fcoe service if running:

# ifconfig eno3 down
# ifconfig eno4 down

and

# systemctl stop fcoe      ### if running

5. Configure a new VLAN by copying an existing network script i.e. vi /etc/fcoe/cfg-eth0 to the name of the Ethernet device that supports FCoE. The configuration file /etc/fcoe/cfg-eth0 will give you the default settings to configure. Given that the FCoE device is ethX, run:

# cp /etc/fcoe/cfg-eth0 /etc/fcoe/cfg-ethX

Modify the FCoE config file as follows:

# cp /etc/fcoe/cfg-eth0 /etc/fcoe/cfg-eno3
# cp /etc/fcoe/cfg-eth0 /etc/fcoe/cfg-eno4

6. Modify /etc/fcoe/cfg-[interface] by setting DCB_REQUIRED to “no“. DCB_REQUIRED should be set to no for networking interfaces that implement a hardware DCBX client. Here we are not using any hardware DCBX client. More information on Data Center Bridging can be read at – http://linux.die.net/man/8/fcoemon.

For example:

$ cat /etc/fcoe/cfg-eno3
## Type: yes/no
## Default: no
# Enable/Disable FCoE service at the Ethernet port
# Normally set to "yes"
FCOE_ENABLE="yes"

## Type: yes/no
## Default: no
# Indicate if DCB service is required at the Ethernet port
# Normally set to "yes"
DCB_REQUIRED="no"
#DCB_REQUIRED="yes"

## Type: yes/no
## Default: no
# Indicate if VLAN discovery should be handled by fcoemon
# Normally set to "yes"
AUTO_VLAN="yes"

## Type: fabric/vn2vn
## Default: fabric
# Indicate the mode of the FCoE operation, either fabric or vn2vn
# Normally set to "fabric"
MODE="fabric"

## Type: yes/no
## Default: no
# Indicate whether to run a FIP responder for VLAN discovery in vn2vn mode
#FIP_RESP="yes"

Do the same for eno4 interface.

7. Activate Interface on boot and enable Jumbo Frames – Set “ONBOOT=yes” in the corresponding /etc/sysconfig/network-scripts/ifcfg-eno3 and /etc/sysconfig/network-scripts/ifcfg-eno4 file to activate automatically during boot time. Also enable jumbo frames (MTU=9000) on the FCoE enabled interfaces. With jumbo frames enabled on the interfaces, it will avoid unnecessary IP fragmentation.

# cat /etc/sysconfig/network-scripts/ifcfg-eno3
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno3
UUID=fae9a6da-f05b-497e-b625-ba5826db0d78
DEVICE=eno3
#ONBOOT=no 
ONBOOT=yes
MTU=9000
# cat /etc/sysconfig/network-scripts/ifcfg-eno4
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno4
UUID=63157ec1-3013-4895-8ea4-079c5865d364
DEVICE=eno4
#ONBOOT=no 
ONBOOT=yes
MTU=9000

8. For interfaces that require a software DCBX client – Enable data center bridging on the Ethernet interface using the following commands:

# dcbtool sc ethX dcb on

Then, enable FCoE on the Ethernet interface by running:

# dcbtool sc ethX app:fcoe e:1

Enable Data Center Bridging on the Ethernet interface using the following commands:

# dcbtool sc eno3 dcb on
# dcbtool sc eno4 dcb on

Then, enable FCoE on the Ethernet interface by running:

# dcbtool sc eno3 app:fcoe e:1
# dcbtool sc eno4 app:fcoe e:1

9. Bring up interfaces and start fcoe service:

# ifconfig eno3 up
# ifconfig eno4 up

10. Start FCoE using:

# systemctl start fcoe

11. Start the data center bridging daemon (dcbd) using the following command:

# /etc/init.d/lldpad start

12. View configured FCoE devices:

# fcoeadm -i

FCoE configuration on CentOS RHEL 6 and 7

After correctly configuring the Ethernet interface to use FCoE, set “fcoe” and “lldpad” to run at startup. To do so, use chkconfig in CentOS/RHEL 6.x, as in:

# chkconfig lldpad on
# chkconfig fcoe on

Or in CentOS/RHEL 7.x:

# systemctl enable fcoe
# systemctl enable lldpad

Filed Under: CentOS/RHEL 6, CentOS/RHEL 7, Linux

Some more articles you might also be interested in …

  1. man Command Examples in Linux
  2. Understanding rsyslog Filter Options
  3. jpegoptim for image optimization
  4. How to verify if NX/XD is Enabled or Disabled in CentOS/RHEL 7 and 8
  5. Linux OS Service ‘psacct’
  6. How to monitor NVME drives on Centos 6
  7. How to run ssh on multiple ports
  8. ipcs Command Examples in Linux
  9. lnstat Command Examples in Linux
  10. logname Command Examples in Linux

You May Also Like

Primary Sidebar

Recent Posts

  • JavaFX ComboBox: Set a value to the combo box
  • Nginx load balancing
  • nginx 504 gateway time-out
  • Images preview with ngx_http_image_filter_module

© 2022 · The Geek Diary

  • Archives
  • Contact Us
  • Copyright