• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer navigation

The Geek Diary

  • OS
    • Linux
    • CentOS/RHEL
    • Solaris
    • Oracle Linux
    • VCS
  • Interview Questions
  • Database
    • oracle
    • oracle 12c
    • ASM
    • mysql
    • MariaDB
  • DevOps
    • Docker
    • Shell Scripting
  • Big Data
    • Hadoop
    • Cloudera
    • Hortonworks HDP

How to Configure a Fiber-Channel Over Ethernet (FCoE) Interface in CentOS/RHEL

by admin

The Setup

This configuration uses below OS, Storage and Server Hardware –

OS release – “RHEL 7.1”
Kernel – 3.8.13-55.1.6.el7.x86_64
Hardware: HP ProLiant BL460c Gen8
Ethernet Interfaces with FCoE capability –

04:00.0 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.1 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.2 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.3 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.4 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)
04:00.5 Ethernet controller: Broadcom Corporation BCM57840 NetXtreme II 10/20-Gigabit Ethernet (rev 11)

Storage – HP 3PAR StoreServ Storage server (FCoE target).
Switch – FCoE-enabled switch.

The Steps

Follow the steps outlined below to configure the FCoE interface in a CentOS/RHEL system.

1. Configure FCoE using system BIOS – Consult your Server HW documentation for instructions of how to enable FCoE on available interfaces.

2. Install the required packages:

# yum install fcoe-utils lldpad

3. Find out the interfaces configured for FCoE – In this example its eno3 (6C:C2:17:2B:4D:11) and eno4 (6C:C2:17:2B:4D:19).

4. Configure Ethernet interface to use FCoE – bring down the interfaces and fcoe service if running:

# ifconfig eno3 down
# ifconfig eno4 down

and

# systemctl stop fcoe      ### if running

5. Configure a new VLAN by copying an existing network script i.e. vi /etc/fcoe/cfg-eth0 to the name of the Ethernet device that supports FCoE. The configuration file /etc/fcoe/cfg-eth0 will give you the default settings to configure. Given that the FCoE device is ethX, run:

# cp /etc/fcoe/cfg-eth0 /etc/fcoe/cfg-ethX

Modify the FCoE config file as follows:

# cp /etc/fcoe/cfg-eth0 /etc/fcoe/cfg-eno3
# cp /etc/fcoe/cfg-eth0 /etc/fcoe/cfg-eno4

6. Modify /etc/fcoe/cfg-[interface] by setting DCB_REQUIRED to “no“. DCB_REQUIRED should be set to no for networking interfaces that implement a hardware DCBX client. Here we are not using any hardware DCBX client. More information on Data Center Bridging can be read at – http://linux.die.net/man/8/fcoemon.

For example:

$ cat /etc/fcoe/cfg-eno3
## Type: yes/no
## Default: no
# Enable/Disable FCoE service at the Ethernet port
# Normally set to "yes"
FCOE_ENABLE="yes"

## Type: yes/no
## Default: no
# Indicate if DCB service is required at the Ethernet port
# Normally set to "yes"
DCB_REQUIRED="no"
#DCB_REQUIRED="yes"

## Type: yes/no
## Default: no
# Indicate if VLAN discovery should be handled by fcoemon
# Normally set to "yes"
AUTO_VLAN="yes"

## Type: fabric/vn2vn
## Default: fabric
# Indicate the mode of the FCoE operation, either fabric or vn2vn
# Normally set to "fabric"
MODE="fabric"

## Type: yes/no
## Default: no
# Indicate whether to run a FIP responder for VLAN discovery in vn2vn mode
#FIP_RESP="yes"

Do the same for eno4 interface.

7. Activate Interface on boot and enable Jumbo Frames – Set “ONBOOT=yes” in the corresponding /etc/sysconfig/network-scripts/ifcfg-eno3 and /etc/sysconfig/network-scripts/ifcfg-eno4 file to activate automatically during boot time. Also enable jumbo frames (MTU=9000) on the FCoE enabled interfaces. With jumbo frames enabled on the interfaces, it will avoid unnecessary IP fragmentation.

# cat /etc/sysconfig/network-scripts/ifcfg-eno3
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno3
UUID=fae9a6da-f05b-497e-b625-ba5826db0d78
DEVICE=eno3
#ONBOOT=no 
ONBOOT=yes
MTU=9000
# cat /etc/sysconfig/network-scripts/ifcfg-eno4
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno4
UUID=63157ec1-3013-4895-8ea4-079c5865d364
DEVICE=eno4
#ONBOOT=no 
ONBOOT=yes
MTU=9000

8. For interfaces that require a software DCBX client – Enable data center bridging on the Ethernet interface using the following commands:

# dcbtool sc ethX dcb on

Then, enable FCoE on the Ethernet interface by running:

# dcbtool sc ethX app:fcoe e:1

Enable Data Center Bridging on the Ethernet interface using the following commands:

# dcbtool sc eno3 dcb on
# dcbtool sc eno4 dcb on

Then, enable FCoE on the Ethernet interface by running:

# dcbtool sc eno3 app:fcoe e:1
# dcbtool sc eno4 app:fcoe e:1

9. Bring up interfaces and start fcoe service:

# ifconfig eno3 up
# ifconfig eno4 up

10. Start FCoE using:

# systemctl start fcoe

11. Start the data center bridging daemon (dcbd) using the following command:

# /etc/init.d/lldpad start

12. View configured FCoE devices:

# fcoeadm -i

FCoE configuration on CentOS RHEL 6 and 7

After correctly configuring the Ethernet interface to use FCoE, set “fcoe” and “lldpad” to run at startup. To do so, use chkconfig in CentOS/RHEL 6.x, as in:

# chkconfig lldpad on
# chkconfig fcoe on

Or in CentOS/RHEL 7.x:

# systemctl enable fcoe
# systemctl enable lldpad

Filed Under: CentOS/RHEL 6, CentOS/RHEL 7, Linux

Some more articles you might also be interested in …

  1. Understanding RPM Versions and Naming Schemes
  2. unzip: command not found
  3. autorandr: command not found
  4. authconfig: command not found
  5. CentOS/RHEL: Mount Command Fails With “mount: does not contain SELinux labels”
  6. ddcutil: command not found
  7. esearch: command not found
  8. 10 Sed (Stream Editor) Command Examples
  9. How to Create/Format/Extend Virtual Data Optimizer(VDO) Volumes using Cockpit Web Console in CentOS/RHEL 8
  10. How To Customize The Screensaver Options In Gnome on CentOS/RHEL 7

You May Also Like

Primary Sidebar

Recent Posts

  • netselect Command Examples in Linux
  • netselect: command not found
  • netselect-apt Command Examples in Linux
  • netselect-apt: command not found

© 2023 · The Geek Diary

  • Archives
  • Contact Us
  • Copyright