• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer navigation

The Geek Diary

  • OS
    • Linux
    • CentOS/RHEL
    • Solaris
    • Oracle Linux
    • VCS
  • Interview Questions
  • Database
    • oracle
    • oracle 12c
    • ASM
    • mysql
    • MariaDB
  • DevOps
    • Docker
    • Shell Scripting
  • Big Data
    • Hadoop
    • Cloudera
    • Hortonworks HDP

Unable to Start RDMA Services on CentOS/RHEL 7

by admin

The Problem

The RDMA service fails to start on CentOS/RHEL 7 machine. ‘journalctl -r’ command shows the below errors:

Oct 11 11:01:15 geeklab  systemd-modules-load: Failed to insert 'rds_rdma': Address family not supported by protocol
Oct 11 11:01:15 geeklab  systemd-modules-load: Failed to insert 'rds_rdma': Address family not supported by protocol
Oct 11 11:01:15 geeklab  systemd-modules-load: Failed to insert 'resilient_rdmaip': Address family not supported by protocol
Oct 11 11:01:15 geeklab  systemd-modules-load: Failed to insert 'resilient_rdmaip': Address family not supported by protocol
Oct 11 11:01:15 geeklab  systemd: rdma-load-modules@rdma.service: main process exited, code=exited, status=1/FAILURE
Oct 11 11:01:15 geeklab  systemd: rdma-load-modules@rdma.service: main process exited, code=exited, status=1/FAILURE
Oct 11 11:01:15 geeklab  systemd: Failed to start Load RDMA modules from /etc/rdma/modules/rdma.conf.
Oct 11 11:01:15 geeklab  systemd: Failed to start Load RDMA modules from /etc/rdma/modules/rdma.conf.
Oct 11 11:01:15 geeklab  systemd: Unit rdma-load-modules@rdma.service entered failed state.
Oct 11 11:01:15 geeklab  systemd: Unit rdma-load-modules@rdma.service entered failed state.

The Solution

The issue is caused by the following setup were the rds_rdma module was not being loaded to the kernel due to “ipv6.disable=1” parameter on /etc/default/grub/ and on /boot/grub2/grub.cfg

/etc/default/grub configuration file:

########### BEGIN DO NOT REMOVE Added by Oracle Exadata ###########
GRUB_DEFAULT=0
GRUB_DISABLE_OS_PROBER=true
GRUB_DISABLE_RECOVERY=true
GRUB_DISABLE_SUBMENU=y
GRUB_DISTRIBUTOR="Oracle Linux"
GRUB_TIMEOUT=5
GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL="console serial"
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"
export EXADATA_IMAGE_TYPE=BareMetal
GRUB_CMDLINE_LINUX_DEFAULT="root=LABEL=DBSYS bootarea=dbsys bootfrom=BOOT ro loglevel=7 panic=60 pci=noaer log_buf_len=1m nmi_watchdog=0 transparent_hugepage=never rd_NO_PLYMOUTH audit=1 console=tty1 console=ttyS0,115200n8 crashkernel=448M processor.max_cstate=1 clocksource=tsc nohpet nopmtimer hda=noprobe hdb=noprobe ide0=noprobe pci=nocrs ifnames_skip=100 biosdevname=0 net.ifnames=0 "
########### END DO NOT REMOVE Added by Oracle Exadata ###########

#OS hardening start
GRUB_CMDLINE_LINUX='ipv6.disable=1'
#OS hardening end

/boot/grub2/grub.cfg configuration file:

linux16 /vmlinuz-4.14.35-1902.301.1.el7uek.x86_64 root=/dev/mapper/VGExaDb-LVDbSys1 
ro ipv6.disable=1 root=LABEL=DBSYS bootarea=dbsys bootfrom=BOOT 
ro loglevel=7 panic=60 pci=noaer log_buf_len=1m nmi_watchdog=0 transparent_hugepage=never rd_NO_PLYMOUTH audit=1 
console=tty1 console=ttyS0,115200n8 crashkernel=448M processor.max_cstate=1 clocksource=tsc nohpet nopmtimer 
hda=noprobe hdb=noprobe ide0=noprobe pci=nocrs ifnames_skip=100 biosdevname=0 net.ifnames=0
  initrd16 /initramfs-4.14.35-1902.301.1.el7uek.x86_64.img

The RDMA is dependent to the ipv6 code to be available which is being prevented by this parameter “ipv6.disable=1”. In order to resolve the issue follow the steps outlined below:

1. Please back up the /etc/default/grub before any change. Please remove the following option (or line) from the /boot/grub2/grub.cfg boot command line:

# vi /boot/grub2/grub.cfg

Remove the below line:

ipv6.disable=1

It is because the RDMA depends on the IPv6 code to be available, but this prevents it from loading at all.

2. Please back up the /boot/grub2/grub.cfg. Please run the following command to rebuld the grub file.

# grub2-mkconfig -o /boot/grub2/grub.cfg

3. Please back up the current initramfs file. Then rebuild the initramfs.

# cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).bak.$(date +%m-%d-%H%M%S).img
# dracut -f

4. Please reboot the server to check the result.

Filed Under: CentOS/RHEL, CentOS/RHEL 7, Linux

Some more articles you might also be interested in …

  1. How to activate and mount 2 Volume groups with same names in CentOS/RHEL
  2. How to Disable/Enable services in Zimbra Mail Server
  3. How to Hot-add and Remove Logical Memory in CentOS/RHEL 7
  4. Oracle Database Environment Variables and Their Functions
  5. ncview: command not found
  6. Set Password for Single User Mode on Linux (CentOS/RHEL)
  7. How to Re-Create the Yum Cache and/or Force a Fetch of the Package List of the Enabled Repositories
  8. How to Trace Python Scripts using trace.py
  9. lvremove failing to remove volume after using ‘shred’ command
  10. How to Debug systemd boot process in CentOS/RHEL 7 and 8

You May Also Like

Primary Sidebar

Recent Posts

  • vgextend Command Examples in Linux
  • setpci command – configure PCI device
  • db_load command – generate db database
  • bsdtar command – Read and write tape archive files

© 2022 · The Geek Diary

  • Archives
  • Contact Us
  • Copyright