• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

The Geek Diary

CONCEPTS | BASICS | HOWTO

  • OS
    • Linux
    • CentOS/RHEL
    • Solaris
    • Oracle Linux
    • Linux Services
    • VCS
  • Database
    • oracle
    • oracle 12c
    • ASM
    • mysql
    • MariaDB
    • Data Guard
  • DevOps
    • Docker
    • Shell Scripting
  • Interview Questions
  • Big Data
    • Hadoop
    • Cloudera
    • Hortonworks HDP

HDPCA Exam Objective – Configure ResourceManager HA

By admin

Note: This is post is part of the HDPCA exam objective series

In a Hadoop cluster, if the RM goes offline for any reason, all the jobs on the cluster will fail. In production, there will be critical jobs that might be running for a long time and it does not make sense to start them again due to the failure of RM. HA for ResourceManager was introduced in Hadoop 2.4 and it supports both manual and automatic failover.

As the exam objective in HDPCA exam, we will see how to configure the ResourceManager HA using ambari in this post. Similar to Namenode HA discussed in the earlier recipes, ResourceManager HA also has only one active node at any given point of time. The failover is either initiated by an admin command or by using ZooKeeper for automatic failover.

Verify Current Configuration

Before enabling ResourceManager HA, let’s first verify the current configuration. Go to the Services > YARN. You would only see a single ResourceManager. In case of HA, you would see an extra Standby ResourceManager as well here.

Verify current configuration - ResourceManager HA HDPCA exam

Note: You should plan a cluster maintenance window while performing a ResourceManager HA.

Configuring ResourceManager HA using ambari wizard

To configure ResourceManager HA, goto Services > YARN. Click “Enable ResourceManager HA” under Service Actions drop-down.

configure ResourceManager HA using ambari

1. Enable ResourceManager HA Wizard

The first page is informational and informs us that we need a cluster level downtime to perform the HA. Also once the HA mode is enabled you would have an active-standby setup with another standby ResourceManager.

enable ResourceManager HA wizard HDPCA exam objective

2. Select Host

On the next page we will select the nn2 host as the additional ResourceManager.

select hosts for ResourceManager HA - HDPCA exam

3. Review

On the Review page, you can review the configuration. As we have selected nn2 as our additional ResourceManager, Ambari wizard will go ahead and install the required software on this node. The wizard will also take care of the properties to be modified in the configuration files.

review page - configuring ResourceManager HA using ambari

4. Configure components

The wizard will continue installing and configuring the additional ResourceManager. The detailed steps involved are:

  • Stop required services
  • Install additional ResourceManager
  • Reconfigure YARN
  • Reconfigure HDFS
  • Start all services

configure components - Resource Manager HA HDPCA exam objective

Once all the components are completed, click the complete button to proceed to the dashboard.

Verify

We can go to the Services > YARN page to verify the ResourceManager HA setup. As shown below, we can now see a Standby ResourceManager along with the active ResourceManager.

verify ResourceManager HA - HDPCA exam objective

You may need to restart a few component post the HA configuration for ResourceManager. For example, in our setup, we have to restart the HDFS service on all 5 nodes.

restart HDFS service after configuring ResourceManager HA using ambari

HDPCA Exam Objective – Configure NameNode HA

Filed Under: Hadoop, HDPCA, Hortonworks HDP

Some more articles you might also be interested in …

  1. CCA 131 – Install Cloudera Manager server and agents
  2. HDPCA Exam Objective – Install ambari agent
  3. CCA 131 – Add a new node to an existing cluster
  4. HDPCA Exam Objective – Install and configure Ranger
  5. HDPCA Exam Objective – Configure HiveServer2 HA ( Part 2 – Configure HA )
  6. HDPCA Exam Objective – Add an HDP service to a cluster using Ambari
  7. HDPCA Exam Objective – Configure HDFS ACLs
  8. HDPCA Exam Objective – Create a snapshot of an HDFS directory
  9. How to configure Capacity Scheduler Queues Using YARN Queue Manager
  10. HDPCA Exam Objective – Add a new node to an existing cluster

You May Also Like

Primary Sidebar

Recent Posts

  • How to set the default character set in MySQL and how to propagate it in a master-master replication scenario
  • “Connection reset by peer” – error while ssh into a CentOS/RHEL system with a specific user only
  • MySQL: how to figure out which session holds which table level or global read locks
  • Recommended Configuration of the MySQL Performance Schema
  • Archives
  • Contact Us
  • Copyright

© 2021 · The Geek Diary