• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer navigation

The Geek Diary

  • OS
    • Linux
    • CentOS/RHEL
    • Solaris
    • Oracle Linux
    • VCS
  • Interview Questions
  • Database
    • oracle
    • oracle 12c
    • ASM
    • mysql
    • MariaDB
  • DevOps
    • Docker
    • Shell Scripting
  • Big Data
    • Hadoop
    • Cloudera
    • Hortonworks HDP

HDPCA Exam Objective – Configure HiveServer2 HA ( Part 1 – Installing HiveServer )

by admin

Note: This is post is part of the HDPCA exam objective series

It is important to configure high availability in production so that if one of the hiveserver2 fails, the others can respond to client requests. This can be achieved by using the ZooKeeper discovery mechanism to point the clients to the active Hive servers.

Adding HiveServer2

Before configuring the HiveServer2 HA, lets first add the HiveServer2 service using ambari.

Pre requsites

We need to have a database named hive created before we start the service addition for Hive2. I will create a database using MariaDB. You can also create the database in Oracle, MySQL or Postgres if you want.

1. As the MariaDB Database Management System (DBMS) is not installed by default on RHEL/CentOS 7, we will start this recipe by installing the required packages.

# yum install -y mariadb mariadb-server

2. Enable the service to ensure the service starts at boot and start the mariadb service:

# systemctl enable mariadb
# systemctl start mariadb

3. Create the database “hive” using the root user.

# mysql -u root -p
MariaDB [(none)]> CREATE DATABASE hive;
Query OK, 1 row affected (0.04 sec)

4. Create a new user “hiveuser” and grant full priviledges to the hiveuser on the hive database objects.

MariaDB [(none)]> GRANT ALL ON hive.* TO 'hiveuser'@'localhost' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.01 sec)

5. Connect to the database using the new user to verify:

# mysql -u hiveuser -p

list the databases available:

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| hive               |
| test               |
+--------------------+
3 rows in set (0.04 sec)
How to install and configure MariaDB in CentOS / RHEL 7

Adding HiveServer service using Ambari

1. To add the HiveServer Service, goto Services page and select “Add Service” under the “Actions” drop-down.

Add HiveServer service using Ambari HDPCA

2. This will start the Add Service Wizard. Select the services “HiveServer”, “Tez”, “Pig” and “Slider”. The other 3 services are dependencies of the HiveServer and thus compulsary to be installed. Even if you do not select them the wizard will add it for you.

add service wizard HiveServer2 HA configuration HDPCA exam objective

3. On the “Assign Masters” page assign the roles “HiveServer2” and “Hive Metastore” to the cluster nodes. The role “WebHCat Server” is automatically assigned to the cluster node with “HiveServer2” role.

assign roles HiveServer2 Hive Metastore and WebHCat Server HDPCA exam

4. On the “Assign Slaves and Clients”, Select the client cluster nodes to nstall Tez Client, HCat Client, Hive Client, Pig Client and Slider Client.

assign slaves and clients HiveServer2 Service HA HDPCA exam

5. On the “Customize Services” page, we will have to provide the database login details which we have created as a part of prerequisites. Provide Below information:

  • Database Name : hive
  • Database Username : hiveuser
  • Database Password : password
  • Database URL : jdbc:mysql://ambari-server.localdomain/hive (I have created the MariaDB database on the ambari server )

We will have to download the MySQL Connector/JDBC driver from the MySQL site and install it in the Ambari server which hosts our MariaDB database.

https://dev.mysql.com/downloads/connector/j/
download mysql connector HDPCA HiveServer HA configuration

Select the appropriate OS and download and install the mysql connector in the ambari-server.

# yum install mysql-connector-java-8.0.12-1.el7.noarch.rpm

Once the MySQL connector is installed, execute the below command on the ambari-server:

# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java-8.0.12.jar
Using python  /usr/bin/python
Setup ambari-server
Copying /usr/share/java/mysql-connector-java-8.0.12.jar to /var/lib/ambari-server/resources
If you are updating existing jdbc driver jar for mysql with mysql-connector-java-8.0.12.jar. Please remove the old driver jar, from all hosts. Restarting services that need the driver, will automatically copy the new jar to the hosts.
JDBC driver was successfully initialized.
Ambari Server 'setup' completed successfully.

We can now test the connection to the MariaDB database and it should go through successfully.

customize service page HDPCA configure HiveServer2 HA

6. On the next page, ambari wizard will prompt us for some recommended property changes to be done for the HiveServer Install. I will keep the changes intact and proceed further.

recommended property settings HiveServer2 HA configuration HDPCA

7. Review the configuration before you begin the installation.

review configuration HiveServer2 HA

8. The installation will begin now. The wizard will install all the required components, start the services and even test the started services as a part of the installation.

install start and test HiveServer2 HA configuration HDPCA exam objective

9. Review the summary page and complete the installation. You will have to restart a few services such as HDFS, MapReduce2, YARN etc to complete the installation.

summary page HiveServer2 HA configuration HDPCA

Verify if all the components of the service are up and running.

verify HiveServer configuration HDPCA

HDPCA Exam Objective – Configure HiveServer2 HA ( Part 2 – Configure HA )

Filed Under: Hadoop, HDPCA, Hortonworks HDP

Some more articles you might also be interested in …

  1. CCA131 – Configuring HDFS snapshot Policy
  2. HDPCA Exam Objective – Configure a local HDP repository
  3. How to configure Capacity Scheduler Queues Using YARN Queue Manager
  4. CCA 131 – Configure HDFS ACLs
  5. HDPCA Exam Objective – Create a snapshot of an HDFS directory
  6. CCA 131 – Set up a local CDH repository
  7. CCA 131 – Create/restore a snapshot of an HDFS directory (Using Cloudera Manager)
  8. HDPCA Exam Objective – Configure HDFS ACLs
  9. HDPCA Exam Objective – Install HDP using the Ambari install wizard
  10. HDPCA Exam Objective – Define and deploy a rack topology script

You May Also Like

Primary Sidebar

Recent Posts

  • ncat Command Examples in Linux
  • ncat: command not found
  • nautilus Command Examples in Linux
  • namei: command not found

© 2023 · The Geek Diary

  • Archives
  • Contact Us
  • Copyright