Archives for July 2018

Note: This is post is part of the HDPCA exam objective series Hive first started with HiveServer1. However, this version of the Hive server was not very stable. It sometimes suspended or blocked clients' connection quietly. Since version 11, Hive includes a new Hive server called HiveSever2 as an addition to HiveServer1. HiveServer2 is an enhanced Hive server designed for multiclient concurrency and improved authentication. HiveServer2 also supports Beeline as the alternative command-line … [Read more...] about HDPCA Exam Objective – Configure HiveServer2 HA ( Part 2 – Configure HA )

Note: This is post is part of the HDPCA exam objective series Capacity Scheduler is mainly designed for multitenancy, where multiple organizations collectively fund the cluster based on the computing needs. There is an added benefit that an organization can access any excess capacity not being used by others. This provides elasticity for the organizations in a cost-effective manner. In the previous post, we have seen installing and configuring capacity scheduler. The fundamental unit of … [Read more...] about How to configure Capacity Scheduler Queues Using YARN Queue Manager

Note: This is post is part of the HDPCA exam objective series Apache Ranger is an application that enables data architects to implement security policies on a big data ecosystem. The goal of this project is to provide a unified way for all Hadoop applications to adhere to the security guidelines that are defined. Here are some of the features of Apache Ranger: Centralized administration Fine grained authorization Standardized authorization Multiple authorization methods Centralized … [Read more...] about How to Create HDFS policies in Ranger

Note: This is post is part of the HDPCA exam objective series Apache Ranger is a framework for enabling, monitoring, and managing the comprehensive data security across the Hadoop platform. Ranger simply helps a Hadoop admin with various security management tasks. It provides a mechanism to manage the security from a single pane for various components. With Ranger, you can control fine-grained access to various components of the Hadoop ecosystem. Ranger has an Administration Portal you can … [Read more...] about How to Configure Hive Authorization Using Apache Ranger

Note: This is post is part of the HDPCA exam objective series It is important to configure high availability in production so that if one of the hiveserver2 fails, the others can respond to client requests. This can be achieved by using the ZooKeeper discovery mechanism to point the clients to the active Hive servers. Adding HiveServer2 Before configuring the HiveServer2 HA, lets first add the HiveServer2 service using ambari. Pre requsites We need to have a database named hive … [Read more...] about HDPCA Exam Objective – Configure HiveServer2 HA ( Part 1 – Installing HiveServer )

Note: This is post is part of the HDPCA exam objective series It is an integral part of Haddop administration to troubleshoot running or failed jobs. In order to troubleshoot a running/failed job, we must view the application’s log file. This post focuses on the HDPCA exam objective "View an application’s log file". We will run a sample map reduce program and view the status of the program using the command line and ResourceManager UI. Running an Example job The HDP installation comes … [Read more...] about HDPCA Exam Objective – View an application’s log file (Troubleshoot a failed job)