• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer navigation

The Geek Diary

  • OS
    • Linux
    • CentOS/RHEL
    • Solaris
    • Oracle Linux
    • VCS
  • Interview Questions
  • Database
    • oracle
    • oracle 12c
    • ASM
    • mysql
    • MariaDB
  • DevOps
    • Docker
    • Shell Scripting
  • Big Data
    • Hadoop
    • Cloudera
    • Hortonworks HDP

How to run Hadoop without using SSH

by admin

The start-all.sh and stop-all.sh scripts in the hadoop/bin directory will use SSH to launch some of the Hadoop daemons. If for some reason SSH is not available on the server, please follow the steps below to run Hadoop without using SSH.

The goal is to modify all “hadoop-daemons.sh” with “hadoop-daemon.sh“. The “hadoop-daemons.sh” simply runs “hadoop-daemon.sh” through SSH.

1. Modify start-dfs.sh script:

from:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR start namenode $nameStartOpt
${bin}/hadoop-daemons.sh --config $HADOOP_CONF_DIR start datanode $dataStartOpt
${bin}/hadoop-daemons.sh --config $HADOOP_CONF_DIR --hosts masters start secondarynamenode

to:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR start namenode $nameStartOpt
${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR start datanode $dataStartOpt
${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR --hosts masters start secondarynamenode

2. Modify stop-dfs.sh script:

from:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR stop namenode
${bin}/hadoop-daemons.sh --config $HADOOP_CONF_DIR stop datanode
${bin}/hadoop-daemons.sh --config $HADOOP_CONF_DIR --hosts masters stop secondarynamenode

to:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR stop namenode
${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR stop datanode
${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR --hosts masters stop secondarynamenode

3. Modify start-mapred.sh script:

from:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR start jobtracker
${bin}/hadoop-daemons.sh --config $HADOOP_CONF_DIR start tasktracker

to:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR start jobtracker
${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR start tasktracker

4. Modify stop-mapred.sh script:

from:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR stop jobtracker
${bin}/hadoop-daemons.sh --config $HADOOP_CONF_DIR stop tasktracker

to:

${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR stop jobtracker
${bin}/hadoop-daemon.sh --config $HADOOP_CONF_DIR stop tasktracker

Note that after this change, start-all.sh and stop-all.sh will not start/stop any other Hadoop nodes outside of this server remotely. All other remote slaves must be started/stopped manually, directly on those servers.

Filed Under: Hadoop

Some more articles you might also be interested in …

  1. CCA 131 – Add a service using Cloudera Manager
  2. How to configure Capacity Scheduler Queues Using YARN Queue Manager
  3. HDPCA Exam Objective – Configure HiveServer2 HA ( Part 2 – Configure HA )
  4. HDPCA Exam Objective – Add an HDP service to a cluster using Ambari
  5. HDPCA Exam Objective – Configure the Capacity Scheduler
  6. HDPCA Exam Objective – Configure and manage alerts
  7. HDPCA Exam Objective – Recover a snapshot
  8. Converting Many Small Files To A Sequence File In HDFS
  9. CCA 131 – Install Cloudera Manager server and agents
  10. HDPCA Exam Objective – Install and configure Ranger

You May Also Like

Primary Sidebar

Recent Posts

  • powertop Command Examples in Linux
  • powertop: command not found
  • powerstat: command not found
  • powerstat Command Examples in Linux

© 2023 · The Geek Diary

  • Archives
  • Contact Us
  • Copyright