Hadoop Installation

How to Install a Single-Node Hadoop Cluster

By Praveen
Assumptions
1. Youre running 64/32-bit Windows
2. Your laptop has more than 2 GB of RAM
Download List
1. Download VMWarePlayer
2. Ubuntu 14.04LTS
Instructions to Install Hadoop
1.
2.
3.
4.
5.

Install VMWare Player

Create a new virtual machine
Point the installer disc image to the ISO file(Ubuntu) that you just download
User name should be hduser
Hard disk space 40GB Hard drive (more is better, but you want to leave some
for your Windows machine).
6. Customize hardware
a. Memory: 2GB RAM (more is better, but you want to leave some for your
Windows machine)
b. Processors: 2 (more is better, but you want to leave some for your
Windows mahicne)
7. Launch your virtual machine (all the instruction after that step will be
performed in Ubuntu)
8. Login to hduser
9. Open terminal window with Ctrl + Alt + T (you will use this keyboard shortcut
a lot)
10.Install java JDK7
a. Download the Java JDK for Linux (jdk-7u21-linux-x64.tar.gz)
b. Unzip the file
i. tar xvf <jdk>.tar.gz
c. Now move the JDK7 directory to /usr/lib
i. sudo mkdir p /usr/lib/jvm
ii. sudo mv jdk1.7.0_21 /usr/lib/jvm/jdk1.7.0
d. Now run
sudo update-alternatives --install /usr/bin/java java
/usr/lib/jvm/jdk1.7.0/bin/java 1
sudo update-alternatives --install /usr/bin/javac javac
/usr/lib/jvm/jdk1.7.0/bin/javac 1
sudo update-alternatives --install /usr/bin/javaws javaws
/usr/lib/jvm/jdk1.7.0/bin/javaws 1

Praveen

Page 1

Hadoop Installation
e. Correct the file ownership and the permissions of the executables: (run
all once Ctrl+C and Shift + Insert)
sudo chmod a+x /usr/bin/java
sudo chmod a+x /usr/bin/javac
sudo chmod a+x /usr/bin/javaws
sudo chown -R <username>:<username> /usr/lib/jvm/jdk1.7.0
f. Check the version of you new JDK 7 installation:
i. Java -version
11.Install SSH Server
a. sudo apt-get install openssh-client
b. sudo apt-get install openssh-server
12.Configure SSH
a. ssh-keygen t rsa P
b. cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
13.Disabling IPv6 Run the following command in the extended(Alt + F2)
terminal
a. gksudo gedit /etc/sysctl.conf
14.Add the following lines to the bottom of the file
a. # disable ipv6
b. net.ipv6.conf.all.disable_ipv6 = 1
c. net.ipv6.conf.default.disable_ipv6 = 1
d. net.ipv6.conf.lo.disable_ipv6 = 1
15.Save the file and close it
16.Restart you Ubuntu
17.Download Apache Hadoop-1.1.2 and store it in Downloads folder (Shift + Ctrl
+ Y)
18.Unzip the file (open the terminal)
cd Downloads
sudo tar xzf hadoop-1.1.2.tar.gz
cd /usr/local
sudo mv /home/<username>/Downloads/hadoop-1.1.2 hadoop
sudo chown -R <username>:<username> hadoop
sudo chmod 750 hadoop
19.Open your .bashrc file in the extended terminal (Alt +F2)
gksudo gedit .bashrc
20.Add the following lines to the bottom of the file
# Set JAVA_HOME (we will also configure JAVA_HOME directly for Hadoop
later on)
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0
# Add Hadoop bin/ directory to PATH
# Set Hadoop-related environment variables

Praveen

Page 2

Hadoop Installation
export HADOOP_HOME=/usr/local/hadoop
export PATH=$PATH:$HADOOP_HOME/bin

21.Save the .bashic file and close it

22.Run
gksudo gedit /usr/local/hadoop/conf/hadoop-env.sh
Add the following lines
#The java implementation to use. Required.
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0/
23.Save and close the file
24.In the terminal window, create a directory and set the required ownerships
and permissions
sudo mkdir p /app/hadoop/tmp
sudo chown R <username>:<username> /app/hadoop/tmp
sudo chmod 750 /app/hadoop/tmp
25.Run
gksudo gedit /usr/local/hadoop/conf/core-site.xml
a. Add the following between the <configuration> </configuration>
tags
<property>
<name>hadoop.tmp.dir</name>
<value>/app/hadoop/tmp</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:1234</value>
</property>

26.Save and close file

27.gksudo gedit /usr/local/hadoop/conf/mapred-site.xml
<property>
<name>mapred.job.tracker</name>
<value>localhost:4321</value>
</property>

28.Save and Close the file

29.gksudo gedit /usr/local/hadoop/conf/hdfs-site.xml
<property>
<name>dfs.replication</name>
<value>1</value>

</property>

30.Save and close the file

Praveen

Page 3

Hadoop Installation
31.Format the HDFS
a. /usr/local/hadoop/bin/hadoop namenode -format
32.Add an application with the following command
Press the Start button and type Startup Applications
a. Name -- Hadoop
b. Command /usr/local/hadoop/bin/start-all.sh
c. Comment Start hadoop
33.Restart Ubuntu and login
34.Type jps on terminal
35.sudo apt-get install openjdk-7-jdk
36.jps
a. Name Node
b. Data Node
c. Job Tracker
d. Task Tracker
e. Secondary Name Node.

Praveen

Page 4