On RHEL-compatible systems:

1. Download the CDH 5 "1-click Install" package (or RPM).

   Click the appropriate RPM and Save File to a directory with write access (for example, your home directory).

   OS Version	Link to CDH 5 RPM
   RHEL/CentOS/Oracle 5	RHEL/CentOS/Oracle 5 link
   RHEL/CentOS/Oracle 6	RHEL/CentOS/Oracle 6 link
   RHEL/CentOS/Oracle 7	RHEL/CentOS/Oracle 7 link

2. Install the RPM for all RHEL versions:

   $ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm 

For instructions on how to add a CDH 5 yum repository or build your own CDH 5 yum repository, see Installing CDH 5 On Red Hat-compatible systems.

  Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo yum clean all

On SLES systems:

1. Download the CDH 5 "1-click Install" package.

   Download the RPM file, choose Save File, and save it to a directory to which you have write access (for example, your home directory).

2. Install the RPM:

   $ sudo rpm -i cloudera-cdh-5-0.x86_64.rpm

3. Update your system package index by running the following:

   $ sudo zypper refresh

$ sudo rpm -i cloudera-cdh-5-0.x86_64.rpm

For instructions on how to add a repository or build your own repository, see Installing CDH 5 on SLES Systems.

  Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo zypper clean --all

On Ubuntu and Debian systems:

1. Download the CDH 5 "1-click Install" package:

   OS Version	Package Link
   Jessie	Jessie package
   Wheezy	Wheezy package
   Precise	Precise package
   Trusty	Trusty package

2. Install the package by doing one of the following:
   • Choose Open with in the download window to use the package manager.
   • Choose Save File, save the package to a directory to which you have write access (for example, your home directory), and install it from the command line. For example:

     sudo dpkg -i cdh5-repository_1.0_all.deb

For instructions on how to add a repository or build your own repository, see the instructions on installing CDH 5 on Ubuntu and Debian systems.

  Important: Clean cached packages and headers to ensure that your system repos are up-to-date: sudo apt-get update

Step 3a: If you are using MRv1, upgrade the MRv1 packages on the appropriate hosts.

Skip this step if you are using YARN exclusively.

1. Install and deploy ZooKeeper as decribed in ZooKeeper Installation. Cloudera recommends that you install (or update) and start a ZooKeeper cluster, and ZooKeeper is required if you are deploying high availability (HA) for the NameNode or JobTracker.
2. Install each daemon package on the appropriate systems, as follows.

   Where to install	Install commands
   JobTracker host running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-0.20-mapreduce-jobtracker
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-0.20-mapreduce-jobtracker
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-0.20-mapreduce-jobtracker
   NameNode host running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-hdfs-namenode
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-hdfs-namenode
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-hdfs-namenode
   Secondary NameNode host (if used) running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-hdfs-secondarynamenode
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-hdfs-secondarynamenode
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-hdfs-secondarynamenode
   All cluster hosts except the JobTracker, NameNode, and Secondary (or Standby) NameNode hosts, running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode
   All client hosts, running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-client
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-client
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-client

Step 3b: If you are using YARN, upgrade the YARN packages on the appropriate hosts.

Skip this step if you are using MRv1 exclusively.

1. Install and deploy ZooKeeper as decribed in ZooKeeper Installation. Cloudera recommends that you install (or update) and start a ZooKeeper cluster, and ZooKeeper is required if you are deploying high availability (HA) for the NameNode or JobTracker.
2. Install each type of daemon package on the appropriate systems(s), as follows.

   Where to install	Install commands
   Resource Manager host (analogous to MRv1 JobTracker) running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-yarn-resourcemanager
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-yarn-resourcemanager
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-yarn-resourcemanager
   NameNode host running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-hdfs-namenode
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-hdfs-namenode
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-hdfs-namenode
   Secondary NameNode host (if used) running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-hdfs-secondarynamenode
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-hdfs-secondarynamenode
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-hdfs-secondarynamenode
   All cluster hosts except the Resource Manager (analogous to MRv1 TaskTrackers) running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce
   One host in the cluster running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver
   Ubuntu or Debian	$ sudo apt-get update; sudo apt-get install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver
   All client hosts, running:
   Red Hat/CentOS compatible	$ sudo yum clean all; sudo yum install hadoop-client
   SLES	$ sudo zypper clean --all; sudo zypper install hadoop-client
   Ubuntu or Debian	sudo apt-get update; sudo apt-get install hadoop-client

     Note:

   The hadoop-yarn and hadoop-hdfs packages are installed on each system automatically as dependencies of the other packages.

Section 5a: Upgrade the HDFS Metadata for HA Deployments

Section 5b: Upgrade the HDFS Metadata for Non-HA Deployments

Step 6a: Start MRv1

  Important:

If you have client hosts, make sure you also update them to CDH 5, and upgrade the components running on those clients.

Step 6b: Start MapReduce with YARN

1. If you have not already done so, create directories and set the correct permissions.
     Note: For more information about YARN configuration and permissions, see Deploying MapReduce v2 (YARN) on a Cluster.
   1. Create a history directory and set permissions; for example:

      $ sudo -u hdfs hadoop fs -mkdir -p /user/history 
      $ sudo -u hdfs hadoop fs -chmod -R 1777 /user/history  
      $ sudo -u hdfs hadoop fs -chown yarn /user/history

   2. Create the /var/log/hadoop-yarn directory and set ownership:

      $ sudo -u hdfs hadoop fs -mkdir -p /var/log/hadoop-yarn  
      $ sudo -u hdfs hadoop fs -chown yarn:mapred /var/log/hadoop-yarn 

      You create this directory because it is the parent of /var/log/hadoop-yarn/apps, which is explicitly configured in the yarn-site.xml.

   3. Verify the directory structure, ownership, and permissions:

      $ sudo -u hdfs hadoop fs -ls -R / 

      You should see:

      drwxrwxrwt - hdfs supergroup 0 2012-04-19 14:31 /tmp  
      drwxr-xr-x - hdfs supergroup 0 2012-05-31 10:26 /user  
      drwxrwxrwt - yarn supergroup 0 2012-04-19 14:31 /user/history  
      drwxr-xr-x - hdfs supergroup 0 2012-05-31 15:31 /var  
      drwxr-xr-x - hdfs supergroup 0 2012-05-31 15:31 /var/log  
      drwxr-xr-x - yarn mapred 0 2012-05-31 15:31 /var/log/hadoop-yarn 

2. Start YARN, and start the ResourceManager and NodeManager services:
     Important:

   Always start ResourceManager before starting NodeManager services.

   1. On the ResourceManager system, run the following command:

      $ sudo service hadoop-yarn-resourcemanager start 

   2. On each NodeManager system (typically the same ones where DataNode service runs):

      $ sudo service hadoop-yarn-nodemanager start 

3. Start the MapReduce JobHistory Server:
   1. On the MapReduce JobHistory Server system, run the following command:

      $ sudo service hadoop-mapreduce-historyserver start 

   2. For each user who will be submitting MapReduce jobs using YARN, or running Pig, Hive, or Sqoop 1 in a YARN installation, set the HADOOP_MAPRED_HOME environment variable as follows:

      $ export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce 

4. Before running production jobs, verify basic cluster operation by running an example from the Apache Hadoop web site.
     Note:

   For important configuration information, see Deploying MapReduce v2 (YARN) on a Cluster.

   1. Create a home directory for user joe:

      $ sudo -u hdfs hadoop fs -mkdir -p /user/joe 
      $ sudo -u hdfs hadoop fs -chown joe /user/joe

      Perform the remaining steps as the user joe.

   2. Make a directory in HDFS called input and copy XML files to it by running the following commands in pseudo-distributed mode:

      $ hadoop fs -mkdir input 
      $ hadoop fs -put /etc/hadoop/conf/*.xml input 
      $ hadoop fs -ls input 
      Found 3 items: 
      -rw-r--r-- 1 joe supergroup 1348 2012-02-13 12:21 input/core-site.xml 
      -rw-r--r-- 1 joe supergroup 1913 2012-02-13 12:21 input/hdfs-site.xml 
      -rw-r--r-- 1 joe supergroup 1001 2012-02-13 12:21 input/mapred-site.xml

   3. Set HADOOP_MAPRED_HOME for user joe:

      $ export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce

   4. Run an example Hadoop job to grep with a regular expression in your input data:

      $ hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep input output23 'dfs[a-z.]+'

   5. After the job completes, find the output in the HDFS directory named output23, which you specified to Hadoop:

      $ hadoop fs -ls 
      Found 2 items 
      drwxr-xr-x - joe supergroup 0 2009-08-18 18:36 /user/joe/input 
      drwxr-xr-x - joe supergroup 0 2009-08-18 18:38 /user/joe/output23

   6. List the output files:

      $ hadoop fs -ls output23 
      Found 2 items 
      drwxr-xr-x - joe supergroup 0 2009-02-25 10:33 /user/joe/output23/_SUCCESS 
      -rw-r--r-- 1 joe supergroup 1068 2009-02-25 10:33 /user/joe/output23/part-r-00000

   7. Read the results in the output file:

      $ hadoop fs -cat output23/part-r-00000 | head 
      1 dfs.safemode.min.datanodes 
      1 dfs.safemode.extension 
      1 dfs.replication 
      1 dfs.permissions.enabled 
      1 dfs.namenode.name.dir 
      1 dfs.namenode.checkpoint.dir 
      1 dfs.datanode.data.dir

This confirms that your cluster is successfully running CDH 5.

  Important:

If you have client hosts, make sure you also update them to CDH 5, and upgrade the components running on those clients as well.