These instructions assume that you are upgrading Sqoop 1 as part of an upgrade to the latest CDH 5 release, and have already performed the steps under Upgrading from an Earlier CDH 5 Release to the Latest Release.

To upgrade Sqoop 1 from an earlier CDH 5 release, install the new version of Sqoop 1 using one of the methods described below: Sqoop 1 Prerequisites or .Installing the Sqoop 1 Tarball.

  Important: Configuration files

• If you install a newer version of a package that is already on the system, configuration files that you have modified will remain intact.
• If you uninstall a package, the package manager renames any configuration files you have modified from <file> to <file>.rpmsave. If you then re-install the package (probably to install a new version) the package manager creates a new <file> with applicable defaults. You are responsible for applying any changes captured in the original configuration file to the new configuration file. In the case of Ubuntu and Debian upgrades, you will be prompted if you have made changes to a file for which there is a new version. For details, see Automatic handling of configuration files by dpkg.

The packaging options for installing Sqoop 1 are:

• RPM packages
• Tarball
• Debian packages

Sqoop 1 requires the following:

• An operating system supported by CDH 5.
• Oracle JDK.
• Services that you want to use with Sqoop, such as HBase, Hive HCatalog, and Accumulo. When you run Sqoop, it checks to see if these services are installed and configured. It logs warnings for services it does not find. These warnings, shown below, are harmless. You can suppress these error messages by setting the variables $HBASE_HOME, $HCAT_HOME and $ACCUMULO_HOME to any existing directory.

  > Warning: /usr/lib/sqoop/../hbase does not exist! HBase imports will fail.
  > Please set $HBASE_HOME to the root of your HBase installation.
  > Warning: /usr/lib/sqoop/../hive-hcatalog does not exist! HCatalog jobs will fail.
  > Please set $HCAT_HOME to the root of your HCatalog installation.
  > Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
  > Please set $ACCUMULO_HOME to the root of your Accumulo installation. 

Installing the Sqoop 1 RPM or Debian packages is more convenient than installing the Sqoop 1 tarball because the packages:

• Handle dependencies
• Provide for easy upgrades
• Automatically install resources to conventional locations

The Sqoop 1 packages consist of:

• sqoop — Complete Sqoop 1 distribution
• sqoop-metastore — For installation of the Sqoop 1 metastore only

  Note: Install Cloudera Repository

Before using the instructions on this page to install or upgrade:

• Install the Cloudera yum, zypper/YaST or apt repository.
• Install or upgrade CDH 5 and make sure it is functioning correctly.

For instructions, see Installing the Latest CDH 5 Release and Upgrading Unmanaged CDH Using the Command Line.

To install Sqoop 1 on a RHEL-compatible system:

$ sudo yum install sqoop

To install Sqoop 1 on an Ubuntu or other Debian system:

$ sudo apt-get install sqoop

To install Sqoop 1 on a SLES system:

$ sudo zypper install sqoop

If you have already configured CDH on your system, there is no further configuration necessary for Sqoop 1. You can start using Sqoop 1 by using commands such as:

$ sqoop help
$ sqoop version
$ sqoop import

The Sqoop 1 tarball is a self-contained package containing everything necessary to use Sqoop 1 with YARN on a Unix-like system.

  Important:

Make sure you have read and understood the section on tarballs before you proceed with a tarball installation.

To install Sqoop 1 from the tarball, unpack the tarball in a convenient location. Once it is unpacked, add the bin directory to the shell path for easy access to Sqoop 1 commands. Documentation for users and developers can be found in the docs directory.

To install the Sqoop 1 tarball on Linux-based systems:

Run the following command:

$ (cd /usr/local/ && sudo tar -zxvf _<path_to_sqoop.tar.gz>_)

  Note:

When installing Sqoop 1 from the tarball package, you must make sure that the environment variables JAVA_HOME and HADOOP_MAPRED_HOME are configured correctly. The variable HADOOP_MAPRED_HOME should point to the root directory of Hadoop installation. Optionally, if you intend to use any Hive or HBase related functionality, you must also make sure that they are installed. Configure the variables HIVE_HOME and HBASE_HOME to point to the root directory of their respective installation.

Sqoop 1 does not ship with third party JDBC drivers. You must download them separately and save them to the /var/lib/sqoop/ directory on the server. The following sections show how to install the most common JDBC Drivers.

  Note:

• The JDBC drivers need to be installed only on the machine where Sqoop runs; you do not need to install them on all hosts in your Hadoop cluster.

• Kerberos authentication is not supported by the Sqoop Connector for Teradata.

Before you begin:

mkdir -p /var/lib/sqoop
chown sqoop:sqoop /var/lib/sqoop
chmod 755 /var/lib/sqoop

This sets permissions to drwxr-xr-x.

For JDBC drivers for Hive, Impala, Teradata, or Netezza, see the Connectors documentation.

• For each user who will be submitting MapReduce jobs using MapReduce v2 (YARN), or running Pig, Hive, or Sqoop 1 in a YARN installation, make sure that the HADOOP_MAPRED_HOME environment variable is set correctly, as follows:

  $ export HADOOP_MAPRED_HOME=/usr/lib/hadoop-mapreduce

• For each user who will be submitting MapReduce jobs using MapReduce v1 (MRv1), or running Pig, Hive, or Sqoop 1 in an MRv1 installation, set the HADOOP_MAPRED_HOME environment variable as follows:

  $ export HADOOP_MAPRED_HOME=/usr/lib/hadoop-0.20-mapreduce

For additional documentation see the Sqoop user guides.