Note: Install Cloudera Repository

Before using the instructions on this page to install or upgrade:

• Install the Cloudera yum, zypper/YaST or apt repository.
• Install or upgrade CDH 5 and make sure it is functioning correctly.

For instructions, see Installing the Latest CDH 5 Release and Upgrading Unmanaged CDH Using the Command Line.

To see which version of Spark is shipping in the current release, check the CDH Version and Packaging Information. For important information, see the CDH 5 Release Notes, in particular:

• New Features and Changes in CDH 5
• Apache Spark Incompatible Changes and Limitations
• Apache Spark Known Issues

  Note: If you are using Cloudera Manager 5.9.0, 5.9.1, or 5.10.0 and have hosts with the NodeManager role but without the Spark Gateway role, you must add the Spark Gateway role to all NodeManager hosts and redeploy the client configurations.

• RHEL-compatible system:

  • To install all Spark packages:

    $ sudo yum install spark-core spark-master spark-worker spark-history-server spark-python

  • To install only the packages needed to run Spark on YARN:

    $ sudo yum install spark-core spark-history-server spark-python

• SLES system:

  • To install all Spark packages:

    $ sudo zypper install spark-core spark-master spark-worker spark-history-server spark-python

  • To install only the packages needed to run Spark on YARN:

    $ sudo zypper install spark-core spark-history-server spark-python

• Ubuntu or Debian system:

  • To install all Spark packages:

    $ sudo apt-get install spark-core spark-master spark-worker spark-history-server spark-python

  • To install only the packages needed to run Spark on YARN:

    $ sudo apt-get install spark-core spark-history-server spark-python

You are now ready to configure and start Spark. See Managing Spark Standalone Using the Command Line.

  Note:

If you uploaded the Spark JAR file as described under Optimizing YARN Mode in Unmanaged CDH Deployments, use the same instructions to upload the new version of the file each time you upgrade to a new minor release of CDH (for example, any CDH 5.4.x release, including 5.4.0).