Cloudera Enterprise 5.15.x | Other versions

Configuring Kerberos for Flume Thrift Source and Sink Using Cloudera Manager

The Thrift source can be configured to start in secure mode by enabling Kerberos authentication. To communicate with a secure Thrift source, the Thrift sink should also be operating in secure mode.

  1. Open the Cloudera Manager Admin Console and go to the Flume service.
  2. Click the Configuration tab.
  3. Select Scope > Agent.
  4. Select Category > Main.
  5. Edit the Configuration File property and add the Thrift source and sink properties listed in the tables below to the configuration file.
    Table 1. Thrift Source Properties
    Property Description
    kerberos Set to true to enable Kerberos authentication. The agent-principal and agent-keytab properties are required for successful authentication. The Thrift source in secure mode, will accept connections only from Thrift sinks that have Kerberos-enabled and are successfully authenticated to the KDC.
    agent-principal The Kerberos principal used by the Thrift Source to authenticate to the KDC.
    agent-keytab The path to the keytab file used by the Thrift Source in combination with the agent-principal to authenticate to the KDC.
    Table 2. Thrift Sink Properties
    Property Description
    kerberos Set to true to enable Kerberos authentication. In Kerberos mode, client-principal, client-keytab and server-principal are required for successful authentication and communication to a Kerberos enabled Thrift Source.
    client-principal The principal used by the Thrift Sink to authenticate to the Kerberos KDC.
    client-keytab The path to the keytab file used by the Thrift Sink in combination with the client-principal to authenticate to the KDC.
    server-principal The principal of the Thrift Source to which this Thrift Sink connects.
      Note: Since Cloudera Manager generates the Flume keytab files for you, and the locations of the keytab files cannot be known beforehand, substitution variables are required for Flume. Cloudera Manager provides two Flume substitution variables called $KERBEROS_PRINCIPAL and $KERBEROS_KEYTAB to configure the principal name and the keytab file path respectively on each host.

    Make sure you are configuring these properties for each Thrift source and sink instance managed by Cloudera Manager. For example, for agent a1, source r1, and sink k1, you would add the following properties:

    # Kerberos properties for Thrift source s1 
a1.sources.r1.kerberos=true
a1.sources.r1.agent-principal=<source_principal>
a1.sources.r1.agent-keytab=<path/to/source/keytab>

# Kerberos properties for Thrift sink k1 
a1.sinks.k1.kerberos=true
a1.sinks.k1.client-principal=<sink_principal>
a1.sinks.k1.client-keytab=<path/to/sink/keytab>
a1.sinks.k1.server-principal=<path/to/source/keytab>
  6. Click Save Changes to commit the changes.
  7. Restart the Flume service.
Page generated May 18, 2018.