Security

DC/OS NiFi Service encryption, authentication, and authorization

DC/OS Apache NiFi Security

TThe DC/OS Apache NiFi service supports NiFi’s native transport encryption, authentication, and authorization mechanisms. The service provides automation and orchestration to simplify the usage of these important features.

A good overview of these features can be found in the Apache NiFi Administration Guide.

Transport Encryption and Kerberos Authentication

With transport encryption enabled, DC/OS Apache NiFi will automatically deploy all nodes with the correct configuration to encrypt communication via SSL. The nodes will communicate securely between themselves using SSL. SSL authentication requires that all NiFi Nodes present a valid certificate from which their identity can be derived for communicating between themselves. DC/OS Apache NiFi uses the CN of the SSL certificate as the principal for a given Node. For e.g. CN=nifi-0-node.demonifi, O=“Mesosphere, Inc”, L=San Francisco, ST=CA, C=US.

The service uses the DC/OS CA to generate the SSL artifacts that it uses to secure the service. Any client that trusts the DC/OS CA will consider the service’s certificates valid.

Note: Enabling transport encryption is required to use SSL authentication for authentication, but is optional for Kerberos authentication.

Prerequisites

Configure Transport Encryption

Tip: A complete guide to Configuring DC/OS Access for NiFi can be found here.

Set up the Service Account

Grant the service account the correct permissions.

In DC/OS 1.10, the required permission is dcos:superuser full

curl -k -v $(dcos config show core.dcos_url)/ca/dcos-ca.crt -o ../dcos-ca.crt
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
dcos package install dcos-enterprise-cli
dcos security org service-accounts keypair nifi-private-key.pem nifi-public-key.pem
dcos security org service-accounts create -p nifi-public-key.pem -d "dcos_nifi" <service name>
dcos security secrets create-sa-secret --strict nifi-private-key.pem <service name> <service name secret>
dcos security org users grant <service name> dcos:superuser full --description "grant permission to superuser"

where is the name of the service to be installed.

Transport Encryption for Clients

With Transport Encryption enabled, service clients will need to be configured to use the DC/OS CA bundle to verify the connections they make to the service. Consult your client’s documentation for trusting a CA and configure your client appropriately.

Authentication

DC/OS Apache NiFi supports two authentication mechanisms, SSL and Kerberos. The two are supported independently and may not be combined. If both SSL and Kerberos authentication are enabled, the service will use Kerberos authentication.

Note: Kerberos authentication can, however, be combined with transport encryption.

CA based authentication between nodes

DC/OS Apache NiFi requires certificated based authentication between nodes. DC/OS Apache NiFi uses the CN of the SSL certificate as the principal for a given Node. For e.g. CN=nifi-0-node.demonifi, O=“Mesosphere, Inc”, L=San Francisco, ST=CA, C=US.

Kerberos Authentication for End Users

Kerberos authentication relies on a central authority to verify that NiFi Users clients are who they say they are. DC/OS Apache NiFi integrates with your existing Kerberos infrastructure to verify the identity of clients.

Prerequisites

  • The hostname and port of a Key Distribution Center (KDC) reachable from your DC/OS cluster
  • Sufficient access to the KDC to create Kerberos principals
  • Sufficient access to the KDC to retrieve a keytab for the generated principals
  • The DC/OS Enterprise CLI
  • DC/OS Superuser permissions

Configure Kerberos Authentication

1. Create principals

The DC/OS Apache NiFi service requires a Kerberos principal for the service principal and user principal.Each principal must be of the form

nifiprincipal@<service realm>
nifiadmin@<service realm>

2. Place Service Keytab in DC/OS Secret Store

The DC/OS Apache NiFi service uses a keytab containing above service and user principals (service keytab). After creating the principals above, generate the service keytab making sure to include all the node principals. This will be stored as a secret in the DC/OS Secret Store by name __dcos_base64__secret_name .The DC/OS security modules will handle decoding the file when it is used by the service. More details here.

Create secret by name “nifiadmin_kerberos_secret” for password of Kerberos User Principal: nifiadmin

Documentation for adding a file to the secret store can be found here.

Note: Secrets access is controlled by DC/OS Spaces, which function like namespaces. Any secret in the same DC/OS Space as the service will be accessible by the service.

3. Install the Service

Install the DC/OS Apache NiFi service with the following options in addition to your own:

{
 "service": {
  "name": "/demo/nifi",
   "security": {
   "kerberos": {
        "kdc": {
       "hostname": "kdc.marathon.autoip.dcos.thisdcos.directory",
       "port": 2500
     },
     "keytab_secret": "__dcos_base64___keytab",
     "primary": "nifi",
     "realm": "LOCAL",
     "service_principal": "nifiprincipal@LOCAL",
     "user_principal": "nifiadmin@LOCAL",
     "user_principal_keytab": "nifiadmin_kerberos_secret"
   },
   "kerberos_tls": {
     "enable": true
   }
 },
 "service_account": "dcos_nifi",
 "service_account_secret": "dcos_nifi_secret",
 "virtual_network_enabled": true,
 "virtual_network_name": "dcos",
 }
}