Configuring Edge-LB to Connect to Kubernetes

ENTERPRISE

Learn to connect to Kubernetes of top of your DC/OS Enterprise cluster using DC/OS' Edge-LB load balancer.

To connect to your Kubernetes cluster from outside your DC/OS cluster, you will now need to set up and configure the Edge-LB service for inbound load-balancing to your private Kubernetes cluster.

To set up Edge-LB for DC/OS Kubernetes you will need to:

  • Configure Edge-LB on your cluster and deploy the configured pool for Kubernetes including:
    • setting up a service account for Edge-LB
    • installing Edge-LB, including the CLI plugin

Set up and Install Edge-LB with service account

  1. First, add the Edge-LB repositories, replacing the download links with the latest available binaries:

    dcos package repo add --index=0 edgelb https://<insert download link>/stub-universe-edgelb.json
    dcos package repo add --index=0 edgelb-pool https://<insert download link>/stub-universe-edgelb-pool.json
    

    IMPORTANT: If you wish to download the repositories from the support site using your browser, you will need to be logged in with an enterprise customer service account.

  2. Next, create an Edge-LB service account:

    dcos security org service-accounts keypair edge-lb-private-key.pem edge-lb-public-key.pem
    dcos security org service-accounts create -p edge-lb-public-key.pem -d "Edge-LB service account" edge-lb-principal
    dcos security org service-accounts show edge-lb-principal
    dcos security secrets create-sa-secret --strict edge-lb-private-key.pem edge-lb-principal dcos-edgelb/edge-lb-secret
    dcos security org groups add_user superusers edge-lb-principal
    
  3. Then, create an options JSON file to install Edge-LB with its service account.

    Here we create edge-lb-options.json and add the following configuration snippet:

    {
        "service": {
            "secretName": "dcos-edgelb/edge-lb-secret",
            "principal": "edge-lb-principal",
            "mesosProtocol": "https"
        }
    }
    

    IMPORTANT: If you wish to not auto expose the api via Edge-LB Auto Pools and wish to disable the auto pool creation, add `"defaultPoolTemplate": "false"` to the `service` definition to prevent the creation of the `default` Auto Pool.

Save the file.
  1. Lastly, install the edgelb package on your DC/OS Enterprise cluster.

    In your CLI, enter:

    dcos package install --options=edge-lb-options.json edgelb --yes
    

Expose API with Edge-LB Auto Pools

  1. If you have not disabled the default Edge-LB Auto Pool, the MKE cluster will be exposed out automatically and you can skip the pool creation and proceed with finding the ip.

Create and launch an Edge-LB configured pool deployment for your Kubernetes services

  1. Next, save the following Kubernetes/Edge-LB Service configuration as edgelb.json.

    Copy for your convenience:

    {
        "apiVersion": "V2",
        "name": "edgelb-kubernetes-cluster-proxy-basic",
        "count": 1,
        "autoCertificate": true,
        "haproxy": {
            "frontends": [{
                    "bindPort": 6443,
                    "protocol": "HTTPS",
                    "certificates": [
                        "$AUTOCERT"
                    ],
                    "linkBackend": {
                        "defaultBackend": "kubernetes-cluster1"
                    }
                },
                {
                    "bindPort": 6444,
                    "protocol": "HTTPS",
                    "certificates": [
                        "$AUTOCERT"
                    ],
                    "linkBackend": {
                        "defaultBackend": "kubernetes-cluster2"
                    }
                }
            ],
            "backends": [{
                    "name": "kubernetes-cluster1",
                    "protocol": "HTTPS",
                    "services": [{
                        "mesos": {
                            "frameworkName": "kubernetes-cluster1",
                            "taskNamePattern": "kube-control-plane"
                        },
                        "endpoint": {
                            "portName": "apiserver"
                        }
                    }]
                },
                {
                    "name": "kubernetes-cluster2",
                    "protocol": "HTTPS",
                    "services": [{
                        "mesos": {
                            "frameworkName": "kubernetes-cluster2",
                            "taskNamePattern": "kube-control-plane"
                        },
                        "endpoint": {
                            "portName": "apiserver"
    
                        }
                    }]
                }
            ],
            "stats": {
                "bindPort": 6090
            }
        }
    }
    

    and paste in the file using your text editor.

  2. Next, deploy the Edge-LB configured pool for your Kubernetes service:

    In your CLI, enter:

    dcos edgelb create edgelb.json
    
  3. List your Edge-LB configured pools.

    dcos edgelb list
    

    and confirm your output looks similar to the following:

    $ dcos edgelb list
    NAME                                   APIVERSION  COUNT  ROLE          PORTS
    edgelb-kubernetes-cluster-proxy-basic  V2          1      slave_public  6090, 6443, 6444
    
  4. Check that the status of your Edge-LB deployment is in TASK_RUNNING state (under STATE):

    dcos edgelb status edgelb-kubernetes-cluster-proxy-basic
    

    The responding output should look like the following:

    $ dcos edgelb status edgelb-kubernetes-cluster-proxy-basic
    NAME                  TASK ID                                                     STATE
    edgelb-pool-0-server  edgelb-pool-0-server__a6e4b1a1-e63c-4579-a27e-a54328f31321  TASK_RUNNING
    

Find the IP

  1. Find the public IP of the deployment (substitute edgelb-kubernetes-cluster-proxy-basic if not using Auto Pool Exposure):

    dcos edgelb endpoints auto-default
      NAME                   PORT  INTERNAL IPS  EXTERNAL IPS
      frontend_0.0.0.0_6443  6443  172.16.7.60   54.184.41.74
      stats                  9090  172.16.7.60   54.184.41.74
    Public/private IPs metadata is inaccurate in case of pools that use virtual networks.
    
  2. Save the IP as a variable:

    export EDGELB_PUBLIC_AGENT_IP=<external ip of port 6443 from above>
    

Next Step: Test Kubernetes connections and view the Kubernetes dashboard

Now you have configured Edge-LB to connect to your Kubernetes clusters. In the last section, you will test connections and view the Kubernetes Dashboard via a web proxy on your browser.