Make the New Cluster Self-Managed

Konvoy deploys all cluster lifecycle services to a bootstrap cluster, which then deploys a workload cluster. When the workload cluster is ready, move the cluster lifecycle services to the workload cluster, which is now self-managed. This guide describes how to make a workload cluster self-managed.

Before you start, make sure you have created a workload cluster, as described in Create the Cluster.

Explore the cluster

NOTE: If you have not already retrieved the kubeconfig after creating the cluster, use this command before proceeding: dkp get kubeconfig -c ${CLUSTER_NAME} > ${CLUSTER_NAME}.conf

Before setting the cluster to manage itself, explore your cluster with this command:

kubectl get pods -A --kubeconfig ${CLUSTER_NAME}.conf

Make the new Kubernetes cluster manage itself

Deploy cluster lifecycle services on the workload cluster:

dkp create bootstrap controllers --kubeconfig ${CLUSTER_NAME}.conf

INFO[2021-09-06T23:15:16-05:00] Initializing bootstrap controllers            src="bootstrap/controllers.go:96"
INFO[2021-09-06T23:16:28-05:00] Created bootstrap controllers                 src="bootstrap/controllers.go:101"
INFO[2021-09-06T23:16:28-05:00] Waiting for bootstrap controllers to be ready  src="bootstrap/controllers.go:104"
INFO[2021-09-06T23:22:57-05:00] Bootstrap controllers are ready               src="bootstrap/controllers.go:109"
INFO[2021-09-06T23:22:57-05:00] Patching ClusterRoleBinding for CAPPP         src="bootstrap/controllers.go:112"
INFO[2021-09-06T23:22:57-05:00] Initializing Tigera operator                  src="bootstrap/clusterresourceset.go:37"
INFO[2021-09-06T23:22:58-05:00] Created Tigera operator                       src="bootstrap/clusterresourceset.go:42"
INFO[2021-09-06T23:22:59-05:00] Initializing AWS EBS CSI CustomResourceSet    src="bootstrap/clusterresourceset.go:109"
INFO[2021-09-06T23:23:00-05:00] Created AWS EBS CSI CustomResourceSet         src="bootstrap/clusterresourceset.go:114"
INFO[2021-09-06T23:23:00-05:00] Initializing Local Volume Provisioner CustomResourceSet  src="bootstrap/clusterresourceset.go:116"
INFO[2021-09-06T23:23:00-05:00] Created Local Volume Provisioner CustomResourceSet  src="bootstrap/clusterresourceset.go:121"
INFO[2021-09-06T23:23:00-05:00] Initializing Cluster Autoscaler CustomResourceSet  src="bootstrap/clusterresourceset.go:181"
INFO[2021-09-06T23:23:01-05:00] Created Cluster Autoscaler CustomResourceSet  src="bootstrap/clusterresourceset.go:186"
INFO[2021-09-06T23:23:01-05:00] Initializing Node Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:239"
INFO[2021-09-06T23:23:01-05:00] Created Node Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:244"
INFO[2021-09-06T23:23:01-05:00] Initializing NVIDIA GPU Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:297"
INFO[2021-09-06T23:23:02-05:00] Created NVIDIA GPU Feature Discovery CustomResourceSet  src="bootstrap/clusterresourceset.go:302"

Move the Cluster API objects from the bootstrap to the workload cluster:

The cluster lifecycle services on the workload cluster are ready, but the workload cluster configuration is on the bootstrap cluster. The move command moves the configuration, which takes the form of Cluster API Custom Resource objects, from the bootstrap to the workload cluster. This process is also called a Pivot.
```
dkp move --to-kubeconfig ${CLUSTER_NAME}.conf
```
```
INFO[2021-09-06T23:23:52-05:00] Checking move syntax                          fromClusterContext= fromClusterKubeconfig= src="move/move.go:84" toClusterContext= toClusterKubeconfig=preprovisioned-cluster.conf
INFO[2021-09-06T23:23:52-05:00] Running pivot command                         fromClusterContext= fromClusterKubeconfig= src="move/move.go:129" toClusterContext= toClusterKubeconfig=preprovisioned-cluster.conf
INFO[2021-09-06T23:24:21-05:00] Pivot operation complete.                     src="move/move.go:157"
INFO[2021-09-06T23:24:21-05:00] You can now view resources in the moved cluster by using the --kubeconfig flag with kubectl. For example: kubectl --kubeconfig=preprovisioned-cluster.conf get nodes  src="move/move.go:158"
```
NOTE: To ensure only one set of cluster lifecycle services manages the workload cluster, Konvoy first pauses reconciliation of the objects on the bootstrap cluster, then creates the objects on the workload cluster. As Konvoy copies the objects, the cluster lifecycle services on the workload cluster reconcile the objects. The workload cluster becomes self-managed after Konvoy creates all the objects. If it fails, the move command can be safely retried.

Wait for the cluster control-plane to be ready:

kubectl --kubeconfig ${CLUSTER_NAME}.conf wait --for=condition=ControlPlaneReady "clusters/${CLUSTER_NAME}" --timeout=20m

cluster.cluster.x-k8s.io preprovisioned-cluster condition met

Use the cluster lifecycle services on the workload cluster to check the workload cluster status:

NOTE: After moving the cluster lifecycle services to the workload cluster, remember to use Konvoy with the workload cluster kubeconfig.

dkp describe cluster --kubeconfig ${CLUSTER_NAME}.conf -c ${CLUSTER_NAME}

NAME                                                                          READY  SEVERITY  REASON  SINCE  MESSAGE
/preprovisioned-cluster                                                       True                     2m31s
├─ClusterInfrastructure - PreprovisionedCluster/preprovisioned-cluster
├─ControlPlane - KubeadmControlPlane/preprovisioned-cluster-control-plane     True                     2m31s
│ └─3 Machines...                                                             True                     2m33s
└─Workers
  └─MachineDeployment/preprovisioned-cluster-md-0
    └─4 Machines...                                                           True                     2m33s

Remove the bootstrap cluster, as the workload cluster is now self-managed:

dkp delete bootstrap

INFO[2021-06-07T14:53:36-07:00] Deleting bootstrap cluster                    src="bootstrap/bootstrap.go:182"

Known Limitations

NOTE: Be aware of these limitations in the current release of Konvoy.

Konvoy supports moving only one set of cluster objects from the bootstrap cluster to the workload cluster, or vice-versa.
Konvoy only supports moving all namespaces in the cluster; Konvoy does not support migration of individual namespaces.