Before you start, make sure you have completed the steps in Bootstrap.
Create a new AWS Kubernetes cluster
-
Set the environment variable to the name you assigned this cluster.
export CLUSTER_NAME=eks-example
See Get Started with EKS for information on naming your cluster.
-
Make sure your AWS credentials are up to date. Refresh the credentials using this command:
dkp update bootstrap credentials aws
-
Inspect or edit the cluster objects:
Use your favorite editor.
The objects are Custom Resources defined by Cluster API components, and they belong in three different categories:
-
Cluster
A Cluster object has references to the infrastructure-specific and control plane objects. Because this is an AWS cluster, there is an AWSCluster object that describes the infrastructure-specific cluster properties. Here, this means the AWS region, the VPC ID, subnet IDs, and security group rules required by the Pod network implementation.
-
Control Plane
A AWSManagedControlPlane object describes the control plane, which is the group of machines that run the Kubernetes control plane components, which include the etcd distributed database, the API server, the core controllers, and the scheduler. The object describes the configuration for these components. The object also has a reference to an infrastructure-specific object that describes the properties of all control plane machines.
-
Node Pool
A Node Pool is a collection of machines with identical properties. For example, a cluster might have one Node Pool with large memory capacity, another Node Pool with GPU support. Each Node Pool is described by three objects: The MachinePool references an object that describes the configuration of Kubernetes components (for example, kubelet) deployed on each node pool machine, and an infrastructure-specific object that describes the properties of all node pool machines. Here, it references a KubeadmConfigTemplate, and an AWSMachineTemplate object, which describes the instance type, the type of disk used, the size of the disk, among other properties.
For in-depth documentation about the objects, read Concepts in the Cluster API Book.
-
-
Create the cluster from the objects.
dkp create cluster eks --cluster-name=${CLUSTER_NAME} --additional-tags=owner=$(whoami)
cluster.cluster.x-k8s.io/eks-example created awsmanagedcontrolplane.controlplane.cluster.x-k8s.io/eks-example-control-plane created clusterresourceset.addons.cluster.x-k8s.io/calico-installation-eks-example created configmap/calico-cni-eks-example created machinedeployment.cluster.x-k8s.io/eks-example-md-0 created awsmachinetemplate.infrastructure.cluster.x-k8s.io/eks-example-md-0 created eksconfigtemplate.bootstrap.cluster.x-k8s.io/eks-example-md-0 created
-
Wait for the cluster control-plane to be ready:
kubectl wait --for=condition=ControlPlaneReady "clusters/${CLUSTER_NAME}" --timeout=20m
cluster.cluster.x-k8s.io/eks-example condition met
The
READY
status will becomeTrue
after the cluster control-plane becomes ready in one of the following steps. -
Once the objects are created on the API server, the Cluster API controllers reconcile them. They create infrastructure and machines. As they progress, they update the Status of each object. Konvoy provides a command to describe the current status of the cluster:
dkp describe cluster -c ${CLUSTER_NAME}
NAME READY SEVERITY REASON SINCE MESSAGE /eks-example True 9m4s ├─ControlPlane - AWSManagedControlPlane/eks-example-control-plane True 9m4s └─Workers └─MachineDeployment/eks-example-md-0 True 7m39s └─4 Machines... True 8m24s
-
As they progress, the controllers also create Events. List the Events using this command:
kubectl get events | grep ${CLUSTER_NAME}
For brevity, the example uses
grep
. It is also possible to use separate commands to get Events for specific objects. For example,kubectl get events --field-selector involvedObject.kind="AWSCluster"
andkubectl get events --field-selector involvedObject.kind="AWSMachine"
.24m Normal SuccessfulCreateVPC awsmanagedcontrolplane/eks-example-control-plane Created new managed VPC "vpc-0d5f41c645c7c6b98" 24m Normal SuccessfulSetVPCAttributes awsmanagedcontrolplane/eks-example-control-plane Set managed VPC attributes for "vpc-0d5f41c645c7c6b98" 24m Normal SuccessfulCreateSubnet awsmanagedcontrolplane/eks-example-control-plane Created new managed Subnet "subnet-08d795f7f61b4c45c" 24m Normal SuccessfulModifySubnetAttributes awsmanagedcontrolplane/eks-example-control-plane Modified managed Subnet "subnet-08d795f7f61b4c45c" attributes 24m Normal SuccessfulCreateSubnet awsmanagedcontrolplane/eks-example-control-plane Created new managed Subnet "subnet-025b963cb7b3c42b9" 24m Normal SuccessfulCreateSubnet awsmanagedcontrolplane/eks-example-control-plane Created new managed Subnet "subnet-0bea7db9735b647af" 24m Normal SuccessfulModifySubnetAttributes awsmanagedcontrolplane/eks-example-control-plane Modified managed Subnet "subnet-0bea7db9735b647af" attributes 24m Normal SuccessfulCreateSubnet awsmanagedcontrolplane/eks-example-control-plane Created new managed Subnet "subnet-0025d52a41d9ca901" 24m Normal SuccessfulCreateSubnet awsmanagedcontrolplane/eks-example-control-plane Created new managed Subnet "subnet-02b7d46fbb044e2f1" 24m Normal SuccessfulModifySubnetAttributes awsmanagedcontrolplane/eks-example-control-plane Modified managed Subnet "subnet-02b7d46fbb044e2f1" attributes 24m Normal SuccessfulCreateSubnet awsmanagedcontrolplane/eks-example-control-plane Created new managed Subnet "subnet-031ff53aae8da27ab" 24m Normal SuccessfulCreateInternetGateway awsmanagedcontrolplane/eks-example-control-plane Created new managed Internet Gateway "igw-03993e04cee3fc57b" 24m Normal SuccessfulAttachInternetGateway awsmanagedcontrolplane/eks-example-control-plane Internet Gateway "igw-03993e04cee3fc57b" attached to VPC "vpc-0d5f41c645c7c6b98" 24m Normal SuccessfulCreateNATGateway awsmanagedcontrolplane/eks-example-control-plane Created new NAT Gateway "nat-07b1bf537aac56688" 24m Normal SuccessfulCreateNATGateway awsmanagedcontrolplane/eks-example-control-plane Created new NAT Gateway "nat-0677e665b2019c726" 24m Normal SuccessfulCreateNATGateway awsmanagedcontrolplane/eks-example-control-plane Created new NAT Gateway "nat-0afd0e1fea275d3b0" 22m Normal SuccessfulCreateRouteTable awsmanagedcontrolplane/eks-example-control-plane Created managed RouteTable "rtb-050db62d744a8d614" 22m Normal SuccessfulCreateRoute awsmanagedcontrolplane/eks-example-control-plane Created route {... 22m Normal SuccessfulAssociateRouteTable awsmanagedcontrolplane/eks-example-control-plane Associated managed RouteTable "rtb-050db62d744a8d614" with subnet "subnet-08d795f7f61b4c45c" 22m Normal SuccessfulCreateRouteTable awsmanagedcontrolplane/eks-example-control-plane Created managed RouteTable "rtb-01650d4aa0551f70c" 22m Normal SuccessfulCreateRoute awsmanagedcontrolplane/eks-example-control-plane Created route {... 22m Normal SuccessfulAssociateRouteTable awsmanagedcontrolplane/eks-example-control-plane Associated managed RouteTable "rtb-01650d4aa0551f70c" with subnet "subnet-025b963cb7b3c42b9" 22m Normal SuccessfulCreateRouteTable awsmanagedcontrolplane/eks-example-control-plane Created managed RouteTable "rtb-04e91e2d7688e3a47" 22m Normal SuccessfulCreateRoute awsmanagedcontrolplane/eks-example-control-plane Created route {... 22m Normal SuccessfulAssociateRouteTable awsmanagedcontrolplane/eks-example-control-plane Associated managed RouteTable "rtb-04e91e2d7688e3a47" with subnet "subnet-0bea7db9735b647af" 22m Normal SuccessfulCreateRouteTable awsmanagedcontrolplane/eks-example-control-plane Created managed RouteTable "rtb-0f4bb2cb4e3da7515" 22m Normal SuccessfulCreateRoute awsmanagedcontrolplane/eks-example-control-plane Created route {... 22m Normal SuccessfulAssociateRouteTable awsmanagedcontrolplane/eks-example-control-plane Associated managed RouteTable "rtb-0f4bb2cb4e3da7515" with subnet "subnet-0025d52a41d9ca901" 22m Normal SuccessfulCreateRouteTable awsmanagedcontrolplane/eks-example-control-plane Created managed RouteTable "rtb-03af4834c7c8c2d5f" 22m Normal SuccessfulCreateRoute awsmanagedcontrolplane/eks-example-control-plane Created route {... 22m Normal SuccessfulAssociateRouteTable awsmanagedcontrolplane/eks-example-control-plane Associated managed RouteTable "rtb-03af4834c7c8c2d5f" with subnet "subnet-02b7d46fbb044e2f1" 22m Normal SuccessfulCreateRouteTable awsmanagedcontrolplane/eks-example-control-plane Created managed RouteTable "rtb-049093fd28c082deb" 22m Normal SuccessfulCreateRoute awsmanagedcontrolplane/eks-example-control-plane Created route {... 22m Normal SuccessfulAssociateRouteTable awsmanagedcontrolplane/eks-example-control-plane Associated managed RouteTable "rtb-049093fd28c082deb" with subnet "subnet-031ff53aae8da27ab" 22m Normal SuccessfulCreateSecurityGroup awsmanagedcontrolplane/eks-example-control-plane Created managed SecurityGroup "sg-0d668ab6a67b5f806" for Role "bastion" 22m Normal SuccessfulCreateSecurityGroup awsmanagedcontrolplane/eks-example-control-plane Created managed SecurityGroup "sg-09c050536025c517f" for Role "node-eks-additional" 22m Normal SuccessfulAuthorizeSecurityGroupIngressRules awsmanagedcontrolplane/eks-example-control-plane Authorized security group ingress rules [{SSH tcp 22 22 [0.0.0.0/0] []}] for SecurityGroup "sg-0d668ab6a67b5f806" 22m Normal SuccessfulAuthorizeSecurityGroupIngressRules awsmanagedcontrolplane/eks-example-control-plane Authorized security group ingress rules [{SSH tcp 22 22 [] [sg-0d668ab6a67b5f806]}] for SecurityGroup "sg-09c050536025c517f" 22m Normal InitiatedCreateEKSControlPlane awsmanagedcontrolplane/eks-example-control-plane Initiated creation of a new EKS control plane default_eks-example-control-plane 10m Normal SuccessfulCreateEKSControlPlane awsmanagedcontrolplane/eks-example-control-plane Created new EKS control plane default_eks-example-control-plane 10m Normal SucessfulCreateKubeconfig awsmanagedcontrolplane/eks-example-control-plane Created kubeconfig for cluster "eks-example" 10m Normal SucessfulCreateUserKubeconfig awsmanagedcontrolplane/eks-example-control-plane Created user kubeconfig for cluster "eks-example" 10m Normal DeletedVPCCNI awsmanagedcontrolplane/eks-example-control-plane The AWS VPC CNI has been removed from the cluster. Ensure you enable a CNI via another mechanism 9m41s Normal SuccessfulSetNodeRef machine/eks-example-md-0-6645677545-5drlr ip-10-0-94-48.us-west-2.compute.internal 9m41s Normal SuccessfulSetNodeRef machine/eks-example-md-0-6645677545-g9ngw ip-10-0-117-5.us-west-2.compute.internal 9m45s Normal SuccessfulSetNodeRef machine/eks-example-md-0-6645677545-klpm9 ip-10-0-81-221.us-west-2.compute.internal 9m50s Normal SuccessfulSetNodeRef machine/eks-example-md-0-6645677545-ldcrj ip-10-0-115-179.us-west-2.compute.internal 24m Normal SuccessfulCreate machineset/eks-example-md-0-6645677545 Created machine "eks-example-md-0-6645677545-5drlr" 24m Normal SuccessfulCreate machineset/eks-example-md-0-6645677545 Created machine "eks-example-md-0-6645677545-g9ngw" 24m Normal SuccessfulCreate machineset/eks-example-md-0-6645677545 Created machine "eks-example-md-0-6645677545-ldcrj" 24m Normal SuccessfulCreate machineset/eks-example-md-0-6645677545 Created machine "eks-example-md-0-6645677545-klpm9" 10m Normal SuccessfulCreate awsmachine/eks-example-md-0-sq4q9 Created new node instance with id "i-0074f255b40d0374a" 9m41s Normal SuccessfulDeleteEncryptedBootstrapDataSecrets awsmachine/eks-example-md-0-sq4q9 AWS Secret entries containing userdata deleted 10m Normal SuccessfulCreate awsmachine/eks-example-md-0-tbv7c Created new node instance with id "i-0064878f89bd2a521" 9m45s Normal SuccessfulDeleteEncryptedBootstrapDataSecrets awsmachine/eks-example-md-0-tbv7c AWS Secret entries containing userdata deleted 10m Normal SuccessfulCreate awsmachine/eks-example-md-0-zrgg2 Created new node instance with id "i-07fb4ba5103e4d6ef" 9m41s Normal SuccessfulDeleteEncryptedBootstrapDataSecrets awsmachine/eks-example-md-0-zrgg2 AWS Secret entries containing userdata deleted 10m Normal SuccessfulCreate awsmachine/eks-example-md-0-zw56b Created new node instance with id "i-0ced4184741b7ded6" 9m49s Normal SuccessfulDeleteEncryptedBootstrapDataSecrets awsmachine/eks-example-md-0-zw56b AWS Secret entries containing userdata deleted 24m Normal SuccessfulCreate machinedeployment/eks-example-md-0 Created MachineSet "eks-example-md-0-6645677545"
Known Limitations
- The Konvoy version used to create a bootstrap cluster must match the Konvoy version used to create a workload cluster.
- Konvoy supports deploying one workload cluster.
- Konvoy generates a set of objects for one Node Pool.
- Konvoy does not validate edits to cluster objects.