Skip to content

DKE (kubeadm clusters)

"DKE" is this lab's own naming for its custom kubeadm clusters — there's no vendor product behind the name. Each one comes from the same reusable Terraform module (terraform/vsphere/modules/k8s), instantiated once per cluster with different parameters.

Clusters running on this module

Instance Control-plane host Network plugin Pod CIDR Service CIDR Notes
dke-mgmt esxi.devhome.cloud Flannel 10.244.0.0/16 10.96.0.0/16 The management cluster — hosts ArgoCD/Rancher; see Architecture Overview
dkes esxi-dev.devhome.cloud Calico 10.68.0.0/16 10.69.0.0/16 Calico BGP-advertises LoadBalancerIPs (10.70.0.0/24) — new nodes must be added as BGP neighbors in pfSense's FRR config

Every instance gets its own non-overlapping pod/service/LB CIDR block — this is a manual per-cluster allocation, not automatic, so check existing modules before picking a new range.

What the module actually does

  1. Clones the cluster's VMs (control-plane + worker/system pools) from a Packer-built template — see Packer.
  2. Runs kubeadm init/kubeadm join via an Ansible provisioner (dke-init-playbook.yml), templating kubeadm-config.yaml for the chosen network plugin and Kubernetes version (dke_version).
  3. Installs the chosen CNI (Calico or Flannel).
  4. For Calico clusters: sets up the pfSense BGP peering (ASN 64700) so LoadBalancerIPs actually get advertised onto the network — see pfSense Overview.

Floating control-plane VIP (kube-vip)

Newer DKE clusters get a kube-vip-managed floating VIP as controlPlaneEndpoint, BGP-advertised the same way Calico advertises LoadBalancer IPs — but with its own dedicated ASN (64800; FRR peer-groups can only carry one remote-as each, so it can't share Calico's) and its own dedicated source IP on the control-plane node (same NIC/VLAN, no new vNIC — a second BGP speaker can't share a source IP with pfSense's FRR either). The manifest is hand-templated to match the repo's existing sed-substitution convention rather than generated by kube-vip's own Docker-based generator, since these nodes run containerd.

dke-mgmt predates this design and still pins its API endpoint to the control-plane node's own IP — this is exactly why a DHCP lease change is disruptive there. See Kubernetes Troubleshooting for the recovery procedure, and Terraform for why dke-mgmt gets migrated to kube-vip on its next rebuild rather than a live retrofit.

dke-mgmt also self-installs ArgoCD

Unlike every other cluster (which gets registered into an already-running ArgoCD on dke-mgmt), dke-mgmt's own Terraform installs ArgoCD onto itself directly via a helm_release resource, triggered on the same replacement token as the cluster itself — there's no chicken-and-egg "register with ArgoCD" step for the cluster that is ArgoCD's host.