Skip to content

DHCP & PXE Boot

DHCP (Kea)

  • Dynamic pools serve LAN/VLAN 20/VLAN 30 clients with no reservations by default — see Network Design for why that's a deliberate trade-off for Kubernetes nodes specifically.
  • ESXi hosts are the exception: they get static DHCP mappings (Kea reservations keyed to the physical NIC MAC) so their management IP never drifts, since so much else (Terraform state, Ansible inventory, vCenter host objects) is keyed to a stable ESXi hostname/IP.
  • DHCP-lease DNS registration is not the DNS Resolver checkbox under Kea. It's gated by the global kea/dnsreg config flag (Services → DHCP Server → Settings → DNS Registration) plus a per-subnet dnsregpolicy. When off, pfSense generates the kea2unbound hook with no --subnet-id args, which registers nothing — silently. Leases sync into /var/unbound/leases/leases4.conf via /cf/conf/kea4_scripts.d/kea2unbound.sh.

PXE boot chain

  1. A booting client's DHCP request hits Kea, which hands out PXE options (next-server, boot filename) alongside the lease.
  2. The client fetches its bootloader over TFTP from TrueNAS, not from pfSense — pfSense's Kea PXE options simply point at TrueNAS's TFTP service (tftpd-hpa, a TrueNAS SCALE community app; see TrueNAS Services).
  3. TrueNAS's iPXE menu (/mnt/share/tftpboot/pxe/menu.ipxe) presents a boot menu — pick an OS/installer and it chains into the matching kickstart/autoinstall config, also served from TrueNAS (an HTTP server on the TFTP box for Ubuntu autoinstall seed files, for example).
  4. This is what makes bare-metal ESXi installs (and Ubuntu VM installs) a "walk up, pick from a menu, walk away" experience rather than a USB-stick affair.

TFTP survives; hand-rolled systemd units don't

The TFTP app is middleware-managed and survives a TrueNAS version upgrade. Anything you set up by hand with a raw systemd unit (like the routing-PBR fix) does not — see config/truenas/asymmetric-routing-fix.md's upgrade caveat.

Netboot options live per-subnet in Kea

TFTP server address and "Network Booting" fields (which generate the option-93 arch client-classes PXE clients send) are configured per-subnet in Kea, not globally — if PXE stops working for one VLAN but not another after a migration, check that subnet's netboot settings specifically before assuming it's a TFTP-server problem.

Physical Windows machines: PXE + unattended install + Setup.ps1

The same "walk up, pick from a menu, walk away" experience extends to physical Windows machines, not just VM/ESXi installs — via tftpboot/autounattend.xml (a standard Windows unattended-install answer file) plus a <FirstLogonCommands> block that chains straight into Developer Laptop Setup's Setup.ps1:

  1. Windows Setup boots via PXE/WinPE and installs from the answer file, auto-logging in as psharma once (<AutoLogon><LogonCount>1</LogonCount>).
  2. <FirstLogonCommands> runs, in that real interactive session: maps the NAS share (net use Z: \\<nas>\share ... /persistent:yes), then runs Setup.ps1 with no -PostReboot flag — its documented physical-machine entry point, which registers its own RunOnce key and drives the reboot → Phase 2 continuation itself, identical to the terraform remote-exec flow used for VMs. The mid-script reboot interrupting FirstLogonCommands "ungracefully" is expected.

Never commit a real plaintext password to autounattend.xml. The committed file is autounattend.xml.template with an __ADMIN_PASSWORD__ placeholder in all 4 spots (Administrator password, psharma local-account password, AutoLogon password, and the net use command). generate-autounattend.sh pulls vm_admin_password from Vault (devhome/vmware — see Vault Consumers), substitutes it in with sed, and writes the real autounattend.xml (gitignored, chmod 600). Re-run the generator before each PXE deploy, or whenever the Vault password rotates. It needs an interactive-equivalent shell (zsh -lic, not plain bash -c) so the shell profile's Vault auth actually loads before the script runs.