Skip to content

Vault Kubernetes Auth

External Secrets Operator (ESO) in each cluster uses Vault's kubernetes auth method — a ClusterSecretStore (vault-devhome) configured with kubernetes_host pointed at the cluster's API endpoint by DNS name, never a raw IP (same reasoning as kubeconfigs — see Kubernetes Troubleshooting).

When the store shows InvalidProviderConfig / permission denied

Work through these in order — don't jump straight to the last one:

  1. Config and role are probably fine. Confirm kubernetes_host really is the DNS name, not a stale IP.
  2. Verify TokenReview manually using the reviewer token from the kube-system/vault-auth-token secret, called directly against the apiserver. If this succeeds, Vault's config is objectively correct.
  3. If manual TokenReview works but Vault still 403s — without even contacting the apiserver — the kubernetes-auth plugin has a stale cached client left over from before a node's IP changed. This is not a config problem and no amount of re-checking the role/policy will find it. 
    docker restart vault   # on truenas-dr; the unsealer re-unseals automatically
  4. Then roll the ESO deployment so it re-establishes its own session: 
    kubectl -n external-secrets rollout restart deploy
    The store should go Valid within about a minute.

This exact sequence was the root cause during the 2026-07-10 dke-mgmt IP-change recovery — see the Kubernetes Troubleshooting page for the full cascade this was one step of.