Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

kubectl: You must be logged in to the server (Unauthorized)

A 401 is authentication, not RBAC. Read the kubeconfig block, check token expiry, refresh provider credentials, and tell it apart from a stale CA or a stripped header.

~5 min readKubernetes
#Kubernetes#kubectl#Authentication#Troubleshooting
3D diagram of an API server gate where an expired credential card is rejected and a bright one enters, headline 401 UNAUTHORIZED

Introduction & Quick Summary

Symptom. Every request fails, including ones that worked an hour ago:

error: You must be logged in to the server (Unauthorized)

or, from a client library, (401) Unauthorized with the server has asked for the client to provide credentials.

Cause. The API server rejected your credentials, which is an authentication failure (HTTP 401), not an authorization one. The usual sources: an expired bearer token (EKS tokens live 15 minutes, GKE and AKS tokens about an hour), an expired or missing client certificate, a credential plugin that is not installed or cannot find its own credentials, or a kubeconfig copied from somewhere else that points at a cluster whose CA changed.

Fix. Find which authentication method the context uses, verify that credential's expiry, and refresh it. kubectl config view --minify and the token cache are the two places to look; --v=8 shows the request headers if the message is not enough.

Core Mechanics, Signatures & Reproducible Setup

A 401 means the server did not accept the identity. A 403 means it accepted the identity and refused the action:

OutputStatusMeaningFix
You must be logged in to the server (Unauthorized)401Bad, missing or expired credentialsRefresh the token, cert or plugin
... is forbidden: User "x" cannot list resource ...403Authenticated, not allowedRBAC role or role binding
Unable to connect to the server: x509: certificate signed by unknown authorityTLSWrong CA, usually a recreated clusterRefresh kubeconfig
Unable to connect to the server: dial tcp ... i/o timeoutnoneNetwork, VPN, private endpointReachability first

Find out how the current context authenticates before touching anything:

bash
kubectl config current-context kubectl config view --minify -o jsonpath='{.users[0].user}{"\n"}' # {"client-certificate-data":"...","client-key-data":"..."} -> certificate auth # {"token":"eyJhbGciOi..."} -> static bearer token # {"exec":{"command":"aws","args":["eks","get-token",...]}} -> exec plugin

Then confirm the identity the server sees, which only works when the credentials are still valid:

bash
kubectl auth whoami # ATTRIBUTE VALUE # Username arn:aws:eks:eu-west-1:123456789012:cluster/prod # Groups [system:masters system:authenticated]

kubectl auth whoami is a 1.27+ command backed by the SelfSubjectReview API; on older clusters use kubectl auth can-i --list and read the User line.

Key Comparison Table

The common kubeconfig authentication blocks and how each expires:

MethodBlockExpiresRefresh
Client certificateclient-certificate-dataMonths to one yearRegenerate kubeconfig (kubeadm, provider CLI)
Static tokentoken:Whatever issued it; often 24 hoursReissue; do not paste tokens into files
EKSexec with aws eks get-token~15 minutesPlugin calls AWS each time; needs valid AWS credentials
GKEexec with gke-gcloud-auth-plugin~1 hourgcloud auth login, plugin must be on PATH
AKSexec with kubelogin~1 hourkubelogin convert-kubeconfig -l azurecli
OIDCauth-provider (legacy) or execToken lifetime of the IdPkubelogin or a custom exec plugin

Refresh commands by provider:

ProviderCommand
EKSaws eks update-kubeconfig --name <cluster> --region <region> --profile <profile>
GKEgcloud container clusters get-credentials <cluster> --zone <zone> --project <project>
AKSaz aks get-credentials -g <rg> -n <cluster> --admin or kubelogin convert-kubeconfig -l azurecli
kubeadm / self-managedcopy the refreshed admin.conf, or kubeadm init phase kubeconfig admin

Practical Edge Cases, Error Fixes & Deep Dive Code Examples

1. Is the token actually expired?

A JWT bearer token carries its own exp. Decode it instead of guessing:

bash
kubectl config view --minify -o jsonpath='{.users[0].user.token}' | \ cut -d. -f2 | tr '_-' '/+' | base64 -d 2>/dev/null | jq '{iss, sub, exp}'

Or paste the token into the JWT decoder to read exp as a date alongside the issuer. For exec plugins the token is not in the kubeconfig:

bash
aws eks get-token --cluster-name prod --region eu-west-1 | jq -r '.status.token' | \ cut -d. -f2 | tr '_-' '/+' | base64 -d 2>/dev/null | jq '{exp}'

If exp is in the past, the message is explained. If it is in the future, the server rejected a valid token for another reason — wrong cluster, revoked role, or a proxy in between.

2. EKS: the plugin fails silently

aws eks get-token needs valid AWS credentials of its own, and its failure surfaces as a 401 because kubectl only sees "no token". Two checks:

bash
aws sts get-caller-identity # is there an identity at all? kubectl config view --minify -o jsonpath='{.users[0].user.exec.env}{"\n"}' # AWS_PROFILE overrides

An exec block with env: [{"name":"AWS_PROFILE","value":"old-account"}] overrides the profile in your shell, and a --region that does not match the cluster produces an authentication error rather than a "wrong cluster" message. Re-running aws eks update-kubeconfig with explicit --region and --profile rewrites the block and is usually faster than editing it.

3. GKE and AKS: the missing plugin

GKE clusters created after 1.26 use gke-gcloud-auth-plugin, and its absence shows up as an exec failure or a 401 depending on kubectl's version:

bash
gcloud components install gke-gcloud-auth-plugin export USE_GKE_GCLOUD_AUTH_PLUGIN=True gcloud container clusters get-credentials <cluster> --zone <zone>

For AKS, the kubelogin plugin must be on PATH, and the kubeconfig must be converted to use it:

bash
kubelogin convert-kubeconfig -l azurecli kubectl get nodes

A CI runner that has az but not kubelogin authenticates nothing; the error is the same 401 as an expired token.

4. Cluster recreated, kubeconfig stale

If the cluster was deleted and recreated with the same name, the endpoint may be the same and the CA different. Then the failure is usually TLS (x509), but a 401 appears when the server's certificate is trusted through a shared wildcard while the client certificate belongs to the old CA. Regenerate rather than patch:

bash
kubectl config delete-context old-prod aws eks update-kubeconfig --name prod --region eu-west-1 --alias prod kubectl config get-contexts

Keeping an --alias per cluster makes the stale-context mistake visible; without it, two kubeconfig files merge and the first one wins.

5. Tokens pasted from a dashboard

A token copied from a web console or a kubeadm token create output is a bearer token with a short lifetime, and pasting it into ~/.kube/config creates a file that works for a day and then fails with this error. The same applies to service account tokens mounted into pods: a projected token expires (one hour by default) and a client that caches it forever gets 401s.

# projected service account token with an explicit audience and expiry volumes: - name: token projected: sources: - serviceAccountToken: path: token expirationSeconds: 3600

In-cluster clients should re-read the file, or use a client library that does; a long-lived Secret token still works on current versions only where it was explicitly created.

6. When a proxy strips the header

Behind an authenticating proxy, an ingress, or a service mesh that terminates and re-originates traffic, the Authorization header can be dropped before it reaches the API server. The request arrives without credentials and the server answers 401 with no hint:

bash
kubectl get nodes --v=8 2>&1 | grep -E "Authorization|GET https|Response Status"

With --v=8 the request line shows the URL and the response shows 401 Unauthorized; if the same request works with curl and the token attached (see the curl reference for the -H form), the header is being stripped in between.

Common Compilation/Runtime Mistakes & Version Gotchas

  • Reading 401 as a permissions problem. Adding RBAC bindings to an unauthenticated request changes nothing; check kubectl auth whoami first.
  • export KUBECONFIG=a:b. The first file that defines a context wins, so a stale file earlier in the list silently overrides the one you updated.
  • Editing ~/.kube/config by hand for a provider cluster. The exec block has provider-specific arguments; regenerate with the provider CLI.
  • Cached tokens in the exec plugin. Some plugins cache to ~/.kube/cache; a deleted cache fixes a token that the provider says is valid.
  • Clock skew on the workstation or node. A token nbf in the future is invalid; a VM that resumed from suspend can be minutes off. Check date -u against the API server's Date header.
  • Impersonation headers left in a wrapper script. --as=system:serviceaccount:... requires the impersonating identity to have permission, and a 401 for the impersonated user is reported against the wrapper.
  • Interactive prompts in CI. An exec plugin that wants a browser or a device code blocks or fails; CI needs a non-interactive credential (service account, workload identity, or a converted kubeconfig).
  • Legacy auth-provider blocks. OIDC providers configured under auth-provider were removed from client-go in recent versions; the context must move to an exec plugin such as kubelogin or oidc-login.

Related tools. Paste the failing bearer token into the JWT decoder to read exp, iss and the audience without sending it anywhere, and keep the exact curl request in the ticket with the curl reference — a request that works with curl and fails with kubectl narrows the problem to the kubeconfig in one step.

Official references

primary documentation

The Kubernetes documentation that defines the behaviour this guide explains.

Try matching browser tools

All tools

Small utilities that fit Kubernetes work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Kubernetes guides

More in Kubernetes

All Kubernetes guides

The guides filed next to this one in Kubernetes, in library order.