Vault Token Renewal Failing: Fixing "token Is Not Renewable" And TTL Expiry Issues
Vault Token Renewal Failing: Fixing "token is not renewable" and TTL Expiry Issues
If you've ever stared at a production alert screaming token is not renewable at 2 AM, you know the particular joy of Vault token management. These errors tend to surface at the worst possible time — right when an application loses access to secrets and starts throwing 403s everywhere.
Let me walk you through why this happens, how to diagnose it quickly, and how to fix it properly so it doesn't bite you again.
Understanding Vault Token Lifecycle
Before fixing anything, you need to understand how Vault tokens actually work. Every token in Vault has a few key properties:
- TTL (Time To Live): How long until the token expires
- Max TTL: The absolute ceiling — even if you keep renewing, the token cannot live beyond this
- Renewable: A boolean flag that determines if renewal is even allowed
- Periodic: A special class of tokens with no max TTL (only a period)
The token is not renewable error has two distinct causes that get conflated all the time:
- The token's
renewableflag is explicitly set tofalse - The token has hit its
max_ttland Vault won't allow further renewal
These require different fixes, so diagnosis first.
Diagnosing the Problem
Start with a token lookup. This is the first thing I run:
# Look up the current token
vault token lookup
# Or look up a specific token
vault token lookup -accessor <accessor_id>
# JSON output for scripting
vault token lookup -format=json | jq '{
renewable: .data.renewable,
ttl: .data.ttl,
explicit_max_ttl: .data.explicit_max_ttl,
creation_ttl: .data.creation_ttl,
issue_time: .data.issue_time,
expire_time: .data.expire_time
}'
The output tells you everything. Look for these fields:
{
"renewable": false, // <- This is your problem if false
"ttl": 3600,
"explicit_max_ttl": 0, // 0 means "use system default"
"creation_ttl": 3600,
"expire_time": "2024-01-15T14:30:00Z"
}
If renewable is false, you've hit case #1. If it's true but renewal still fails, you're hitting the max TTL ceiling — case #2.
Fix #1: Token Created with renewable=false
Tokens get the renewable=false flag when they're explicitly created that way, or when the auth method that issued them has renewability disabled.
Check the auth method configuration:
# List auth methods
vault auth list
# Check token configuration for a specific auth method
vault read auth/token/roles/<role-name>
# For AppRole
vault read auth/approle/role/<role-name>
Check the token role settings:
vault read auth/token/roles/my-app-role
If you see renewable: false in the role definition, that's your culprit. Fix it:
vault write auth/token/roles/my-app-role \
renewable=true \
ttl=1h \
max_ttl=24h
Creating a renewable token manually:
# Explicitly renewable token
vault token create \
-renewable=true \
-ttl=1h \
-max-ttl=24h \
-policy=my-app-policy
For AppRole — make sure the role allows renewability:
vault write auth/approle/role/my-app \
token_ttl=1h \
token_max_ttl=24h \
token_renewable=true \
secret_id_ttl=0
Fix #2: Token Hit the Max TTL
This is the sneakier one. The token is renewable: true, but every time your application calls vault token renew, it gets a diminishing TTL until it hits zero and dies. The renewal "works" technically, but the TTL just keeps shrinking.
# This will fail or return a tiny TTL when approaching max_ttl
vault token renew my-token
Vault will let you renew but clamp the TTL to whatever time is left before max_ttl. When there's nothing left, you get the error.
The right fix here is periodic tokens. For long-running services, you almost certainly want a periodic token rather than fighting max TTL:
# Create a token role with a period instead of max_ttl
vault write auth/token/roles/long-running-service \
period=1h \
renewable=true \
policies=my-app-policy
Generate a token from this role:
vault token create -role=long-running-service
Periodic tokens reset their TTL to the period value on each renewal. There's no max TTL. As long as you renew before the period expires, the token lives forever.
Check and adjust system-level max TTL:
# Check current system max TTL
vault read sys/auth/token/tune
# Increase it if needed (requires root token)
vault write sys/auth/token/tune \
max_lease_ttl=720h
Implementing Token Renewal in Application Code
Here's where most applications get this wrong — they don't implement renewal at all, or they implement it badly.
Bad pattern (just use a long TTL and hope for the best):
# Don't do this
client = hvac.Client(url='https://vault:8200', token='s.static-token-with-24h-ttl')
Good pattern — renewal with a background thread:
import hvac
import threading
import time
import logging
logger = logging.getLogger(__name__)
class VaultClient:
def __init__(self, url, token, renewal_threshold=0.75):
self.client = hvac.Client(url=url, token=token)
self.renewal_threshold = renewal_threshold # Renew at 75% of TTL elapsed
self._start_renewal_thread()
def _get_token_ttl(self):
result = self.client.auth.token.lookup_self()
return result['data']['ttl']
def _renew_token(self):
try:
self.client.auth.token.renew_self()
logger.info("Vault token renewed successfully")
except hvac.exceptions.InvalidRequest as e:
logger.error(f"Token renewal failed: {e}")
# Trigger re-authentication here
self._reauthenticate()
def _renewal_loop(self):
while True:
try:
ttl = self._get_token_ttl()
# Sleep until renewal_threshold of TTL has elapsed
sleep_time = ttl * (1 - self.renewal_threshold)
time.sleep(sleep_time)
self._renew_token()
except Exception as e:
logger.error(f"Error in renewal loop: {e}")
time.sleep(30) # Back off and retry
def _start_renewal_thread(self):
thread = threading.Thread(
target=self._renewal_loop,
daemon=True,
name="vault-token-renewal"
)
thread.start()
def _reauthenticate(self):
# Re-auth via AppRole, k8s, AWS, etc.
# Implementation depends on your auth method
pass
For Go-based services using the Vault agent or SDK:
package main
import (
"context"
"log"
"time"
vault "github.com/hashicorp/vault/api"
"github.com/hashicorp/vault/api/auth/kubernetes"
)
func renewToken(ctx context.Context, client *vault.Client, secret *vault.Secret) {
if secret == nil || !secret.Auth.Renewable {
log.Println("Token is not renewable, skipping renewal loop")
return
}
watcher, err := client.NewLifetimeWatcher(&vault.LifetimeWatcherInput{
Secret: secret,
Increment: 3600, // Request 1h TTL on each renewal
})
if err != nil {
log.Fatalf("Failed to create lifetime watcher: %v", err)
}
go watcher.Start()
defer watcher.Stop()
for {
select {
case renewal := <-watcher.RenewCh():
log.Printf("Token renewed, new TTL: %d", renewal.Secret.Auth.LeaseDuration)
case err := <-watcher.DoneCh():
if err != nil {
log.Printf("Token renewal failed: %v", err)
}
// Token expired or hit max TTL — re-authenticate
log.Println("Token expired, re-authenticating...")
return
case <-ctx.Done():
return
}
}
}
The LifetimeWatcher in the Vault Go SDK is the right tool here — it handles the renewal math for you and signals when you need to re-authenticate rather than renew.
Using Vault Agent as a Sidecar
Honestly, the cleanest solution for containerized workloads is to offload this problem entirely to Vault Agent. Let it handle token lifecycle and just mount secrets into your application:
# vault-agent.hcl
auto_auth {
method "kubernetes" {
mount_path = "auth/kubernetes"
config = {
role = "my-app-role"
}
}
sink "file" {
config = {
path = "/vault/token"
}
}
}
template {
source = "/vault/templates/config.tpl"
destination = "/app/config/secrets.env"
}
Your application reads the rendered file and never touches Vault directly. Vault Agent handles authentication, token renewal, and re-authentication transparently.
Quick Diagnostic Checklist
When you hit token renewal errors, run through this:
# 1. Check renewable flag
vault token lookup -format=json | jq '.data.renewable'
# 2. Check remaining TTL vs max TTL
vault token lookup -format=json | jq '{ttl: .data.ttl, max_ttl: .data.explicit_max_ttl}'
# 3. Check if it's a periodic token
vault token lookup -format=json | jq '.data.period'
# 4. Check the auth role that issued the token
vault read auth/approle/role/<role-name> -format=json | jq '{
token_ttl: .data.token_ttl,
token_max_ttl: .data.token_max_ttl,
token_renewable: .data.token_renewable,
token_period: .data.token_period
}'
# 5. Check system-level TTL limits
vault read sys/mounts/auth/token/tune
The Bottom Line
Token renewal failures almost always trace back to one of three root causes:
- Token was created non-renewable — fix the auth role, recreate the token
- Max TTL reached — switch to periodic tokens for long-running services
- Application isn't renewing — implement proper renewal logic or use Vault Agent
For services that run indefinitely, periodic tokens are the right architecture. For short-lived workloads (CI/CD, batch jobs), a non-renewable token with an appropriate TTL is actually cleaner — just size the TTL to match your job's expected duration with some headroom.
Don't fight the token lifecycle — design around it.
Was this article helpful?
Network & Traffic Engineer
Packets don't lie. I design and troubleshoot the network layer that everything else depends on — Nginx, Envoy, HAProxy, DNS, CDNs, and everything in between. If it touches a socket, it's my problem.
Related Articles
Fix Vault 'permission denied' on Secret Read
How to diagnose and fix HashiCorp Vault 'permission denied' errors when reading secrets, including ACL policy debugging and token inspection.
Fix Vault 'Sealed' Status After Restart
How to handle HashiCorp Vault entering sealed state after a restart, including manual unseal, auto-unseal configuration, and HA considerations.
Vault Kubernetes Authentication: Injecting Secrets into Pods Without Hardcoding
Set up Vault's Kubernetes auth method so pods authenticate using their ServiceAccount tokens — then inject secrets via Vault Agent sidecar or the Vault Secrets Operator. Zero hardcoded credentials in your cluster.
HashiCorp Vault: Secrets Management from First Run to Production
How to set up and use HashiCorp Vault for secrets management — KV secrets engine, dynamic credentials, policies, AppRole authentication, and production hardening. Stop hardcoding credentials in environment variables.
Vault Dynamic Secrets: Short-Lived Credentials on Demand
Generate short-lived database credentials, AWS IAM roles, and PKI certificates with Vault dynamic secrets — eliminating long-lived credentials from your infrastructure.
HashiCorp Vault Fundamentals: Installation and First Secrets
Install HashiCorp Vault, understand seal/unseal mechanics, configure secret engines, and store your first secrets with policies and authentication methods.
More in Vault
View all →Vault with Kubernetes: Injecting Secrets into Pods
Inject HashiCorp Vault secrets into Kubernetes pods using the Agent Injector and CSI provider — with practical examples for database credentials and TLS certificates.
Vault in Production: HA, Auto-Unseal, and Disaster Recovery
Deploy Vault for production — HA with Raft storage, auto-unseal with cloud KMS, audit logging, performance tuning, backup strategies, and disaster recovery.