← Blog

JWKS Rotation for LLM Gateways Without Authentication Outages

Parminder Singh
Parminder Singh··8 min read
Summarize with AI

JWKS rotation without LLM gateway outages depends on a controlled overlap between old and new signing keys, issuer-bound retrieval, bounded cache refresh, and tests that cross every gateway replica. This guide defines the rotation state machine, unknown-key behavior, cache and outage policy, observability fields, rollback conditions, and release evidence for authenticated user and agent requests routed to LLM endpoints.

Platform & Architectureai-securityllm-securityidentity-and-authorizationzero-trustarchitectureaudit
JWKS Rotation for LLM Gateways Without Authentication Outages

Signing material changes at the authorization server. One LLM gateway replica still holds the old JWK Set, another has fetched the new set, and a third has lost outbound access to the metadata host. The first user request after deployment lands on replica three with a JWT whose kid names the new signing material. A rotation that looked complete in the identity console has become an authentication outage at the model route.

JWKS rotation without LLM gateway outages requires a deployment protocol around the standards. The verifier needs trusted issuer metadata, a bounded cache policy, an overlap period, deterministic behavior for an unknown kid, and production evidence across every replica. I would stop a release if the evidence showed only one successful token on one gateway process.

TL;DR

  • Bind each JWK Set URL to a trusted issuer record. Never discover a signing-material location from an untrusted token header.
  • Publish the new verification material before issuing tokens that reference it, then retain the retiring material until its accepted token population has expired.
  • Treat an unknown kid as a bounded refresh event, with request coalescing, rate limits, and a closed result if the required material remains absent.
  • Test rotation through the deployed load balancer, including stale replicas, metadata failure, rollback, and direct-route bypass.

Rotation changes a verifier state machine

RFC 7517 Section 4.5 defines kid as a case-sensitive identifier used to select verification material from a JWK Set, including during rollover. The RFC recommends distinct identifiers for different entries inside one set, while leaving the identifier structure unspecified. A gateway therefore treats kid as a selector within an already trusted issuer context. The value carries no authority by itself.

The operational state machine has four useful states. In steady state, current tokens verify under the current set. During introduction, the new public material appears in the set while the authorization server continues signing with the old private material. During active overlap, both public entries remain available and the issuer starts signing new tokens with the new material. Retirement begins only after the last token accepted under the old material has passed its validity horizon, including the verifier's configured clock-skew allowance.

The standards define the data structures and trust bindings. They leave publication timing, overlap length, refresh cadence, and rollback policy to the deployment. Put those choices in a versioned rotation plan rather than assuming a library default will fit the LLM route.

Retrieval begins with a trusted issuer

RFC 8414 Section 2 defines jwks_uri as an HTTPS URL for the authorization server's JWK Set. Section 3.3 of that RFC requires the metadata issuer value to match the issuer identifier used to retrieve that metadata. RFC 8725 Section 3.8 then requires an application to bind the cryptographic material used for a JWT to its issuer.

Build one trusted configuration record per accepted issuer. Include the exact issuer identifier, metadata URL, approved JWK Set URL, accepted token profile, algorithms, expected audience, and tenant or policy domain. The JWT's iss claim may select among those configured records after normal parsing limits apply. An unknown issuer receives a closed authentication result without causing metadata discovery.

Token-controlled jku and x5u headers deserve the same boundary. RFC 8725 Section 3.10 warns that blindly following those URLs can create server-side request forgery. Match any permitted location to trusted configuration before retrieval. A rotation runbook that repairs unknown identifiers by fetching arbitrary token URLs trades an outage risk for an attacker-controlled network request.

The LLM JWT issuer validation guide owns the complete issuer-to-verification-material trust chain. Rotation operates inside that established chain.

The overlap window protects in-flight tokens

Publish the new public material before the authorization server signs a token with the corresponding private material. Then verify that every gateway replica can retrieve and parse the set containing both entries. Only after that proof should the issuer switch its active signing material.

Keep the retiring public material available while a valid token signed under it can still reach the resource. The calculation belongs to the token contract: maximum token lifetime, maximum issuance delay, clock skew, queue or retry behavior, and any accepted grace period. I prefer a recorded retirement timestamp derived from that contract over a round number copied into a change ticket. The timestamp tells an incident responder exactly which tokens may still depend on the retiring material.

A blue-green authorization-server deployment needs the same discipline. Old and new issuer instances can sign concurrently during rollout. Their public verification material must coexist until traffic and token validity prove the older signer is finished. Removing the old JWK immediately after the final old instance stops can still reject tokens it issued moments earlier.

The visual check is simple: two JWK rows should remain on the identity console during overlap, and the gateway evidence should show successful validation under each kid. One row disappears only after the old-token fixture fails on time validity rather than missing-material lookup.

Cache behavior decides the outage radius

A JWK Set cache reduces metadata traffic and shields the authentication path from brief upstream faults, but each replica can hold a different version. Define the cache as part of the verifier, with a known refresh schedule, maximum staleness policy, last-success timestamp, set fingerprint, and issuer association.

An unknown kid from a known issuer can trigger one controlled refresh. Coalesce concurrent requests so a burst of tokens produces one upstream retrieval rather than a request storm. Apply an outbound timeout and rate limit. Parse the new document under strict JSON limits, reject malformed verification material, and confirm the retrieved set still belongs to the configured issuer record. Retry policy needs a bound too.

If the refreshed set still lacks the required entry, reject the token under a stable reason such as signing_key_unavailable. Avoid looping through refreshes for every request. Negative caching can dampen a burst, but its duration should remain short enough to permit a legitimate publication to become visible. Record the selected duration and its owner in the verifier policy.

The last known valid set can preserve validation for material already trusted during a metadata outage. This is an availability choice with a security boundary. Give it a maximum age and an operator-visible state. Never extend token validity, bypass signature checks, or accept new material merely because retrieval failed. The gateway should distinguish a token whose verification material is absent from a token whose signature failed under a present JWK.

Replica tests must use the production path

A unit test with an in-memory JWK Set proves parsing. The release gate needs the deployed topology: external load balancer, gateway replicas, metadata egress, cache layer, issuer, policy engine, LLM adapter, and audit sink.

Run the following sequence with synthetic tokens and a non-production model route:

  1. Publish JWK A and issue a token signed by A. Route repeated requests across every gateway replica.
  2. Publish JWK B beside A while continuing to sign with A. Confirm each replica records the same set fingerprint or the expected cache version.
  3. Start signing new tokens with B. Send A-signed and B-signed tokens concurrently through the load balancer.
  4. Force one replica to retain its pre-rotation cache. Confirm the B token triggers the bounded refresh and then passes.
  5. Block metadata egress for one replica. Confirm A continues under the documented last-known-set policy and B receives the documented closed result until retrieval recovers.
  6. Restore retrieval, confirm a single refresh wave, then remove A only after its token acceptance horizon closes.
  7. Reintroduce an A-signed token. It should fail on expiry or retired-material policy according to the test timing, without reaching the LLM provider.

Repeat the sequence during a rolling gateway restart. A process-local cache may begin empty while a shared cache carries an older set. The test should expose both conditions. The API credential rotation runbook covers provider credentials; JWKS rotation concerns inbound token verification and needs separate evidence.

Failure records need enough detail to locate the stale layer

A public authentication response should reveal little about accepted issuers and verification material. A protected event can be precise. Record the issuer policy ID, observed kid, token fingerprint, algorithm, JWK Set fingerprint, cache age, refresh attempt and outcome, verifier reason, gateway replica, route, and final dispatch state. Exclude raw bearer tokens and private signing material.

Separate these reasons:

  • issuer_untrusted for a value outside configured trust.
  • kid_missing when the selected token profile requires an identifier.
  • kid_unknown_before_refresh for the initial cache miss.
  • kid_unknown_after_refresh when the trusted source still lacks the required entry.
  • jwks_retrieval_failed when the trusted source cannot be reached or validated.
  • signature_invalid when selected verification material fails verification.
  • token_expired when cryptography passes but the acceptance period has ended.

That taxonomy lets an operator see a stale cache instead of treating every failure as a bad token. The AI audit log chain-of-custody guide describes how to retain the joined authentication and route-decision records without turning the event store into a credential repository.

The release packet should prove rollback too

Rotation rollback has two distinct cases. Before B signs any token, remove B from the set and return to the previous steady state. After B has signed tokens, keep B published while switching issuance back to A. Removing B at that point strands valid B-signed tokens and recreates the outage.

Capture the approved rotation plan and trusted issuer record. Preserve the JWK Set fingerprints before the change, during overlap, and after retirement, along with timestamps for publication and the signing switch. Add per-replica cache evidence and synthetic requests under A and B. Include metadata failure, malformed-set, duplicate-identifier, and rollback results. RFC 7517 recommends distinct kid values, so the gate should flag duplicate values for operator review even if a particular library can disambiguate by JWK type.

The release fails when any replica accepts a token under verification material absent from the trusted issuer record. It also fails when a valid overlap token reaches a missing-material result, when unknown issuers trigger outbound retrieval, or when authentication failure still produces a provider request ID.

DeepInspect

DeepInspect can evaluate authenticated user or agent requests deliberately routed through its HTTP path to an LLM endpoint. It can apply identity-aware policy to the resolved principal, requested model, route, and prompt classification, then produce a per-decision record before model dispatch.

The authorization server and upstream resource-server verifier still own JWKS publication, retrieval, caching, signature checks, and rotation state. DeepInspect should receive identity context only after that chain passes. A correlation reference can join the verifier's selection event to DeepInspect's later policy decision. Direct provider calls, local inference, and traffic that bypasses the routed HTTP path remain outside that evidence chain.

Book a technical deep dive at deepinspect.ai.

Frequently asked questions

How long should two JWKS signing entries overlap?

The overlap should cover the complete acceptance horizon for tokens signed by the retiring material. Derive it from the maximum token lifetime, delayed issuance, accepted clock skew, and any queue or retry behavior in the calling path. The RFCs leave a numeric duration to the deployment. Record a retirement timestamp tied to the token contract, then prove that a token signed with the retiring material remains verifiable until time validation closes its acceptance window.

Should an unknown kid trigger a JWKS refresh?

A known issuer can use one bounded refresh as recovery for legitimate rotation. Coalesce concurrent misses, enforce timeouts and rate limits, and retrieve only the issuer's configured JWK Set URL. If the refreshed set still lacks the required entry, reject the token and apply a short, documented negative-cache rule. An unknown issuer should trigger no retrieval.

Can a gateway keep using its last valid JWK Set during an outage?

A deployment may retain a last known valid set for a bounded period so already trusted material continues to verify. The policy needs a maximum set age, monitoring, and a closed result for entries absent from that set. It must preserve signature, issuer, audience, token-type, and time validation. This availability policy cannot create trust in new material.

Does JWKS rotation revoke old access tokens?

Removing verification material makes tokens depending on it unverifiable, but it is a blunt mechanism that affects every such token. OAuth token revocation and token expiry carry different semantics. A planned rotation normally retains the retiring verification material until its accepted tokens expire. Use the authorization server's revocation controls for targeted invalidation.

Which component owns JWKS rotation for an LLM route?

The authorization server owns signing-material generation, publication, signing activation, and retirement. The resource server or authentication gateway owns trusted metadata, JWK Set retrieval, caching, signature validation, and failure behavior. LLM route policy begins after that verifier has produced authenticated identity context.