JWT vs Opaque Tokens for LLM Gateways: Choose the Failure Model
JWT vs opaque tokens for LLM gateways is a choice between local verification and authorization-server lookup, with different freshness, privacy, latency, and outage behavior. This comparison maps both token forms to authenticated user and agent requests, shows where revocation state lives, defines failure and caching tests, and gives security teams a practical decision record for selecting one format or operating both.

A JWT reaches an LLM gateway carrying verifiable claims and a signature. An opaque token reaches the same route as a handle whose authorization context lives at the authorization server. Both can represent the same subject and audience, with the same scope and expiry. Their operational behavior diverges at the first validation step.
The JWT verifier needs issuer-bound signing material and a token profile. The opaque-token verifier needs a protected lookup or introspection service. That choice determines which system sits on the request path and how quickly revocation takes effect. It also determines which identity data travels through clients and what happens when an upstream service fails.
TL;DR
- Choose JWT access tokens when local verification and distributed availability fit the route. Then manage explicit signed claims and signing-material rotation, with a defined revocation window.
- Choose opaque tokens when the authorization server should retain token context and answer current state. Then engineer introspection latency and availability.
- Compare complete validation profiles rather than token appearance. Either format still needs issuer and audience checks, plus scope, time, and principal checks.
- Many LLM gateways can accept both through separate issuer profiles and normalize only validated attributes into one local identity object.
OAuth permits handles and self-contained tokens
RFC 6749 Section 1.4 defines an access token as a string representing authorization granted to a client. The string is usually opaque to the client. It may act as an identifier used to retrieve authorization data, or it may carry that data in a verifiable form.
That distinction creates the two common gateway paths. An opaque token is a handle. The resource server asks a trusted authorization service for its state and context. A JWT is a structured, signed token. The resource server can verify it locally when it has the issuer's signing material and implements the accepted profile.
Token form alone grants no route authority. The LLM gateway still needs a trusted issuer relationship and intended audience. It also needs required scope, time validity, subject mapping, and local policy for the requested model. A valid access token can reach a policy denial because the principal lacks permission for that model or region, or for that tenant or prompt data class.
The identity-aware AI gateway guide covers that post-authentication policy boundary. This comparison stays with the credential and verifier behavior immediately before it.
JWTs move validation data to the gateway
A JWT access token contains protected claims that the gateway can parse and verify. RFC 9068 defines an OAuth 2.0 JWT access-token profile. It requires signed tokens with explicit at+jwt typing. It also requires issuer and audience claims. Expiration and a validation procedure that uses trusted metadata and signing material are required as well.
Local verification removes a per-request authorization-server lookup. The gateway selects a trusted profile and retrieves its issuer-bound JWK Set material. It then verifies the allowed algorithm and signature before validating token type and issuer. It also validates audience, time claims, and required authorization claims. That path can continue during a brief authorization-server outage when the verifier still has valid JWK material and the token remains inside its acceptance period.
State freshness is the main trade. A self-contained token can remain cryptographically valid after the authorization server changes account state or records revocation. Short token lifetimes reduce that window, while a revocation feed or denylist adds server-side state. An introspection check is another option, but each addition changes the original local-verification failure model.
JWTs also expose their readable claims to any party holding the token unless the deployment adds encryption. A signed JWT protects integrity, while its base64url payload remains inspectable. Keep the claim set limited to fields the resource server needs. Avoid placing prompt content or sensitive profile data in the token. Provider credentials and internal policy details should remain outside it as well.
The LLM JWT issuer validation guide describes the issuer binding required before those claims become trusted input.
Opaque tokens keep context at the authorization server
An opaque access token gives the client and gateway no locally meaningful claim set. The gateway presents the token to the authorization server's protected lookup interface. RFC 7662 standardizes token introspection: an authenticated protected resource sends the token by HTTP POST and receives a JSON result whose one required member is active.
The authorization server can evaluate current state at lookup time. Its checks may include expiry and start time, plus revocation, resource-server applicability, and any deployment-specific state. The response can also include scope, client ID, subject, and audience. It may include issuer, token type, and time fields as well. The gateway validates the subset required by its route contract before resolving a principal.
This path gives the authorization server direct control over the active decision and keeps authorization context away from the client-facing token string. It also makes introspection latency and availability part of the LLM authentication path. Every uncached request depends on network access and endpoint authentication. It also depends on TLS validation, response parsing, and authorization-server capacity.
Caching reduces that dependency at the cost of freshness. RFC 7662 warns that an active response can become stale after token revocation and prohibits caching beyond exp when the response supplies it. The chosen cache duration is therefore part of the security decision, not a generic performance knob.
The active token introspection guide provides the complete negative-test pack for this route.
The useful comparison is operational
A security review should compare properties that appear in failure and acceptance evidence:
- Validation location: JWTs support local cryptographic and claim validation at the gateway. Opaque tokens require a trusted server lookup to recover state and context.
- Revocation freshness: JWT freshness normally follows token lifetime unless the deployment adds state. Opaque tokens can reflect authorization-server revocation on the next uncached lookup.
- Request-path dependency and offline operation: JWT validation depends on cached issuer metadata and JWK material. It can continue inside the accepted token and JWK window. Opaque validation depends on an available introspection or lookup endpoint for every cache miss and lacks current context when that service is unavailable.
- Signing operations: JWT verifiers need issuer-bound signing material and a rotation protocol. Opaque-token resource servers primarily protect their introspection client identity. The authorization server still manages its internal token protections.
- Privacy at the client: JWT payload claims are readable to a token holder unless encrypted. Opaque strings keep those claims at the server, though introspection responses still require minimization and access control.
- Gateway latency: JWT verification uses local cryptographic work after JWK retrieval. Opaque verification adds a network call on an uncached request.
- Cache risk: JWT JWK caches can become stale during rotation. Opaque result caches can preserve an active decision after revocation.
- Troubleshooting evidence: JWT events can name profile and issuer, along with
kid, algorithm, and claim outcomes. Opaque events can name endpoint policy and cache state, along with the active result and validated response fields. - Blast radius: Removing JWT verification material can invalidate every token signed by it. Disabling an opaque handle can target one token when the authorization server supports that state.
I would reject any comparison that labels JWTs stateless and stops there. The gateway still holds trusted issuer configuration, cached JWK material, and clock policy. Revocation exceptions are often present in the same verifier. The operational state simply sits in different places.
Revocation exposes the core difference
RFC 7009 Section 3 describes the contrast directly. A self-contained token lets a resource server make an authorization decision without contacting the authorization server. A handle requires the resource server to retrieve authorization data. The handle model lets server-side revocation appear when the resource server relays the token, while immediate revocation for a self-contained token needs another backend mechanism or a short token lifetime.
For an LLM route, define the maximum acceptable time between revocation and enforcement. Then test the selected design against that objective. A ten-minute JWT with local verification can present a revocation window approaching its remaining lifetime. Caching an active opaque-token result creates another stale window. Per-request opaque lookup can narrow that interval while adding synchronous availability risk.
Avoid treating signing-material rotation as targeted revocation. Retiring a JWK can invalidate a population of JWTs sharing that material and may create an authentication outage when valid tokens remain. The JWKS rotation guide covers the overlap and replica evidence required for planned changes.
Failure policy belongs in the decision record
Write the failure matrix before selecting the token format. For JWTs, include metadata retrieval failure and unknown kid, plus stale JWK Set and unsupported algorithm. Include invalid signature, wrong type, issuer mismatch, and audience mismatch. Also cover expiry and clock error. State when the last known valid JWK set remains usable and for how long.
For opaque tokens, include DNS and TLS failure, introspection timeout, invalid endpoint credential, and HTTP 429. Include server error, malformed response, missing active, and inactive result. Also cover wrong audience, missing scope, and stale cache. Separate an inactive token from an unavailable introspection service. They lead to the same closed route result but carry different operational meaning.
Set bounded parsing and token size limits for either path. Public responses should avoid disclosing accepted issuers or JWK identifiers. They should also avoid account status and internal policy. Protected events need a token digest rather than the raw bearer credential. The final proof is the same: authentication failure creates no provider request ID.
The physical review artifact should fit on one page, with two vertical paths ending at the same local principal object. Red crosses mark every failure before the model adapter. If one path has a side arrow around the verifier, the comparison has uncovered a bypass rather than a token-format difference.
Privacy and disclosure differ by layer
A JWT can carry subject and client, plus scope and groups. It may also carry roles, tenant, and other claims. Every holder can inspect unencrypted payload fields. Minimize claims to the attributes the resource server needs and use pairwise or service-specific identifiers where appropriate. Keep dynamic policy data at the gateway or authorization server.
An opaque token keeps the client-facing string free of readable authorization context. The introspection response can still reveal privacy-sensitive information. RFC 7662 allows an authorization server to return different response detail to different protected resources and recommends measures that prevent disclosure to unintended parties. Give the LLM gateway a narrow introspection permission and copy only approved fields into the local identity object.
Audit storage needs the same restraint. A token digest and issuer policy, with the validation method and principal reference, are usually enough when combined with the route, model, policy version, and result to reconstruct the decision. Raw access tokens create replay risk. Full JWT payloads and introspection responses can accumulate identity attributes unrelated to the LLM decision.
One gateway can support both formats safely
A migration or multi-issuer environment may need both, using separate validation profiles. The JWT profile defines the exact issuer and the metadata and JWK Set locations. It also defines allowed algorithms, token type, audience, and required claims. The opaque profile defines the introspection endpoint and endpoint-authentication method. It also defines expected issuer and audience, required response fields, cache policy, and failure result.
Dispatch between profiles using trusted configuration and an unambiguous mechanism defined by the deployment. Avoid guessing token format from the presence of two dots and then applying permissive fallbacks. A malformed JWT should fail its selected JWT profile rather than falling through to opaque introspection. Mutually exclusive rules reduce token-substitution risk.
After validation, normalize only established attributes into a local principal object. Preserve the validation method and issuer policy. Also preserve token class and authentication time. Keep the correlation reference as well. Route policy can then evaluate one stable identity structure while audit records still show how that identity was established.
Acceptance tests should use matched identities
Build fixtures for the same synthetic subject and client, with the same audience, scope, and expiry under both token forms. Send them through the deployed load balancer to the same model route. Compare outcomes and evidence rather than raw latency alone.
The release pack should include:
- A valid JWT and valid opaque token that resolve to the same local principal and reach the same route decision.
- Wrong-audience and missing-scope fixtures for both forms.
- JWT failures for unknown JWK and stale JWK cache, plus invalid signature and wrong type. Add expiry and issuer mismatch.
- Opaque failures for inactive state and endpoint timeout, plus invalid endpoint credential and malformed response. Add cache expiry and wrong response audience.
- Revocation tests measuring the actual enforcement window under the selected JWT lifetime and opaque-response cache duration.
- Replica and rolling-restart tests that expose process-local JWK or introspection caches.
- Provider-side evidence proving that every authentication failure stopped before model dispatch.
- A bypass inventory covering direct provider routes and local inference, plus alternate hosts and any route that skips the selected verifier.
Record the chosen token profiles and their owners. The authorization team owns issuance and revocation semantics. It also owns metadata and introspection. The gateway authentication team owns verification and caching. It also owns failure behavior and principal resolution. The LLM policy team owns model and prompt decisions after authentication.
DeepInspect
DeepInspect can consume validated identity context for authenticated user or agent requests deliberately routed through its HTTP path to an LLM endpoint. The later policy decision can use the resolved principal and model destination, along with the route, tenant, and prompt classification. It can then produce an independent allow or deny record before provider dispatch.
The upstream authorization and resource-server layers own JWT verification or opaque-token introspection. They also own signing-material rotation and revocation semantics, along with cache freshness and endpoint availability. DeepInspect should retain the validation method and correlation reference without claiming to establish token state itself. Local inference and direct provider calls remain outside that policy evidence. Bypass routes remain outside it as well.
Book a technical deep dive at deepinspect.ai.
Frequently asked questions
- Are opaque tokens more secure than JWTs for LLM gateways?
Neither format wins across every property. Opaque tokens centralize current state and hide authorization context from token holders, while adding a lookup dependency. JWTs support local verification and distributed availability, while requiring disciplined claims and signing-material rotation, plus a revocation strategy. Choose against the route's freshness and privacy requirements, as well as its latency and outage requirements.
- Can a JWT use token introspection?
Yes. RFC 7662 can introspect structured tokens when the authorization server understands and validates them. A deployment may use local JWT validation for ordinary requests and introspection when current revocation state is required. That hybrid adds another dependency, so document which decision is authoritative and test its failure behavior.
- Do opaque tokens remove signing-credential rotation work?
They remove JWK Set verification from the resource server's normal token path. The authorization server still protects token issuance and storage, and the gateway must rotate the credential used to authenticate to introspection. If the introspection response itself uses a signed structure, additional signing-material management may return at that layer.
- Which token format gives faster revocation?
An uncached opaque-token lookup can reflect authorization-server state on the next request. A locally verified JWT normally remains acceptable until expiry unless the deployment adds a denylist or event feed. Introspection can also supply that state. Cached opaque responses create a stale interval too. Measure the complete design rather than assigning freshness to the format alone.
- Should a gateway convert opaque tokens into JWTs?
Token exchange can be valid when a defined authorization service performs it under an explicit profile. A gateway should avoid inventing a new signed token merely to simplify downstream code. That creates an issuer and signing lifecycle. It also creates an audience contract and expiry policy. A new revocation window follows. Normalize validated identity attributes locally when no downstream token is required.