Hop-by-Hop Header Stripping in AI Gateways
An AI gateway terminates one HTTP connection and creates another, so connection-specific fields from the inbound hop cannot pass through unchanged. RFC 9110 requires an intermediary to parse Connection, remove every field it names, and remove or replace Connection itself. A safe gateway also handles known connection-specific fields, protects authorization metadata from hostile nominations, and serializes a clean outbound LLM request.

An AI gateway receives an HTTP message on one connection and emits a new message on another. Connection state belongs to the first hop. Forwarding that state to an LLM adapter or provider can change framing and upgrades. It can also change persistence and trailer handling in ways the gateway never approved. The required control begins with one deceptively small field: Connection can name other fields that must disappear before forwarding.
TL;DR
- Parse
Connectionfirst and remove every field it nominates. Then remove or replaceConnectionfor the outbound hop. - Strip known connection-specific fields after applying their local semantics, including
Keep-Alive,TE,Transfer-Encoding, andUpgrade. - Reject attempts to nominate protected authorization, identity, routing, or audit fields.
- Build the provider request from a clean allowlist instead of mutating and forwarding the inbound field map.
RFC 9110 makes stripping a parser obligation
RFC 9110 Section 7.6.1 defines Connection as a comma-separated list of case-insensitive option tokens. An intermediary must parse that list before forwarding. For each token, it removes every header or trailer field with the same name. It then removes Connection itself or replaces it with options that apply to the new outbound connection.
That dynamic behavior matters. A static denylist catches familiar names and misses a field named by a future extension. The correct order is parse, validate, remove nominated fields, apply known local semantics, and serialize a fresh outbound message.
RFC 9110 also tells intermediaries to remove or replace fields known to require hop-local handling, even when they were omitted from Connection. Its examples include Proxy-Connection, Keep-Alive, TE, Transfer-Encoding, and Upgrade. The AI gateway architecture guide places this work in the proxy layer before model routing.
Dynamic nominations create a protected-field problem
A malicious client can send Connection: Authorization followed by an Authorization field. Blind compliance strips the credential before the next service. That behavior follows the mechanical removal rule, yet it can produce an authentication gap when the gateway authenticates the request and a downstream adapter assumes the credential will still be present.
The safe response is to classify fields before stripping. Maintain a protected set for application authorization, verified identity context, tenant, request correlation, signed metadata, and policy controls. If any protected name appears as a connection option, reject the request with 400 and close the inbound connection. Never reinterpret the absence downstream as anonymous access.
This is a blunt opinion: a proxy that silently strips a nominated authorization field and continues is broken. The gateway should stop at the malformed trust transition.
Keep the protected set narrow and explicit. Provider credentials should be generated by the gateway on the outbound side rather than copied from the caller. The identity-aware AI gateway architecture describes the authenticated user and agent context that belongs beside that provider credential.
A clean outbound message is safer than in-place mutation
Frameworks often expose request headers as a mutable map. Deleting a handful of keys from that map leaves duplicate-field behavior, casing, combined values, and framework-specific normalization in play. A safer gateway constructs the outbound request from an allowlist of fields whose semantics the provider adapter understands.
Start with the validated method, route, authority, content type, accepted response type, tracing identifier, and provider-specific API version. Add the gateway's provider credential. Recalculate message framing after body inspection or redaction. Generate forwarding metadata under the configured proxy-trust rule. Everything else needs an explicit adapter decision.
The physical detail is visible in a side-by-side capture. The inbound request may contain Connection: keep-alive, x-debug-hop plus X-Debug-Hop: 1. The outbound provider request contains neither field. Its connection pool can still use persistence because the gateway controls that separate socket without copying the client's Keep-Alive preferences.
This rebuilding pattern also narrows accidental disclosure. Browser cookies and internal bearer tokens never reach the model provider merely because a generic proxy library forwarded unknown fields. Debugging fields stay behind for the same reason.
Transfer-Encoding and TE require separate treatment
Transfer-Encoding describes message framing on a particular HTTP/1.1 connection. An intermediary that decodes chunked content removes or updates that field and creates valid framing for the next hop. Copying the inbound value after decoding makes the outbound message claim a coding that is gone. The HTTP/1.1 specification links incorrect handling directly to request smuggling risk.
TE advertises transfer codings the client accepts in a response and can include the trailers keyword. It is connection-specific. A gateway deciding to forward trailer capability has to know that the next hop supports it and create the correct outbound declaration for that hop. Copying the inbound list is unjustified.
Protocol translation strengthens the requirement. HTTP/2 and HTTP/3 prohibit connection-specific fields that HTTP/1.1 uses. A gateway receiving HTTP/2 and calling an HTTP/1.1 model adapter must generate HTTP/1.1 connection state itself. The reverse path requires the same discipline. The inbound protocol never grants permission to smuggle hop-local semantics through the translator.
Upgrade and tunnel behavior needs a route-level policy
Upgrade requests a protocol switch on the current connection. General-purpose proxies may support WebSocket upgrades or other tunnel modes. An authenticated LLM HTTP route should have an explicit list of permitted transports and endpoints. A surprise upgrade request deserves rejection before any provider connection is opened.
Streaming LLM responses commonly use server-sent events over ordinary HTTP semantics. SSE does not require forwarding an arbitrary Upgrade field. Treating every streaming request as a tunnel would remove the gateway's ability to inspect response events and enforce output policy. It would also prevent the gateway from producing a complete decision record. The AI gateway streaming guide covers incremental response handling without surrendering the HTTP boundary.
CONNECT deserves the same route-level decision. A generic tunnel hides the HTTP LLM messages from the enforcement layer. Keep tunnels outside the controlled AI route unless a separate inspection design explicitly terminates and parses the resulting protocol.
Header and trailer fields stay separate
RFC 9110 requires an intermediary to remove fields named by Connection from both header and trailer sections. Trailer processing happens later in the message, so the gateway needs the connection-option set available through end-of-message processing. Deleting Connection from the header map and forgetting its tokens leaves nominated trailers alive.
Store the normalized option names in request state until framing completes. Reject duplicate or malformed option values. Enforce a byte and token limit before inserting names into a set. Then apply the same protected-name policy to any trailer declaration and any received trailer field.
Headers that control framing, routing, authentication, request modifiers, response controls, or content format belong in the initial header section. RFC 9110 prohibits senders from generating them as trailers because recipients need those semantics before reading content. A gateway should reject them there rather than merging them into an earlier map.
Acceptance tests need two independent parsers
Test at the byte-facing listener and at the provider adapter. A unit test of one normalized map misses differences introduced by the ingress server. Include these cases:
- Mixed-case
Connectionoptions nominate fields with different casing; all matching fields disappear. - Multiple
Connectionfield lines combine into one option set under HTTP field rules. - A protected field such as
Authorizationis nominated; the gateway returns 400 and opens no provider request. Proxy-ConnectionandKeep-Alivearrive without nomination; local policy removes them. An un-nominatedUpgradeis removed under the same policy.- Chunked inbound content is decoded and the outbound request carries newly calculated framing.
- A nominated trailer arrives after streamed content; the gateway removes it before any downstream forwarding.
- The option list exceeds configured bytes or tokens; parsing stops with a bounded error.
Record the inbound protocol, normalized option set, removed field names, protected-field result, outbound protocol, and final LLM route. Those details make the control reviewable without retaining secrets.
DeepInspect
DeepInspect operates on authenticated user or agent traffic deliberately routed over HTTP through the proxy to an LLM endpoint. At that boundary, hop-by-hop stripping belongs before prompt policy and provider forwarding. A deployment should reject malformed connection options, reconstruct the outbound provider message, and retain the parsing outcome with the route decision.
For routed traffic, DeepInspect can bind the clean message to verified caller context, prompt classification, destination model, policy version, and outcome. Direct provider calls, opaque tunnels, local model execution, and traffic that bypasses the proxy remain outside this control boundary.
Book a demo today.