mTLS for an AI Gateway: Proving Which Service Made the Model Call
A bearer key in front of an AI gateway proves possession of a secret, not the identity of the service that holds it, and a leaked key is indistinguishable from the real caller. Mutual TLS binds each calling service or agent to a certificate the gateway verifies on every connection. This piece shows what mTLS establishes for AI traffic, what it does not, and how it feeds the authorization decision.
A bearer key presented to an AI gateway proves one thing: whoever sent the request holds the secret. It does not prove which service holds it, and a copied key produces requests the gateway cannot tell from the real caller's. For service-to-service and agent traffic, that is a weak foundation to authorize model calls on, because the whole authorization decision then rests on a string that can be read from an environment variable, a log, or a misconfigured store. Mutual TLS replaces the string with a verified certificate, so the gateway knows which service opened the connection before it evaluates a single call.
I want to walk through what mTLS establishes for AI traffic, where its identity stops and another mechanism has to take over, and how the certificate identity feeds the authorization decision the gateway makes on each model call.
What mTLS establishes that a bearer key does not
In ordinary TLS the client verifies the server's certificate and the connection is encrypted. Mutual TLS adds the other direction: the client also presents a certificate, and the server verifies it against a trusted certificate authority before the connection proceeds. For an AI gateway, that means each calling service or agent carries a certificate the gateway checks on every connection, and a caller without a valid, unexpired, CA-signed certificate never gets to send a request.
The security difference from a bearer key is the difference between possession and provenance. A key says "I have the secret." A certificate says "I am the holder of a private key that a CA vouched for, and I proved it in the handshake." A leaked bearer key works from anywhere; a certificate cannot be replayed without the private key that never leaves the holder, and it can be revoked at the CA when a service is retired. This is the identity foundation zero trust asks for, and applying zero trust to AI traffic starts with knowing the caller cryptographically rather than by a shared secret.
Where certificate identity stops
mTLS authenticates the workload, the service or agent process that opened the connection. It does not by itself establish the human user on whose behalf that service is acting. A support application connecting to the gateway over mTLS proves it is that application; it does not prove which of its thousands of users triggered a given call. That user identity arrives in the request, typically as a token the application forwards, and the gateway resolves it there.
So mTLS and request-level identity are two layers of the same picture. The certificate answers "which service is this," verified at the connection. The forwarded token answers "which user or agent is this acting for," verified per request. Authorization needs both, which is why an identity-aware AI gateway combines the workload identity from mTLS with the user or agent identity in the call. Claiming the certificate alone identifies the end user would overstate what the handshake proves.
Certificate identity feeds the per-call decision
The value of establishing workload identity at the connection is that every call on that connection inherits a verified origin. When the gateway authorizes a model call, it already knows the calling service is the one its certificate names, and it can apply policy that depends on that, for example allowing a particular internal service to reach a model that a general-purpose service may not. Combined with the user or agent identity in the request, the decision rests on verified identity at both layers rather than on a shared key that any component could be holding.
The certificate identity also lands in the record. A per-decision log that names the calling workload from its certificate, alongside the user or agent from the request, attributes each model call to a verified origin, which is stronger evidence than a key that many services might share. That pairing of workload and request identity is what makes per-request LLM authorization rest on something an attacker cannot simply copy.
DeepInspect
This is the gap DeepInspect closes at the connection layer. DeepInspect runs as a stateless proxy between your services, agents, and any LLM, and can require mutual TLS from callers so every connection carries a verified workload certificate. On each model call it combines that workload identity with the user or agent identity in the request, applies per-role and per-route policy, and writes a per-decision record naming both.
Because the proxy holds no long-lived provider keys in the request path and verifies callers cryptographically, a leaked application secret does not hand an attacker the ability to impersonate a trusted service. DeepInspect does not issue your certificates or run your CA; it verifies them and binds the identity they carry to the model-call decision. To put verified identity in front of your model traffic, book a platform demo.
Frequently asked questions
- Does mTLS identify the end user of an AI call?
No. mTLS authenticates the workload, the service or agent that opened the connection, not the human user it is acting for. The end-user identity arrives in the request, usually as a token the application forwards, and is verified there. The two layers combine: the certificate proves which service is calling, and the token proves which user or agent the call is on behalf of. Authorization uses both, so treating the certificate as the user identity would misread what the handshake establishes.
- How is mTLS stronger than an API key for a gateway?
An API key proves possession of a secret, and a copied key is indistinguishable from the real one and works from anywhere. A client certificate proves the holder controls a private key vouched for by a certificate authority, verified in the handshake, and it cannot be replayed without that private key. Certificates can also be revoked at the CA when a service is retired. For service-to-service AI traffic, that shifts the authorization foundation from a shared string to a verified identity.
- Does requiring mTLS add latency to model calls?
The certificate verification happens during the TLS handshake when the connection is established, not on every request, so on a reused connection the per-call cost is negligible. The handshake itself adds a small one-time cost at connection setup, which is minor next to model inference latency. Because callers typically keep connections open and reuse them, the amortized overhead of mTLS on AI traffic is low relative to the identity assurance it provides.
- Can we use mTLS and bearer tokens together?
Yes, and for AI traffic that pairing is the point. mTLS establishes the workload identity at the connection, and a bearer or signed token in the request carries the user or agent identity. The gateway verifies the certificate once per connection and the token per request, then authorizes each model call against both. Using them together gives verified identity at the workload layer and the user layer, which a single mechanism cannot provide on its own.