Is Envoy AI Gateway production-ready?

Yes. It reached 1.0 on June 23, 2026, with stable v1beta1 CRDs, built on Envoy Gateway v1.8.1 and Envoy v1.38. Bloomberg, LY Corporation, Nutanix, Tencent Cloud, and the National Research Platform are named adopters, and maintainers come from Tetrate, Bloomberg, Tencent, Netflix, and Nutanix. For a Kubernetes platform team it is the strongest open-source AI routing layer available, and there is no reason to look elsewhere for the routing job.

Does Envoy AI Gateway do prompt-level data classification?

No. The feature that gets mistaken for this is request and response body redaction, which suppresses bodies from telemetry so sensitive content does not land in traces. That is a privacy control on the observability path rather than a classifier. The gateway makes no determination about whether the content in a prompt was permitted to leave the environment. There is no PII, PHI, or MNPI detection in the product.

What does CEL-based per-tool authorization actually give me?

It lets you write CEL expressions against JWT claims to gate which MCP tools a given caller can invoke, and the implementation applies the same rules to tools/list as to tools/call, so a caller cannot enumerate tools they are not permitted to use. That closes a real information-leak path and it is genuinely good work. What it evaluates is a claim in a token, and what it produces is a routing outcome in a trace. Whether that claim maps to a verified natural person, and whether the decision is recorded as evidence, are both left to the layers around it.

How is DeepInspect different from Envoy AI Gateway?

Envoy AI Gateway is traffic infrastructure: providers, translation, quota, fallback, MCP multiplexing, telemetry. DeepInspect is a policy decision point: it evaluates whether this authenticated person, on this route, may send this class of data to this model, and it commits a signed record of the decision. The overlap is that both are HTTP proxies in the AI path. The difference is that one is optimizing how traffic gets where it is going and the other is deciding whether it should go at all, then proving what it decided.

For an unregulated workload, no. Envoy AI Gateway alone covers routing, quota, observability, and MCP tool gating, and OpenTelemetry traces are an appropriate record. For a regulated workload, the gateway leaves three things open: natural-person identity on the model call, classification of prompt content against PHI and MNPI, and a signed record the application cannot suppress. Those are what the second layer supplies, and the standard topology runs DeepInspect in front with Envoy AI Gateway behind it.

Does running two proxies hurt latency?

Enforcement overhead in DeepInspect measures under 50 ms in internal testing. Envoy's proxy overhead is famously small, on the order of single-digit milliseconds. Model inference runs 500 ms to 5 seconds. The combined proxy cost sits inside the variance of the model call itself, which is why the architectural argument for inline enforcement holds: there is no latency budget being traded away, and I made that case at length in why AI security must be inline.

DeepInspect vs Envoy AI Gateway: Two Layers of the Same Kubernetes Data Path

Envoy AI Gateway reached 1.0 on June 23, 2026, sixteen months after its v0.1 release shipped with two providers. The 1.0 carries 16 LLM providers behind one OpenAI-compatible API, cross-provider translation including Anthropic Messages to Bedrock Converse with streaming and tool use preserved, token-aware rate limiting, provider fallback, and an MCP gateway with CEL-based per-tool authorization. Maintainers come from Tetrate, Bloomberg, Tencent, Netflix, and Nutanix. Bloomberg and LY Corporation run it in production.

It is the best open-source AI routing layer available to a Kubernetes platform team, and I say that without qualification. This piece is about what it does, what it leaves for something else to do, and how the two layers sit in the same data path.

TL;DR

Envoy AI Gateway is a Kubernetes-native routing and traffic layer for LLM and MCP traffic, configured through CRDs on top of Envoy Gateway. Its identity primitive is JWT claim projection and its record is an OpenTelemetry trace. DeepInspect binds policy to the natural person, classifies prompt content across regulated data types, and commits a signed per-decision audit record. In a regulated Kubernetes deployment both run, DeepInspect in front, Envoy AI Gateway behind.

Envoy AI Gateway: what it is and where it sits

It is an additive layer on top of CNCF's Envoy Gateway, configured through Kubernetes CRDs and built on the Gateway API plus the Gateway API Inference Extension. The stable v1beta1 CRDs are AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute.

What 1.0 actually gives you:

Sixteen providers behind one OpenAI-compatible surface, including OpenAI, Azure OpenAI, Gemini, Vertex AI, Bedrock, Anthropic, Mistral, Cohere, Groq, Together AI, DeepSeek, and SambaNova.
Cross-provider request and response translation preserving streaming, tool use, reasoning blocks, and images.
Model virtualization through modelNameOverride, so application-facing model names stay stable while the routing behind them changes.
Upstream auth through BackendSecurityPolicy: API keys plus cloud-native identity, including GKE Workload Identity through Application Default Credentials.
Token- and quota-aware rate limiting through QuotaPolicy, with backend rate-limit filter injection against upstream provider limits.
Provider fallback and InferencePool endpoint selection for self-hosted models.
An MCP gateway multiplexing multiple servers behind one endpoint, with CEL-based per-tool authorization where tools/list applies the same rules as tools/call, and JWT claim projection to MCP backends.
OpenTelemetry tracing with OpenInference compatibility, Prometheus GenAI token metrics, time-to-first-token, inter-token latency, and separate reasoning-token accounting.
Request and response body redaction for compliance.

The CEL per-tool authorization is the most interesting security feature in the 1.0, and the fact that tools/list respects the same rules as tools/call is a detail that shows the maintainers thought about tool enumeration as an information leak. That is good engineering.

What DeepInspect is and where it sits

DeepInspect is a stateless proxy at the AI request boundary. It evaluates identity-bound policy per request, classifies prompt content against the regulated data types the organization recognizes, and commits a per-decision audit record with cryptographic integrity. Deterministic, fail-closed, model-agnostic.

The primitives: natural-person identity attribution at the model call, taken from the application's identity primitive rather than inferred from a credential; per-route and per-role policy bundles; prompt-level classification across PII, PHI, MNPI, source code, and jurisdictional categories; a pass, block, or modify decision before the request reaches the model; and an audit record shaped for EU AI Act Article 12, HIPAA, and DORA review.

Where the two separate

Both run in the same Kubernetes data path. The separation is between traffic management and decision evidence.

Envoy AI Gateway's identity handling is claim projection. It can validate a JWT and forward claims to MCP backends, and CEL expressions can gate tool access on those claims, which is real authorization and better than most gateways offer. What it does not do is treat the natural person as a first-class primitive that gets recorded against the decision. The claim is a routing and gating input. It is not bound into an evidence record.

On data, the relevant feature is body redaction, and precision matters here. Redaction suppresses request and response bodies from telemetry so sensitive content does not land in traces. That is a privacy control on the observability path. It is not a classifier, and it makes no determination about whether the content was permitted to leave the environment in the first place. There is no PII, PHI, or MNPI detection in the gateway.

On records, Envoy AI Gateway emits OpenTelemetry traces and Prometheus metrics. Both are excellent for operating a platform. Neither is a signed, tamper-evident record of a policy decision that an auditor accepts as independent evidence. A trace is written by the system under audit, which is the self-attestation problem in its cleanest form.

Feature comparison

| Capability | Envoy AI Gateway 1.0 | DeepInspect | |---|---|---| | Kubernetes-native, CRD-configured | Yes, Gateway API | Deployed as a service | | Multi-provider routing | Yes, 16 providers | Forwards to a configured upstream | | Cross-provider translation | Yes, including Anthropic to Bedrock | Out of scope | | Token and quota rate limiting | Yes, QuotaPolicy | Per-role and per-route | | Provider fallback | Yes | Out of scope | | MCP multiplexing | Yes, MCPRoute | Policy on MCP calls | | Per-tool authorization | Yes, CEL expressions on JWT claims | Yes, identity-bound policy | | Identity on the model call | JWT claim projection | Natural person from the identity provider | | Prompt data classification | Body redaction only, no classifier | PII, PHI, MNPI, source code, jurisdictional | | Per-decision signed audit record | OpenTelemetry traces, Prometheus metrics | Yes, cryptographically signed | | Article 12 audit format | Traces | Yes | | Fail-closed policy posture | Configurable at the proxy | Default, by design | | License and cost | Apache 2.0, free | Commercial |

Pick Envoy AI Gateway if

Pick Envoy AI Gateway if you are a Kubernetes platform team and the job is routing, failover, token accounting, and provider abstraction. Pick it if you already run Envoy Gateway and want the AI layer on the same CRD surface and the same operator workflow. Pick it if you want an Apache 2.0 project with maintainers from five serious engineering organizations and no license cost. Pick it if MCP multiplexing with per-tool CEL authorization is the control you need, because that is a real capability and it ships in the box. And pick it if the workload has no regulatory audit obligation, in which case OpenTelemetry traces are the right level of record.

Pick DeepInspect if

Pick DeepInspect if a reviewer will ask which natural person sent a specific prompt, and a JWT claim in a trace span will not answer that. Pick it if prompt content has to be classified against PHI or MNPI, which the gateway does not do at all. Pick it if the evidence record has to be signed and written on a path the application cannot suppress. Pick it if the deployment spans Kubernetes and non-Kubernetes environments and the policy has to be identical in both. And pick it if the EU AI Act, HIPAA, or DORA applies to the workload.

Composition in production

The topology is straightforward because both are proxies in the same path.

DeepInspect evaluates the natural-person identity, classifies the prompt, applies the per-route policy, and commits the signed record. It forwards the cleared request to Envoy AI Gateway, which selects the provider, translates the request, applies quota, and routes. The response returns through both. Envoy's OpenTelemetry traces still describe the routing decision. DeepInspect's audit record describes the authorization decision. Together they reconstruct the full path.

CEL per-tool authorization on the Envoy side stays useful. It gates which tools a claim can reach. DeepInspect's policy gates whether this person, with this data, on this route, may reach a tool at all, and records the answer.

Pricing approach

Envoy AI Gateway is Apache 2.0 and free. Commercial support and a hosted routing service come from Tetrate separately. DeepInspect is commercial, priced through sales conversation, deployed self-hosted or managed. The honest framing for a platform team: the routing layer should be free and open source, and the layer that produces regulatory evidence is where the budget goes.

DeepInspect

This is the gap DeepInspect closes. Envoy AI Gateway governs the traffic. DeepInspect governs the decision. Each AI request carries a natural-person identity, a data classification outcome, and a policy version, and the record of all three is committed before the response returns, on a write path the application never has custody of.

For a Kubernetes platform running regulated workloads, that is the difference between an OpenTelemetry trace showing that a request was routed to Bedrock and a signed record showing that a named clinician, on the clinical-documentation route, sent content classified as PHI, under policy version 14, and the request was permitted. The first answers an operator's question. The second answers an auditor's. Enforcement overhead measures under 50 ms in internal testing against model inference of 500 ms to 5 seconds, so the record costs nothing the user can perceive.

If you are running Envoy AI Gateway on a regulated workload with the August 2, 2026 deadline ahead of you, let's talk today.