What is the best AI gateway in 2026?

There is no single answer, because "AI gateway" covers three jobs. For multi-provider routing on Kubernetes, Envoy AI Gateway reached 1.0 in June 2026 and is the strongest open-source choice. For the fastest path to a working proxy, LiteLLM. For agent and MCP traffic, Solo.io agentgateway. For security-vendor consolidation, Prisma AIRS after the Portkey acquisition, or F5's AI Security Platform. For a regulated workload with an audit obligation, no routing gateway closes the governance job on its own, and the production pattern is to compose a governance layer in front of the routing layer.

Which AI gateways are open source?

LiteLLM under MIT. Envoy AI Gateway, Bifrost, MLflow AI Gateway, Helicone, and Solo.io agentgateway under Apache 2.0. Kong's core is Apache 2.0, with semantic caching and analytics gated behind Kong Enterprise. Portkey's gateway remains open source on GitHub after the Palo Alto Networks acquisition. Cloudflare AI Gateway, Vercel AI Gateway, and OpenRouter are SaaS only with no self-host path, which rules them out where data residency is a requirement.

What changed in the AI gateway market in 2026?

Consolidation, in one direction. Palo Alto Networks announced the Portkey acquisition on April 30, 2026 and completed it on May 29, folding the gateway into Prisma AIRS. F5, which had acquired CalypsoAI in September 2025, launched its AI Security Platform on June 22, 2026. Mintlify acquired Helicone, which is now in maintenance mode. Envoy AI Gateway reached 1.0 on June 23. Security vendors bought routing gateways, which is the market conceding that the gateway is where AI policy gets enforced.

Do I need an AI gateway if I only call one model provider?

The routing job disappears with a single provider. The governance job does not. A single-provider deployment still has to answer which authenticated person sent which prompt, whether the data in that prompt was permitted to leave the environment, and what record proves the answer. Those questions come from Article 12, from HIPAA, from DORA, and from a security questionnaire, and none of them care how many providers are in the stack.

Can I run two gateways in front of each other?

Yes, and for regulated workloads it is the standard topology. The application calls the governance layer. The governance layer evaluates identity and data classification, commits the audit record, and forwards the cleared request to the routing gateway. The routing gateway selects a provider, handles caching and fallback, and returns. Each layer does the job it was designed for, and the audit record carries both the policy outcome and the routing outcome so the full request path is reconstructable.

What should be in an AI gateway RFP?

Six questions cut through most vendor decks. Does the audit record carry a natural-person identity or an API key? Is the record signed, and can the application suppress it? What is the data classification taxonomy, and does it cover PHI and MNPI rather than PII alone? Does the gateway fail closed or fail open when the policy engine is unreachable? What is the p95 enforcement overhead relative to model inference latency? And can it run self-hosted in the jurisdiction the data is required to stay in?

Best AI Gateway 2026: The Fifteen Products, Sorted by the Job They Actually Do

Palo Alto Networks completed its acquisition of Portkey on May 29, 2026 and is folding the gateway into Prisma AIRS. Envoy AI Gateway reached 1.0 on June 23, 2026, sixteen months after its first release. F5 launched its AI Security Platform on June 22. Three data points, one direction: a security vendor bought a routing gateway, an infrastructure gateway went stable, and an application-delivery vendor built a security platform around its gateway. The category is converging on the position that the AI gateway is a control point rather than a router.

I want to sort the field by the job each product actually does, because "AI gateway" now covers three different jobs and the buying mistake is assuming one product does all three.

The three jobs

Job 1: routing and cost. One API surface across many providers. Fallbacks, load balancing, semantic caching, token accounting. This is the job most gateways were built for.

Job 2: observability. Traces, token metrics, latency percentiles, prompt and response capture. Usually bundled with job 1.

Job 3: governance. Identity-bound authorization on the model call, prompt-level data classification, and a per-decision audit record that survives regulatory review. This is the job almost nothing does, and it is the one that determines whether a workload can ship in a regulated environment.

The single most useful question when evaluating any product below: does it authenticate the application, or does it authenticate the human behind the request? Nearly every gateway on this list authenticates a gateway API key belonging to a service. That distinction is the difference between an operational log and audit evidence.

The fifteen

Routing-first, open source

LiteLLM (MIT, github.com/BerriAI/litellm). A Python SDK plus a proxy server giving one OpenAI-format interface to 100+ providers, with cost tracking, load balancing, guardrail hooks, and logging. The most widely deployed open-source LLM proxy, and the reason it shows up in so many agent frameworks. Worth reading alongside the June 2026 LiteLLM CVE wave, which is a lesson about control-plane design rather than a knock on the project.

Envoy AI Gateway (Apache 2.0, aigateway.envoyproxy.io). Built on Envoy Gateway, configured through Kubernetes CRDs. Version 1.0 in June 2026 brought 16 providers behind one OpenAI-compatible API, cross-provider translation including Anthropic Messages to Bedrock Converse, token-aware rate limiting, provider fallback, and an MCP gateway with CEL-based per-tool authorization. Maintainers from Tetrate, Bloomberg, Tencent, Netflix, and Nutanix. The strongest choice for a Kubernetes-native platform team.

Bifrost (Apache 2.0, github.com/maximhq/bifrost). Go-based, performance-first. Publishes claims of microsecond-scale overhead at high request rates across 1,000+ models, with cluster mode, adaptive load balancing, semantic caching, and MCP support.

MLflow AI Gateway (Apache 2.0, mlflow.org/ai-gateway). Runs as part of the MLflow Tracking Server. OpenAI-compatible endpoint, credential management, traffic splitting, fallbacks, native MLflow tracing. Naming caution: the original MLflow AI Gateway was deprecated and renamed the MLflow Deployments Server, and the current product carries the old name again. Search results conflate them.

Kong AI Gateway (developer.konghq.com/ai-gateway). AI plugins on the Kong data plane: AI Proxy for multi-provider normalization, semantic caching, prompt templates, AI Prompt Guard for regex allow and deny lists, token-based rate limiting. Kong core is Apache 2.0; semantic caching and analytics need Kong Enterprise. The natural pick for teams already running Kong as their HTTP data plane.

Routing-first, commercial or SaaS

Cloudflare AI Gateway (developers.cloudflare.com/ai-gateway). Edge-hosted proxy adding caching, rate limiting, cost budgets, and per-request logging. SaaS only. No self-host option, which is a hard stop for deployments with data residency constraints.

Vercel AI Gateway (vercel.com/docs/ai-gateway). One API key and string model IDs across hundreds of models, with routing rules, fallbacks, and bring-your-own-key at zero markup. Developer-experience-first. SaaS only.

OpenRouter (openrouter.ai). A hosted marketplace exposing one OpenAI-compatible endpoint to 400+ models from 60+ providers. It is an aggregation layer rather than a governance tool, and it is worth being precise about that: routing your traffic through a marketplace adds a party to the data path.

TrueFoundry (truefoundry.com/ai-gateway). Enterprise AI gateway plus MCP gateway. Routing, guardrails, rate limits, cost tracking, observability across hosted and self-hosted models. Available managed, hybrid, or fully self-hosted.

Helicone (helicone.ai). Rust-based observability plane and gateway; change one base URL and get caching, fallbacks, and rate limiting across 100+ providers. Acquired by Mintlify in 2026 and now in maintenance mode: security patches, bug fixes, and new model support, with active feature development ended. Factor that into a multi-year decision.

Security-first

Portkey, now Palo Alto Networks Prisma AIRS (paloaltonetworks.com). Portkey shipped an open-source gateway routing to 1,600+ models with 40-plus built-in guardrails. Palo Alto closed the acquisition on May 29, 2026 and is integrating it into Prisma AIRS. The open-source gateway remains on GitHub. The strategic read matters more than the feature list: the largest network security vendor decided the AI gateway is where AI security is enforced.

F5 AI Gateway (aigateway.clouddocs.f5.com). Announced November 2024. A containerized Kubernetes service with a core proxy plus pluggable processors that can modify, reject, or annotate traffic. Shipped processors include prompt injection detection (English only, per F5's own docs), data security matchers, language identification, and system prompt handling. Distinct from F5 AI Guardrails, the separate product from the CalypsoAI acquisition that closed September 26, 2025. Do not conflate the two.

Solo.io agentgateway (agentgateway.dev). Rust, Apache 2.0, contributed to the Linux Foundation in August 2025. Purpose-built for agent traffic: MCP OAuth 2.1, tool-level RBAC, secure token exchange, A2A federation, denial-of-wallet budget controls, inline guardrails. This is what the product formerly marketed as Gloo AI Gateway became; Solo.io's Gloo AI Gateway page now redirects here.

Apigee (cloud.google.com/solutions/apigee-ai). Not a separate SKU. Google's API management platform used as an AI gateway, with LLM token policies, semantic caching through Vertex embeddings and Vector Search, and Model Armor policies (SanitizeUserPrompt, SanitizeModelResponse) for prompt injection and jailbreak filtering plus Google's sensitive data protection infotypes.

AWS (a pattern, not a product). Three real things get called "the AWS AI gateway." Amazon Bedrock AgentCore Gateway is the closest named product, a managed entry point converting APIs and Lambda functions into MCP tools with ingress and egress auth. The API Gateway plus Bedrock reference architecture is the DIY pattern. The Multi-Provider Generative AI Gateway guidance deploys LiteLLM into your own account and is explicitly disclaimed as not production-ready as shipped.

Where all fifteen land on job 3

Here is the honest scorecard on the three governance properties that decide whether a regulated workload can ship.

| Product | Identity of the human on the model call | Prompt-level data classification | Per-decision signed audit record | |---|---|---|---| | LiteLLM, Bifrost, Helicone, MLflow, OpenRouter | Gateway API key | No | Operational logs | | Kong AI Gateway | API consumer | Regex allow/deny lists | Operational logs | | Envoy AI Gateway | JWT claim projection to MCP backends | Body redaction, which is suppression rather than classification | OpenTelemetry traces | | Cloudflare, Vercel | Account or project key | No | Request logs | | Apigee | App, developer, or API product | Model Armor plus Google SDP covers PII; no PHI or MNPI taxonomy | Analytics and Cloud Logging | | AWS | IAM principal or Cognito identity | Bedrock Guardrails covers PII with block, anonymize, or custom regex | S3 and CloudWatch invocation logs | | F5 AI Gateway | JWT validated, no binding primitive | Matcher-based redaction; richer controls sit in the separate Guardrails product | Auditable records exported to S3, unsigned | | Solo.io agentgateway | Agent and tool identity | Semantic guardrails, no documented taxonomy | Marketed as cryptographic audit trails; the mechanism is unspecified in the technical docs |

Every product in the routing tier authenticates a service credential. That is correct for its job. It also means the audit record answers "which API key made this call," and an EU AI Act Article 12 reviewer asks "which natural person, under what policy, with what data."

How to choose

Pick on job, then on deployment constraint.

Kubernetes platform team, open source, routing: Envoy AI Gateway.
Fastest path to multi-provider, open source: LiteLLM.
Already running Kong: Kong AI Gateway.
Agent and MCP traffic specifically: Solo.io agentgateway.
Already a Palo Alto or F5 shop: Prisma AIRS or the F5 AI Security Platform, on procurement grounds alone.
Google Cloud with existing Apigee: Apigee with Model Armor.
Regulated workload with an audit obligation: none of the above closes job 3 on its own. Compose.

Composition is the normal production pattern, and it is the answer I give most often. The routing gateway keeps doing routing. The governance layer goes in front of it.

DeepInspect

This is the gap DeepInspect closes. DeepInspect sits at the AI request boundary as a stateless, model-agnostic proxy. Every request is evaluated against the natural-person identity the application supplies, the route, the data classification of the prompt content, and the policy version active at that instant. The decision is deterministic and fail-closed, and it happens before the request reaches the model or the routing gateway behind it.

The audit record is the part the routing tier structurally cannot produce, because a service credential is the only identity it ever sees. DeepInspect commits a signed, per-decision record bound to the human principal, carrying the classification outcome and the policy version, and it commits before the response returns to the application. The application never has custody of the write path, which is what makes the record independent evidence rather than self-attestation. In production this runs in front of Kong, Envoy AI Gateway, LiteLLM, or Bedrock without displacing any of them.

If you are choosing an AI gateway with the August 2, 2026 EU AI Act transparency obligations on the calendar, let's talk today.