LiteLLM vs Portkey: Choose the Operating Model First

LiteLLM vs Portkey is an operating-model decision and is also a capability comparison. Both can present one model-facing API and route requests across providers. LiteLLM gives your platform team an open-source SDK plus a self-hosted gateway. Portkey's current documentation presents Portkey as Prisma AIRS AI Gateway, with routing and guardrails plus administration and OpenTelemetry-compatible observability around the request path. The practical choice is in the pager rotation: which team owns the gateway process and its state? Which team owns the evidence produced when a request fails at 02:13?

TL;DR

• Pick LiteLLM when your platform team wants an open-source SDK or gateway and accepts responsibility for deployment and Postgres. It also owns Redis and scaling, as well as migrations and alerting.
• Pick Portkey when you want gateway routing connected to a broader managed administration and observability surface, while retaining an open-source gateway option.
• Test both with identity propagation and denied-request handling. Also test failover behavior and evidence export. A long capability list says little about the record an investigator can reconstruct.
• Add a separate policy layer when regulated traffic needs an identity-bound authorization decision and independent per-request evidence.

LiteLLM vs Portkey starts with ownership

LiteLLM's current quickstart describes two related products. The Python library gives applications one OpenAI-format interface for more than 100 LLMs, with consistent responses and router-based retries or fallbacks. The full gateway runs as a self-hosted proxy with virtual keys and cost tracking. It also has an administration interface. This split changes the operating owner because a team can embed the SDK inside an application or run the proxy as shared infrastructure.

The proxy puts concrete work on the operator. LiteLLM's production best-practices page covers a master key and trusted proxy ranges; alerts and database writes; machine sizing and workers; Redis and Postgres, including migrations. Those nouns are part of the selection meeting. A team already running Kubernetes and managed Postgres may prefer that custody when it also has an on-call platform function. The gateway is then part of its existing deployment and change-control process.

I would pick LiteLLM only after someone writes their name beside database migration and credential rotation. Someone must also own rollback. An open-source checkbox without an operating owner is procurement theatre.

Portkey combines gateway and observability surfaces

Portkey's AI Gateway documentation lists a universal API and conditional routing; fallbacks and retries; circuit breaking and load balancing; canary testing and budget limits. Rate limits appear there too. It also documents custom hosts for private or local models. The same page says the gateway code is open source and can run locally, with a self-hosted gateway connected back to Portkey.

The surrounding product supplies a broader surface. Portkey's observability documentation names logs and tracing; analytics and filters; custom metadata and feedback. It also names budget limits under an OpenTelemetry-compatible suite. Its documentation now carries the title "Portkey (Prisma AIRS AI Gateway) Docs" and links readers to the Prisma AIRS AI Gateway. Buyers should record that current product identity in the architecture decision. They should use the same identity in the contract file.

That combination is for a team seeking routing and operational telemetry in one product family. The proof still comes from a trial on the real request path. Export a trace and force a provider timeout. Follow one retry across the dashboard. Then verify what identity survived the hop.

The comparison needs six production tests

Capability matrices flatten the differences that create work after deployment. I use six tests for this decision:

• Deployment ownership: name the team that patches and scales the gateway. Then record who manages each database, cache, secret, and certificate.
• Provider failover: force a primary endpoint to time out and confirm retry limits, fallback order, streaming behavior, and the final model selected.
• Identity propagation: send a user-bound and workload-bound request through the same route, then inspect the downstream record for both principals.
• Policy behavior: attempt an allowed request and a denied one with one meaningful attribute changed, such as destination or data class.
• Evidence export: reconstruct one request using timestamps, route, model, retry history, policy result, and correlation identifier.
• Failure posture: remove a required dependency and observe the exact behavior. Document which traffic stops or continues. Record any traffic that loses telemetry.

Run these tests against a production-like topology. A green dashboard beside a terminal window looks reassuring. The useful detail is the correlation identifier that appears in both places after a forced failure.

The AI gateway architecture guide gives the wider topology choices. The AI gateway versus LLM router comparison separates upstream selection from request authorization, which prevents a routing capability from receiving credit for a policy decision it never made.

Self-hosting assigns custody and requires assurance tests

Both products now expose open-source gateway paths in their current documentation. That makes "open source versus proprietary" a poor final decision rule. Source availability helps inspection and customization. Assurance still depends on configuration, dependency management, secret handling, patch speed, tested recovery, and retained evidence.

LiteLLM's production guide makes the operational dependencies unusually visible. Portkey's gateway page makes its local and self-hosted options visible while connecting them to the wider Portkey service. Put each selected topology on one page. Draw the gateway process, persistence services, telemetry destination, policy store, and provider credentials. Mark every network crossing and each team with write access.

The existing LiteLLM alternatives analysis compares broader gateway categories. The Portkey alternatives guide maps Portkey replacements by use case. This direct comparison has a narrower decision: ownership of the running control plane and the evidence it emits.

Routing records and audit records answer different questions

Gateway telemetry can show model destination, latency, token use, retries, and cost. Those fields help platform teams operate a multi-provider service. A compliance investigation asks additional questions. It needs the originating person or agent, delegated authority, content classification, policy version, decision, and integrity of the retained record.

Test those fields rather than inferring them from "logging" or "observability." A service credential may identify the calling application while hiding the employee behind it. A trace may preserve the HTTP sequence while omitting the policy that authorized the data class for that destination. An administration audit can record a configuration change without recording every model request governed by that configuration.

The architecture can compose these records. Keep the gateway's routing trace for operational evidence and produce an independent policy record at the request boundary. Join them with one correlation identifier. This division lets LiteLLM or Portkey perform routing while a separate decision point handles enterprise authorization for regulated traffic.

DeepInspect

DeepInspect can sit in front of LiteLLM or Portkey for authenticated HTTP traffic deliberately routed to an LLM. It evaluates the identity and role supplied by the application. It then classifies the request and applies versioned policy before recording the decision. That decision can permit or deny the request, or redact it, before forwarding allowed traffic. LiteLLM or Portkey can continue to select providers and retry calls. They can also collect operational telemetry.

The enforcement boundary is precise at the routed model request. DeepInspect governs the model request that traverses its proxy. Gateway administration, Postgres and Redis security, local execution, and tool-side authorization retain their own control owners. The joined record gives an investigator the policy decision and the gateway's routing trace under one correlation value. Book a technical deep dive at deepinspect.ai.