How is Portkey different from DeepInspect?

Portkey is an LLM gateway and observability platform that handles routing across 200+ providers, retries, fallbacks, caching, load balancing, cost tracking, traces, evaluations, and prompt template management. The Portkey Guardrails product applies regex filters and PII detection at the gateway. DeepInspect is an identity-bound policy enforcement layer at the HTTP request boundary that classifies prompt data, evaluates per-route policy bundles, and commits per-decision audit records formatted for EU AI Act Article 12 review and sector audit requirements. Portkey covers the operational and observability surface. DeepInspect covers the regulatory audit obligation.

Can Portkey replace DeepInspect for a regulated workload?

For unregulated workloads, Portkey's gateway plus Guardrails can cover the operational and content-filter use case. For regulated workloads, Portkey's traces fall short of the per-decision audit record format the regulator expects. The trace lacks natural-person identity attribution at the model API call (the gateway sees the API key or virtual key, not the application's identity primitive), the policy version that evaluated the decision, the data classification outcome beyond a Guardrail match, and the cryptographic integrity signature that decouples the audit record from the application that took the action.

Can DeepInspect replace Portkey?

For deployments that already have multi-provider operational routing handled elsewhere, DeepInspect forwards cleared requests to a configurable upstream and can address a specific provider directly. For deployments that want the operational features Portkey provides (routing, retries, fallbacks, caching, traces, evaluations, prompt management across 200+ providers), DeepInspect does not replace Portkey. The two layers compose, which is the common production pattern.

DeepInspect vs Portkey: Where LLM Operational Plumbing Stops and Regulatory Audit Starts

Q: How does the deployment topology work when both are in production?

The application points its HTTP client at DeepInspect. DeepInspect evaluates the policy and commits the audit record. DeepInspect forwards the cleared request to Portkey. Portkey applies routing, retries, fallbacks, caching, and load balancing, then forwards to the upstream provider. The response flows back through Portkey to DeepInspect to the application. DeepInspect sees the prompt content and the identity context. Portkey sees the request after DeepInspect has cleared it. The audit record carries both DeepInspect's policy outcome and Portkey's routing outcome so the operator can reconstruct the full request path.

Portkey is an LLM gateway and observability platform shipped as a closed-source product (Portkey Cloud and self-hosted enterprise deployment). The product normalizes the API surface across more than 200 model providers, adds operational features (retries, fallbacks, conditional routing, request caching, load balancing across providers, cost tracking), and exposes the trace, evaluation, prompt management, and guardrails layer on the same control plane. The platform targets the operational concerns of running LLM traffic at scale. DeepInspect sits at the HTTP request boundary and answers a different question. It enforces identity-bound policy on prompt content, classifies prompt data against the regulated data types the organization recognizes, and commits a per-decision audit record that a reviewer under EU AI Act Article 12 or a Fannie Mae LL-2026-04 review accepts.

I want to walk through what Portkey does, what DeepInspect does, and where the two layers compose in production deployments.

TL;DR

Portkey is the operational LLM gateway and observability platform: routing across 200+ providers, retries, fallbacks, caching, load balancing, prompt management, traces, and guardrails. DeepInspect enforces identity-bound policy on prompt content at the request boundary and produces per-decision audit records formatted for regulatory review. Production deployments that need both operational LLM traffic management and regulatory audit records run DeepInspect in front of Portkey (or in front of the provider directly).

Portkey: what it is and where it sits

Portkey ships as an HTTP gateway and observability backend. The gateway speaks the OpenAI API surface and translates inbound requests to the upstream provider's native format across more than 200 supported providers. The control plane (Portkey Cloud or self-hosted Portkey Enterprise) configures the operational rules: which provider serves which request, retries on transient failures, fallback chains across providers, semantic and exact caching, load balancing weights, and per-team or per-user cost attribution.

The Portkey feature surface has grown to cover observability and prompt experimentation. Traces capture the request, response, latency, and metadata at the gateway. Evaluation pipelines run against traces. The Prompts product manages prompt templates with versioning and side-by-side comparison. The Guardrails product applies content checks at the gateway (regex filters, PII detection, custom rule chains) before forwarding the request or after receiving the response.

The architectural sweet spot for Portkey is the operational LLM gateway plus the observability surface for the AI engineering team. A platform team running LLM traffic at scale gets routing, retries, caching, traces, and evaluations on the same operator surface.

What DeepInspect is and where it sits

DeepInspect sits at the HTTP request boundary as a separate enforcement layer. It evaluates identity-bound policy on every request before the request reaches the model provider, classifies prompt data against the regulated data types the organization recognizes, and commits a per-decision audit record with cryptographic integrity. The decisions are deterministic, fail-closed, and independent of the model's behavior.

The feature set covers identity attribution at the model API call from the application's identity primitive (the natural-person identity, the tenant, the role, the route context), per-route policy enforcement for different application surfaces (the support route, the developer route, the legal route, the underwriting route), prompt-level data classification (PII, PHI, MNPI, source code, source-licensed content, regulated jurisdictional data), policy decisions that pass, block, or modify the request, and the per-decision audit record format that downstream audit pipelines consume.

The architectural sweet spot for DeepInspect is the regulated workload. An organization that is the data controller for prompts crossing into a model provider needs evidence that satisfies the deployer obligations under Article 26, the audit obligations under Article 12, the lender record obligations under Fannie Mae LL-2026-04, and the sector-specific regimes (HIPAA, DORA, FedRAMP, ISO 42001) that the workload is subject to.

Where the two products overlap

Both products run as an HTTP gateway in front of LLM providers. Both products produce request records. Both products can apply content filters at the request boundary. The overlap is at the surface level. The underlying audit format and scope differ.

Portkey's Guardrails product applies regex filters, PII detection, and custom rule chains at the gateway. The detection runs at the gateway, with the configurable action either blocking the request, rewriting the content, or logging and passing through. The trace captures the gateway-side metadata and the operational outcome.

DeepInspect's policy evaluation operates on the prompt content, the natural-person identity attached from the application's identity primitive, the route context, the data classification outcome, and the policy version active at the time of the decision. The audit record carries each of these with cryptographic integrity. The audit format is what a regulator reviewing the deployment under Article 12 expects to see.

Both products produce records of requests. Only one of them produces records of policy decisions with the metadata that regulatory review applies.

Feature comparison

| Feature | Portkey | DeepInspect | |---|---|---| | HTTP proxy for LLM traffic | Yes | Yes | | Multi-provider routing | Yes (200+ providers) | Forwards to a configured upstream | | Retries and fallbacks | Yes | Out of scope | | Semantic and exact caching | Yes | Out of scope | | Load balancing across providers | Yes | Out of scope | | Cost tracking per team or user | Yes | Out of scope | | Trace and evaluation surface | Yes | Out of scope | | Prompt template management | Yes | Out of scope | | Identity attribution at model API call | API key or virtual key | Natural-person from IdP | | Per-route policy bundle | Plugin chain | Yes, policy bundle per route | | Prompt data classification | Guardrails (regex, PII detection) | Classification engine for PII, PHI, MNPI | | Per-decision audit record | Operational trace | Cryptographically signed audit record | | Article 12 audit format | Trace plus translation | Native format | | Fannie Mae LL-2026-04 lender record format | Trace plus translation | Native format | | Self-hosted | Enterprise tier | Yes |

Pick Portkey if

Pick Portkey if the primary need is the operational LLM gateway plus the AI engineering team's observability and prompt experimentation surface, and the regulatory audit requirement is satisfied elsewhere or the workload does not trigger a regulatory audit requirement. Portkey covers more operational surface area than any single open-source alternative, with traces, evaluations, and prompt management on the same control plane.

Pick Portkey if the platform team wants a single managed (or self-hosted enterprise) control plane for routing, fallbacks, caching, cost tracking, and the engineering team's observability workflow.

Pick DeepInspect if

Pick DeepInspect if the workload is subject to EU AI Act Article 12, Fannie Mae LL-2026-04, HIPAA, DORA, FedRAMP, ISO 42001, or any sector regime that requires identity-bound per-decision audit records produced independently of the operational gateway's logs. DeepInspect produces the record format that the regulator accepts. Portkey's operational traces satisfy the existence requirement but fall short of the regulatory audit format.

Pick DeepInspect if the security team needs identity-bound enforcement at the request boundary that is independent of the operational gateway's plugin chain. The architectural separation matters when the regulator reviews the audit record for independence from the application that took the action.

Pick both if the deployment needs operational LLM gateway features and regulatory audit records. The composition pattern works in production today.

Composition pattern in production

The deployment topology that runs in production combines the two layers. The application calls DeepInspect (the addressable endpoint that the application points its OpenAI SDK at). DeepInspect verifies the caller's identity from the application's identity primitive, applies the data classification rules, evaluates the policy bundle for the route, commits the per-decision audit record, and forwards the cleared request to Portkey. Portkey applies its routing rules, retries, fallbacks, and caching, and forwards to the upstream provider. The response flows back through Portkey to DeepInspect to the application.

The audit record carries the natural-person identity, the route, the policy version, the data classification outcome, the policy decision outcome, the upstream provider that Portkey selected, the model and version that served the request, and the integrity signature. The Portkey trace carries the routing decision, the retries, the fallback if any, the cache hit or miss, and the cost attribution. The two layers compose without overlap.

Pricing approach

Portkey publishes its pricing tiers (free developer tier, growth tier, enterprise tier) on its website. The enterprise tier includes self-hosted deployment options.

DeepInspect's pricing is communicated through sales conversations and depends on the deployment regime, the workload volume, and the audit-record retention requirements. The cost is meaningfully lower than the cost of an audit miss under EU AI Act Article 12, Fannie Mae LL-2026-04, or a sector regime.

DeepInspect

DeepInspect sits between calling applications and any LLM endpoint over HTTP. It evaluates identity-bound policy on every request, classifies prompt data against the regulated data types the organization recognizes, commits per-decision audit records with cryptographic integrity, and produces the record format that EU AI Act Article 12 and Fannie Mae LL-2026-04 reviewers accept. The architecture composes with Portkey by sitting in front of it, which preserves Portkey's operational and observability surface and adds the regulatory audit layer that Portkey was not designed to provide.

The composition gives organizations the operational LLM gateway they want from Portkey and the per-decision audit records they need for the workload to survive regulatory review. The audit pipeline consumes one record format regardless of which upstream provider Portkey selected for any given request, which keeps the regulatory review tractable.

If you are running Portkey today and the EU AI Act August 2 deadline applies to the workload, let's talk.

The Portkey Guardrails product applies regex filters, PII detection, and custom rule chains at the gateway. The configurable action blocks, rewrites, or passes the request. DeepInspect's classification engine operates against a configurable set of regulated data types (PII, PHI, MNPI, source code, source-licensed content, regulated jurisdictional data), with the classification outcome attached to the per-decision audit record and the policy bundle making the pass-block-modify decision based on classification, identity, and route. The two can run together for layered controls. Portkey Guardrails catch some content at the gateway; DeepInspect's classification and policy enforcement carry the regulatory audit obligation.