DeepInspect vs TrueFoundry: A Kubernetes AI Platform and a Focused Policy Layer
TrueFoundry bundles an AI gateway, MCP governance, model serving, and GPU orchestration into one Kubernetes-native platform that runs in your own cloud, fronting 1600-plus models with rate limiting, input and output guardrails, RBAC, and audit logging. This piece covers what the platform does well, where its audit sits relative to the systems it also runs, and how a decoupled identity-bound policy and audit proxy fits alongside it.
TrueFoundry is a Kubernetes-native AI platform that bundles four things into one system: an AI gateway fronting more than 1,600 models behind an OpenAI-compatible endpoint, MCP governance, model serving, and GPU orchestration with fractional-GPU scheduling. It runs in your own VPC, on-premises, or air-gapped, and its gateway carries rate limiting, input and output guardrails for toxicity, bias, and PII, centralized keys, SSO and RBAC, audit logging, and policy-based routing with fallbacks. For a platform team that wants to serve, route, and govern models in one place, it is a lot of capability under a single control plane.
TrueFoundry and DeepInspect both sit in the AI request path, and both carry policy and audit features. The difference is scope and independence. TrueFoundry is the platform that also runs your models. DeepInspect is a decoupled layer that authorizes and records, and has no other job.
TL;DR
TrueFoundry is a Kubernetes-native platform that serves models, orchestrates GPUs, routes across 1,600-plus models, and governs that traffic with guardrails, RBAC, and audit logging. DeepInspect is a stateless proxy that does one thing: evaluate identity-bound policy per request and commit a signed audit record, independent of the system that serves the model. A team standardizing model serving and routing on Kubernetes wants the platform. A regulated deployment that needs audit independence from the system under audit wants the decoupled layer. DeepInspect can sit in front of TrueFoundry.
TrueFoundry: what it is and where it sits
TrueFoundry's reach is the selling point. The gateway gives one endpoint for every provider, with token-aware rate limiting configured through declarative YAML, load balancing, centralized failovers, and policy-based routing. Guardrails run on input and output for toxicity, bias, PII, and hallucination checks. Governance includes centralized API keys, SSO, RBAC, OAuth2 tool policies, and audit logging. MCP governance is built in. Observability is deep: per-request metrics for tokens, cost, and latency broken down by model, team, and user, with OpenTelemetry tracing and exports to Prometheus, Grafana, and Elastic.
Underneath the gateway, the same platform serves models and schedules GPUs, using fractional-GPU techniques like NVIDIA MIG and time-slicing to raise utilization. It advertises SOC 2, HIPAA, and GDPR alignment and runs entirely in the customer's cloud. The architecture is a unified platform: the thing that routes the call, the thing that serves the model, and the thing that logs the decision are components of one system.
That unification is efficient. It also means the audit record and the system being audited share a trust boundary, which is the property a decoupled proxy is designed to break.
What DeepInspect is and where it sits
DeepInspect is a stateless proxy at the AI request boundary between authenticated users or agents and any HTTP LLM endpoint. It evaluates identity-bound policy per request, classifies prompt content against regulated data types, and commits a per-decision audit record with cryptographic integrity. Deterministic, fail-closed, model-agnostic. It does not serve models, schedule GPUs, or route for cost.
Where the two separate
Audit independence. TrueFoundry's audit logging is a feature of the platform that also serves and routes the model. When the system generating the AI decision also writes the record of it, you have the self-attestation problem: the same platform can, in principle, log selectively, suppress a record, or lose one on a crash between the model response and the log commit. DeepInspect commits the record on a write path the calling system never has custody of, before the response returns. In every other regulated industry, the system under audit is not the system generating the audit record.
Focused policy versus platform RBAC. TrueFoundry's RBAC and OAuth2 tool policies govern who can use the platform and its resources. DeepInspect's policy binds to the natural-person identity behind each request and turns on the data classification in the prompt. Platform RBAC answers who may operate the system. Identity-bound policy answers whether this call, by this person, with this data, is permitted right now.
Breadth versus a chokepoint. TrueFoundry spans serving, GPU orchestration, routing, and governance. DeepInspect claims one line, HTTP AI traffic between users and LLMs, and enforces authorization and audit on it as a non-bypassable chokepoint. Model serving, GPU scheduling, and cost routing are outside its boundary by design.
Feature comparison
| Capability | TrueFoundry | DeepInspect | |---|---|---| | Unified multi-provider gateway | Yes, 1600+ models | Yes, any HTTP LLM endpoint | | Model serving and GPU orchestration | Yes, Kubernetes-native | Out of scope | | Rate limiting and load balancing | Yes, token-aware YAML | Out of scope | | Input and output guardrails | Yes, toxicity, bias, PII | Policy-driven | | Access control primitive | RBAC, SSO, OAuth2 tool policies | Natural-person identity per request | | Prompt-level data classification | PII guardrail | Named taxonomy, PHI and MNPI | | Per-request identity-bound policy | Platform RBAC | Yes, architectural | | Signed per-decision audit record | Audit logging in-platform | Yes, tamper-evident, identity-bound | | Write-path independence from the runtime | Shared platform trust boundary | Yes, decoupled | | MCP governance | Yes | Governs MCP over HTTP | | Deployment | Kubernetes in your cloud, air-gapped | Self-hosted or managed |
Pick TrueFoundry if
Pick TrueFoundry if you are standardizing model serving and routing on Kubernetes and want the gateway, the serving stack, and GPU orchestration under one platform. Pick it if GPU utilization is a cost problem, because fractional-GPU scheduling is a direct lever. Pick it if declarative rate limiting and policy-based routing across 1,600-plus models match how your platform team works. Pick it if a single vendor for the AI runtime, with guardrails and audit logging included, is the operating model you want.
Pick DeepInspect if
Pick DeepInspect if audit independence is a requirement, meaning the record of each AI decision cannot be written by the system that served the model. Pick it if the policy has to bind to a named person and turn on the data class in the prompt. Pick it if you run models across several platforms and providers and want one identity-bound authorization and audit layer in front of all of them. Pick it if an EU AI Act Article 12 review or a HIPAA audit will ask for evidence on a write path the runtime cannot touch.
Composition in production
These compose cleanly because they occupy different scopes. TrueFoundry serves and routes the models. DeepInspect sits at the request boundary in front of the gateway, binds the natural-person identity, classifies the prompt, applies the per-role policy, and commits the signed record before the call proceeds. The platform runs the AI. The decoupled proxy authorizes each call and produces evidence the platform cannot alter, which is exactly the separation a regulator expects between the system under audit and the audit record.
Pricing approach
TrueFoundry is a commercial platform priced through sales, deployed in the customer's own cloud, with no public per-seat list. DeepInspect is also priced through a sales conversation and deploys self-hosted or managed. Because both scope to deployment and volume, the honest comparison is a scoped conversation rather than a sticker number.
DeepInspect
This is the layer DeepInspect owns. TrueFoundry runs and routes the models. DeepInspect authorizes each call against a named identity and produces a record the runtime cannot rewrite.
Every request is evaluated against the natural-person identity the application supplies, the role that person holds, the classification of the data in the prompt, and the policy version active at that moment. The decision is deterministic and fail-closed, and it happens inline before the request reaches the model, with enforcement overhead measured under 50 ms in internal testing. Every decision produces a signed, identity-bound audit record, committed before the response returns, on a write path the serving platform never controls. That write-path independence is what turns a log into evidence. If you are running a Kubernetes AI platform and need audit independence from the system that serves your models, let's talk today.
Frequently asked questions
- Is TrueFoundry a security control?
TrueFoundry includes security features: RBAC, SSO, OAuth2 tool policies, input and output guardrails, and audit logging, and it runs in your own cloud with SOC 2, HIPAA, and GDPR alignment. Those govern who operates the platform and screen content for categories like PII and toxicity. What the platform's audit does not provide is independence from itself, because the same system serves the model and writes the record. For teams whose threat model includes proving the audit trail was not shaped by the runtime, that independence has to come from a decoupled layer.
- Does TrueFoundry produce an audit trail?
Yes. TrueFoundry offers audit logging plus deep observability, with per-request metrics on tokens, cost, and latency, OpenTelemetry tracing, and exports to Prometheus, Grafana, and Elastic. That record lives inside the platform that also serves and routes the model. DeepInspect commits a signed, identity-bound record on a write path the runtime cannot reach, which is the property a regulator tests when the question is whether the system under audit could have altered its own evidence.
- Can I run TrueFoundry and DeepInspect together?
Yes, and it is the pattern for a regulated deployment already invested in TrueFoundry. TrueFoundry serves the models, orchestrates GPUs, and routes traffic. DeepInspect sits at the boundary in front of the gateway, binds the natural-person identity, classifies the prompt, applies the per-role policy, and commits the signed record independently. The platform handles the runtime. The proxy handles authorization and evidence.
- How is DeepInspect different from TrueFoundry?
TrueFoundry is a Kubernetes platform that serves, routes, and governs models across one control plane. DeepInspect is a stateless proxy that authorizes each HTTP AI call against a named identity and signs the decision, independent of any serving platform. TrueFoundry's audit sits inside the runtime. DeepInspect's audit sits outside it. One is the AI platform. The other is the decoupled policy and evidence layer that can front it or any provider.