How is zero trust for AI different from zero trust for traditional APIs?

The unit of policy is different. Traditional APIs have structured payloads and well-defined operations; authorization at the API call boundary is the right grain. AI requests carry unstructured prompt content composed at runtime from many sources, with data classifications that vary request to request. Authorization has to evaluate the prompt content at the moment of the request, which means the enforcement layer has to read the prompt body before the request reaches the model.

What is the post-authentication gap?

The gap between "user is authenticated" and "this specific request is permitted under the policy in effect right now." Authentication answers who is calling. Per-request authorization at the AI layer answers what this caller is permitted to do with this prompt content. Most deployments answer the first and leave the second unaddressed. Meta's March 18 Sev-1 is the canonical example of a fully authenticated request that should have been denied at the prompt level.

Does inline enforcement add unacceptable latency for AI requests?

End-to-end enforcement overhead measures under 50 ms in production tests. LLM inference takes 500 ms to 5 seconds. The enforcement overhead is invisible relative to model response time. There is no architectural cost to making enforcement inline, which means the log-and-alert pattern from earlier security generations is the wrong default for AI traffic.

How do we deploy the enforcement layer without modifying every application?

Service-mesh integration, sidecar deployment, and gateway proxying all work without modifying application code. The mesh or gateway intercepts outbound HTTPS to LLM endpoints and routes through the enforcement proxy. The application continues to call the model API as before; the proxy is transparent to the application's HTTP client.

What policies do we start with in a new zero-trust-for-AI deployment?

Start with three policies. First, identity attachment: every model request carries a verified identity object. Second, PII redaction: customer PII is detected and redacted in prompts to external models. Third, regulated-data block: pre-announcement financials, PHI, and source code are blocked from external models entirely. Production deployments add per-role and per-route policies on top of these foundations as the deployment matures.

Zero Trust AI: Per-Request Evaluation at the Model Boundary

Zero trust is the architectural principle that no request is trusted by default. Every request is authenticated, authorized, and evaluated against the policy in effect at the moment of the request. The principle is well-understood for network traffic, application APIs, and data access. For AI requests, the implementation lags. Most deployments authenticate the user, authorize the API call, and stop there. The prompt content, the data classification, and the policy state at the moment of the request are not evaluated.

I want to walk through what zero trust looks like applied to AI traffic, where the post-authentication gap sits in current deployments, and what an enforcement architecture has to do to make per-request evaluation operational at machine speed.

The Mandiant M-Trends 2026 report measured median attack handoff at 22 seconds. Zero trust applied to AI has to fire inline at that tempo. The decision happens before the prompt reaches the model.

Where zero trust diverges for AI traffic

Network-layer zero trust evaluates packets against identity, device posture, and segmentation policy. Application-layer zero trust evaluates API calls against user identity and role-based permissions. Both work because the unit of policy (the packet, the API call) carries enough context to make the decision.

AI traffic is different. The unit of policy is the prompt. The prompt is unstructured natural-language text composed at runtime from many sources: the user's input, retrieved documents, tool call results, prior agent reasoning, and synthesized context. Identity-based authorization at the API call boundary answers "is this caller permitted to use the API." It does not answer "is this caller permitted to send this prompt content right now."

The post-authentication gap is exactly this distinction. Authentication answers "who is calling." Authorization at the AI call layer answers "what is this caller permitted to do with this prompt content right now." Most deployments answer the first and leave the second unaddressed.

Meta's March 18 Sev-1 is the canonical example. An internal AI agent exposed sensitive user and company data to engineers who were fully authenticated. The exposure lasted two hours. Meta classified it as Sev-1. The engineers had API access. They did not have authorization for the specific data the agent included in its response. Authentication was correct; per-request authorization at the prompt level was absent.

What zero trust applied to AI requires

Verified identity per request

Every AI request carries verified identity context: the natural person, the agent (if applicable), the role, the delegation scope, and a timestamp. Verified means the identity is bound to the deployer's identity system, not asserted by the calling application. NIST Pillar 1 of the AI agent identity and authorization framework names this requirement.

Per-request policy evaluation

Every request is evaluated against the policies in effect at that moment. Per-route policies attach to model endpoints. Per-role policies attach to user and agent roles. Per-decision policies attach to the data classification of the prompt. The evaluation is deterministic and fails closed: ambiguity or error defaults to deny.

Prompt-level classification

The classifier reads the prompt body before the request reaches the model. PII, regulated data, source code, pre-announcement financials are detected at the field level. The classifier feeds into the policy decision point, which selects pass, redact, or block.

Independent audit record

Every decision produces a per-decision audit record committed before the model response returns to the application. The record store has its own retention schedule and access controls, separate from the application's log infrastructure. The system under audit is not the system writing the audit record.

Inline enforcement at machine speed

A blocked request never reaches the model. A blocked response never reaches the application. Enforcement overhead in production tests measures under 50 ms, well inside the 500 ms to 5 second window LLM inference takes. The overhead is invisible relative to the model's response time.

The architectural position of the enforcement layer

The enforcement layer sits at the HTTP AI request boundary, in the path between the application or agent and the LLM endpoint. The proxy intercepts every call to OpenAI, Anthropic, Bedrock, Azure OpenAI, Vertex, or self-hosted inference. The policy decision point evaluates identity, role, prompt classification, and tool authorization. The audit record is signed and committed before the model response returns to the application.

Deployment topologies that work in production include: sidecar deployment alongside the application, service-mesh integration through Envoy or Istio, and gateway integration through Kong, Apigee, or a custom proxy layer. The architectural position is constant; the implementation varies with the deployer's existing infrastructure.

Why the existing stack does not deliver zero trust for AI

Network DLP

Network-layer DLP runs underneath TLS encryption. The HTTPS POST to the model endpoint is encrypted at the network layer. The prompt content is invisible to DLP unless TLS inspection is configured for AI provider domains and the API payload is parsed for prompt fields. Even with that configuration, document-level classification produces false negatives across prompt traffic because prompt context windows are unstructured natural-language text.

CASB

Cloud Access Security Brokers were built for SaaS application visibility. They classify sanctioned and unsanctioned cloud apps. They do not parse prompts. AI requests appear as outbound HTTPS to recognized cloud destinations. Per-request prompt-level classification is outside the CASB control plane.

IAM-only authorization

Identity and access management answers who is permitted to call an API. It does not evaluate the prompt content of each call. IAM is necessary for authentication and for the workload identity Pillar 1 requires. It is not sufficient for the per-request authorization Pillar 2 requires.

Model-side guardrails

Model providers ship safety training into their models. Refusal patterns, RLHF, constitutional AI. These behaviors live inside the inference process. They are probabilistic, not enforceable. Stanford Trustworthy AI research, alongside the AIUC-1 Consortium briefing, found that refusal behaviors of model-level guardrails are significantly degraded under targeted fine-tuning and adversarial pressure.

Application logs

Application-controlled logs fail under selective logging, suppression, and crash-loss conditions. The system under audit is the system generating the audit record. Zero trust requires that audit evidence be independent of the system being audited.

Compliance lens

The same architecture satisfies multiple regulatory regimes at once.

EU AI Act Article 12 mandates automatic recording of events over the system lifetime for high-risk AI systems, effective August 2, 2026. Article 19 specifies that logs include the period of use, the input data, the reference databases checked, and the identification of natural persons involved. Penalties under Article 99 reach €15 million or 3% of global annual turnover.

Fannie Mae LL-2026-04, effective August 6, 2026, requires governance, inventory, audit trails, and disclosure on demand for AI-assisted decisions in mortgage origination and servicing.

NIST AI RMF, ISO 42001, and the NIST AI agent identity and authorization framework reach the same architectural conclusion through different language. Per-decision evaluation, identity-bound records, independent audit trails. Each regime uses different vocabulary for the same infrastructure requirements.

DeepInspect

This is the architecture DeepInspect was built to provide. DeepInspect sits at the AI request boundary as an external enforcement layer between the application or agent and the LLM API. The proxy reads the identity object the application supplies, evaluates per-route and per-role policies, applies prompt-level classification, and produces a per-decision audit record signed at the moment of evaluation.

The proxy is model-agnostic and works in front of OpenAI, Anthropic, Bedrock, Azure OpenAI, Vertex, and on-prem inference endpoints. The audit record store has its own retention schedule and its own access controls. Pillar 1 sits with the application; Pillars 2 and 3 of the NIST framework sit with the proxy.

If you are running AI in a regulated environment and your zero-trust posture stops at authentication and IAM, the post-authentication gap is where the exposure sits.