Anthropic API Gateway Patterns: How To Front api.anthropic.com with Inline Enforcement

An Anthropic API gateway addresses the inspection layer instead of api.anthropic.com. The application replaces the base URL on its Anthropic client and ships every Messages, tool-use, and streaming request through the gateway. The gateway authenticates the caller, attaches identity context, evaluates a policy bundle against the request, commits a per-decision audit record, and forwards the request to Anthropic on a controlled credential. The response returns through the same path with the gateway recording the response handling decision. The pattern covers the synchronous Messages endpoint, the streaming Server-Sent Events response, the tool-use loop, the prompt-caching feature, and the agentic computer-use beta.

I want to walk through the base URL pattern, the system-prompt evaluation, the tool-use policy, the streaming handling, and the audit record format that holds up under EU AI Act Article 12 review.

The base URL replacement pattern

The Anthropic Python SDK accepts a base_url parameter on Anthropic(base_url=...). The TypeScript and Go SDKs accept the same override. The application points the SDK at the inspection layer's URL instead of https://api.anthropic.com/v1, and every subsequent call reaches the inspection layer. The inspection layer authenticates the caller (using the existing application identity primitive: JWT, OIDC, service account credential), evaluates the policy, commits the audit record, and forwards to Anthropic on the gateway's own Anthropic API key.

The application never holds the Anthropic key. The inspection layer holds the key in a controlled secret store with rotation. Most production deployments wrap the SDK construction in an internal package so the base_url and the identity-token attachment are centralized across the codebase. New applications that ship under the package automatically reach the inspection layer.

The pattern works for the official Anthropic SDKs and for any HTTP client that calls the Messages API directly. It works for the LangChain, LlamaIndex, and other framework integrations that respect a base URL override. The pattern fails for hard-coded integrations that bypass the SDK, which is rare in modern codebases; the workaround is a network-layer HTTP proxy that intercepts the Anthropic hostname.

System-prompt evaluation

The Anthropic Messages API takes a system parameter that carries the system prompt separately from the conversation messages. The system prompt is where most production deployments encode their application-level behavior. It is also where attackers target prompt injection that overrides the application's intended behavior.

The inspection layer evaluates the system prompt against the policy on every request. The evaluation has three components. The first is identity-scope match: the system prompt has to come from a registered system-prompt set for the calling route, which prevents an attacker who has gained API access from substituting a different system prompt for the same model. The second is content classification: the policy inspects the system prompt for content that the policy wants to reject (for example, a system prompt that instructs the model to disregard prior instructions, which is one of the prompt-injection signatures). The third is integrity: the system prompt is fingerprinted in the audit record so the auditor can verify what was sent to the model.

The conversation messages get the same data classification as the system prompt but with the route's content rules applied. A regulated-data class detected in a user message can be redacted before the request forwards to Anthropic. The audit record stamps the data classification outcome and the modifications.

Tool-use policy

Anthropic's tool-use feature lets the model emit a structured tool-call response that the application is supposed to execute and feed back. The pattern produces "agentic" loops that the inspection layer has policy implications for at three points.

The first is at request submission, where the inspection layer evaluates the tool list. The application sends a tools array on the request, and the policy can match against the tool names, the tool schemas, and the route identifier to decide whether this caller is permitted to invoke these tools through Claude.

The second is on Claude's tool-call response. Claude returns a tool_use content block that names the tool to call and the arguments. The inspection layer evaluates the proposed call against the policy. A tool_use block that proposes calling a delete_record tool with arguments referencing regulated record types can be blocked at the inspection layer before the application executes the tool. The audit record captures the proposed call, the policy decision, and the outcome.

The third is on the follow-up request that carries the tool result back to Claude. The inspection layer evaluates the result against the policy. A tool result that contains PII or other regulated content can be redacted before it reaches Claude, which closes the leak channel that the tool-use loop opens by default.

The audit record for an agentic loop is a series of per-decision records with a correlation identifier that ties the records together. The auditor reconstructs the full loop from the record series.

Streaming response handling

The Anthropic Messages API supports streaming through Server-Sent Events. The streaming response shape is structured: message_start, content_block_start, content_block_delta (token chunks), content_block_stop, message_delta, message_stop. The inspection layer handles the stream as a pass-through with policy hooks on the request and on the chunked response.

The request-time evaluation runs on the full request body before forwarding. The response-time evaluation runs on the streamed chunks with a fast classifier that detects regulated content in the streamed output. A response-time block aborts the upstream stream and returns a structured error to the application. The audit record commits on stream close (success) or stream abort (block, error) with the request body fingerprint, the streamed response fingerprint, the policy decision, and the integrity signature.

The streaming pattern matters because Anthropic's reasoning-heavy responses (Claude Sonnet 4 and Opus 4 working through long-form analysis) take seconds to complete and stream tokens throughout. The inspection layer's overhead measures in milliseconds and is invisible relative to the model's response time. The end-to-end inspection-layer overhead measures under 50 ms in production deployments.

The prompt-caching pattern

Anthropic introduced prompt caching as a cost-reduction feature: prefix portions of the prompt that recur across requests can be cached on Anthropic's side and reused at a lower per-token cost. The inspection layer has to recognize the cached-prefix marker on the request and apply the policy with the cached prefix understood as part of the prompt. A naive inspection layer that evaluated the policy on only the non-cached suffix would fail policies that depend on the cached prefix (the system instructions, the document context, the tool definitions).

Production deployments handle this by resolving the cached prefix at request time, either by reading the cached content from the application's own copy or by treating the cached prefix as a known fingerprint that the policy was authored against. The audit record captures the cached prefix identifier and the policy evaluation result against the full prompt (cached prefix plus current suffix).

Identity attribution at the Anthropic request

Anthropic's API logs the call against the API key that authenticated it. A direct integration that uses a single organization-bound key produces logs attributed to the organization, with no identity for the natural person whose action triggered the request. The inspection layer attaches the natural-person identity, the tenant, the role, and the route identifier to the audit record on the gateway side. Anthropic's logs and the inspection layer's audit records compose the record series that satisfies the EU AI Act Article 12 traceability test.

The same pattern applies to Amazon Bedrock-served Claude and to Google Vertex-served Claude. The inspection layer addresses the Bedrock or Vertex endpoint instead of api.anthropic.com, but the identity attribution, the policy evaluation, and the audit record format are the same.

DeepInspect

This is the gateway pattern DeepInspect runs in front of Anthropic and any other LLM endpoint over HTTP. DeepInspect addresses api.anthropic.com, the Bedrock Claude endpoints, and the Vertex Claude endpoints. The application sets the base URL on its Anthropic SDK to the inspection layer's URL and ships every request through. DeepInspect verifies the caller's identity from the application's existing identity primitive, evaluates the policy bundle for the route, commits the per-decision audit record, and forwards the request to Anthropic on a controlled credential.

The audit record carries identity, policy version, model and version (claude-sonnet-4-20250514, claude-opus-4-20250514), system-prompt fingerprint, tool-use proposals and outcomes, request and response fingerprints, decision outcome, and a cryptographic integrity signature. The same record format covers every Anthropic request type and the tool-use loop. The architecture works for the Messages API, the streaming responses, the tool-use loop, the prompt-caching feature, and the computer-use beta.

If you are running a direct integration to api.anthropic.com and the security review is asking for identity attribution at the model API call, let's talk.