← All posts

Platform & Architecture

338 posts on platform & architecture.

Anthropic API Gateway Patterns: How To Front api.anthropic.com with Inline Enforcement

An Anthropic API gateway sits between calling applications and api.anthropic.com, attaches identity context, evaluates a per-request policy, and commits a per-decision audit record before the request reaches Claude. The gateway pattern addresses the Anthropic Messages API, the tool-use loop, the streaming response, and the prompt caching feature. This piece walks through the request rewriting pattern, the system-prompt evaluation, the tool-use policy, the streaming SSE handling, and the audit record format that satisfies EU AI Act Article 12 and the deployer obligations under Article 26.

anthropicclaudeai-gatewayapi-gatewayinline-enforcementaudit
Read post →

Amazon Bedrock Gateway Patterns: How To Front Bedrock with Inline Enforcement

An Amazon Bedrock gateway sits between calling applications and the Bedrock runtime endpoints, attaches identity context to every InvokeModel and InvokeModelWithResponseStream call, evaluates a per-request policy, and commits a per-decision audit record before the request reaches Anthropic, Mistral, Meta, Cohere, AI21, or Amazon Titan. The gateway pattern complements Bedrock Guardrails by adding identity-bound policy enforcement and a per-decision audit record format that satisfies EU AI Act Article 12 and the Fannie Mae LL-2026-04 lender record requirement. This piece walks through the AWS SigV4 handling, the model-agnostic policy, and the audit record format.

amazon-bedrockai-gatewayapi-gatewayinline-enforcementawsaudit
Read post →

AI Inline Enforcement Architecture: Where the Policy Decision Sits and What It Has To Commit

AI inline enforcement runs the policy decision in the request path, before the model API call returns to the calling application. The architecture places a deterministic policy decision point between the application identity and the model endpoint and commits a per-decision audit record before the response forwards. This piece walks through the architectural components, the decision-time data shape, the failure modes the implementation has to handle, and the regulatory profile that the inline placement satisfies (EU AI Act Article 12, NIST AI agent identity and authorization Pillar 2 and Pillar 3, Fannie Mae LL-2026-04, DORA Article 6).

inline-enforcementai-architectureai-gatewaypolicy-decision-pointauditeu-ai-act
Read post →

AI Audit Log Immutability: Object Lock, WORM Storage, and the Storage-Layer Contract a Regulator Accepts

The reconstruction test a regulator applies during an AI audit assumes the log record has not been rewritten. The assumption fails when the log lives in a storage layer that permits modification by the same operator who runs the AI application. This piece walks through the immutability contract at the storage layer, S3 Object Lock and Azure Blob immutability policies as implementations, and the audit-record shape that verifies immutability by construction.

ai-audit-logsimmutabilityobject-lockcomplianceai-engineering
Read post →

AI Security for Coding Agents: The Source-Code, Secret, and Action Boundaries the Agent Crosses

Coding agents read source code, write code changes, run shell commands, call external APIs, and commit results back to the repository. The agent crosses multiple action boundaries inside a single workflow with the developer identity at the top and machine credentials at the bottom. This piece walks through the source-code data the agent reads at request time, the secret-handling surface the agent exposes, the action boundaries the inspection layer commits decisions at, and the audit record format the security team and the regulator consume.

coding-agentai-securityidentity-awareaudit-logsinline-enforcementdeveloper-tools
Read post →

Securing the Inference Lifecycle: The Five Stages Where the Enforcement Layer Has To Sit

The AI inference lifecycle is the sequence the application runs every time the model produces a response. Most security programs cover model training and the post-deployment monitoring stages but leave the inference path itself uninstrumented. This piece walks through the five stages of the inference lifecycle, the control points each stage exposes at the request boundary, the per-decision audit record the deployment has to commit, and the architectural pattern that closes the inference-time gaps a 2022-era AppSec program leaves open.

inference-lifecycleai-securityinline-enforcementaudit-logsai-architecturepolicy-enforcement
Read post →

AI Security for Engineering Copilots: The Identity, Source-Code, and Audit Controls a Production Deployment Has To Run

Engineering copilots reach across the source repository, the build infrastructure, the package registry, and the production credential store. The decisions the copilot supports cross export-control boundaries, the customer source-code confidentiality terms, and the secret-handling rules the security team has built. This piece walks through the identity-aware policy decisions an engineering copilot deployment has to commit at the request boundary, the audit record format that survives SOC 2 Type II and customer audit, and the architectural pattern that closes the gap.

engineering-copilotsource-codesecrets-managementai-securitysoc-2audit-logs
Read post →

Model Context Protocol Security: How the MCP Transport Layer Changes the Inspection Boundary

The Model Context Protocol standardizes how an LLM client connects to tool servers and exchanges context, tool calls, and tool results. The 2026-07-28 revision removed protocol-level sessions and the GET stream, and it now mirrors the method and target name into HTTP headers specifically so intermediaries can route and inspect a call without parsing the body. This piece walks through the two transports, what the Streamable HTTP request actually carries, the OAuth 2.1 requirements the spec places on servers, the identity-aware policy decisions a deployment commits per call, and the audit record format that survives an Article 12 review.

mcpmodel-context-protocolagent-securityai-toolinginline-enforcementaudit-logs
Read post →

Zero Trust LLM: How the Zero-Trust Principles Apply to AI Request Flows

Zero trust applied to LLM traffic means three things at the architectural level. Identity is verified at every request, not just at the session. Authorization is evaluated per request against the user, agent, role, and resource. The audit record is written independently of the application or the model that handled the request. The three principles map directly to the inspection-layer pattern that closes the post-authentication gap in AI deployments.

zero-trustllm-securityinline-enforcementai-policy-enforcementidentityaudit-logs
Read post →

AI Gateway Latency: Why Sub-50ms Overhead Sits Below the Noise Floor of LLM Inference

LLM inference takes 500 ms to 5 seconds per response. A well-engineered AI gateway adds under 50 ms of overhead in internal testing. The 10x gap between inference time and gateway overhead is the architectural fact that makes inline enforcement viable for regulated production AI. The latency budget across policy evaluation, prompt classification, identity validation, and audit commit fits inside the 50 ms envelope under realistic load.

ai-gatewaylatencyinline-enforcementperformanceai-policy-enforcementaudit-logs
Read post →

AI Agent Control Plane: Identity, Authorization, and Action Lineage

An AI agent control plane is the architectural layer that authorizes agent actions, enforces identity-bound policy on each action, and records action lineage for audit. The pattern emerged because the chatbot architecture (one prompt, one response, one log) does not cover the action surface autonomous agents produce. This piece walks through the control plane primitives, the integration points with the agent framework, and the performance characteristics the layer needs to maintain under production load.

ai-agentsagentic-aiai-control-planeauthorizationai-securityarchitecture
Read post →

AI Gateway Sub-50ms Latency: What the Number Actually Buys You

Sub-50ms latency on an AI gateway sets the per-request overhead below the noise floor of LLM inference (500ms to 5 seconds). The architectural property the number reflects is local policy evaluation, in-memory classification, and stateless horizontal scaling. This piece walks through how the budget is spent, where the latency typically hides, the benchmark methodology that produces production-actionable numbers, and how sub-50ms behavior changes the decision about inline versus out-of-band enforcement.

ai-gatewaylatencyperformanceinline-enforcementarchitectureengineering
Read post →