← All posts

Platform & Architecture

145 posts on platform & architecture.

AI Security for Coding Agents: The Source-Code, Secret, and Action Boundaries the Agent Crosses

Coding agents read source code, write code changes, run shell commands, call external APIs, and commit results back to the repository. The agent crosses multiple action boundaries inside a single workflow with the developer identity at the top and machine credentials at the bottom. This piece walks through the source-code data the agent reads at request time, the secret-handling surface the agent exposes, the action boundaries the inspection layer commits decisions at, and the audit record format the security team and the regulator consume.

coding-agentai-securityidentity-awareaudit-logsinline-enforcementdeveloper-tools
Read post →

OpenAI API Gateway Setup: An Implementation Walkthrough for Enterprise Deployments

A production OpenAI deployment that satisfies EU AI Act Article 12, NIST AI RMF MANAGE 1.3, and HIPAA audit obligations needs a gateway between the application and api.openai.com. This guide walks through the gateway's request path, the TLS handling, the identity model, the four classification stages, and the audit-record format that holds up under a regulator read. Code samples included.

openaiai-gatewayimplementation-guideenterprise-aiai-securityapi-proxy
Read post →

Anthropic API Gateway Setup: An Implementation Walkthrough for Enterprise Claude Deployments

Direct integrations with api.anthropic.com terminate TLS at Anthropic's edge, which leaves the deployer with no inspection point and no audit record. This guide walks through the gateway architecture that sits between the application and Anthropic's API, with attention to Claude-specific patterns: system prompts, tool use, prompt caching, the SSE streaming format, MCP connector requests, the published IP ranges for firewall rules, and the full error surface a proxy has to handle.

anthropicclaudeai-gatewayimplementation-guideenterprise-aiapi-proxy
Read post →

Securing the Inference Lifecycle: The Five Stages Where the Enforcement Layer Has To Sit

The AI inference lifecycle is the sequence the application runs every time the model produces a response. Most security programs cover model training and the post-deployment monitoring stages but leave the inference path itself uninstrumented. This piece walks through the five stages of the inference lifecycle, the control points each stage exposes at the request boundary, the per-decision audit record the deployment has to commit, and the architectural pattern that closes the inference-time gaps a 2022-era AppSec program leaves open.

inference-lifecycleai-securityinline-enforcementaudit-logsai-architecturepolicy-enforcement
Read post →

AI Security for Engineering Copilots: The Identity, Source-Code, and Audit Controls a Production Deployment Has To Run

Engineering copilots reach across the source repository, the build infrastructure, the package registry, and the production credential store. The decisions the copilot supports cross export-control boundaries, the customer source-code confidentiality terms, and the secret-handling rules the security team has built. This piece walks through the identity-aware policy decisions an engineering copilot deployment has to commit at the request boundary, the audit record format that survives SOC 2 Type II and customer audit, and the architectural pattern that closes the gap.

engineering-copilotsource-codesecrets-managementai-securitysoc-2audit-logs
Read post →

Model Context Protocol Security: How the MCP Transport Layer Changes the Inspection Boundary

The Model Context Protocol standardizes how an LLM client connects to tool servers and exchanges context, tool calls, and tool results. The 2026-07-28 revision removed protocol-level sessions and the GET stream, and it now mirrors the method and target name into HTTP headers specifically so intermediaries can route and inspect a call without parsing the body. This piece walks through the two transports, what the Streamable HTTP request actually carries, the OAuth 2.1 requirements the spec places on servers, the identity-aware policy decisions a deployment commits per call, and the audit record format that survives an Article 12 review.

mcpmodel-context-protocolagent-securityai-toolinginline-enforcementaudit-logs
Read post →

Zero Trust LLM: How the Zero-Trust Principles Apply to AI Request Flows

Zero trust applied to LLM traffic means three things at the architectural level. Identity is verified at every request, not just at the session. Authorization is evaluated per request against the user, agent, role, and resource. The audit record is written independently of the application or the model that handled the request. The three principles map directly to the inspection-layer pattern that closes the post-authentication gap in AI deployments.

zero-trustllm-securityinline-enforcementai-policy-enforcementidentityaudit-logs
Read post →

AI Gateway Latency: Why Sub-50ms Overhead Sits Below the Noise Floor of LLM Inference

LLM inference takes 500 ms to 5 seconds per response. A well-engineered AI gateway adds under 50 ms of overhead in internal testing. The 10x gap between inference time and gateway overhead is the architectural fact that makes inline enforcement viable for regulated production AI. The latency budget across policy evaluation, prompt classification, identity validation, and audit commit fits inside the 50 ms envelope under realistic load.

ai-gatewaylatencyinline-enforcementperformanceai-policy-enforcementaudit-logs
Read post →

AI Gateway Architecture: The Components That Sit Between an Enterprise Caller and an LLM Endpoint

An AI gateway architecture has six core components: TLS termination, identity binding, request inspection, policy evaluation, the model router, and the audit record emitter. Each component is a placement decision that ties to a regulatory obligation or an operational property. This piece walks through the components, the placement decisions, and how the gateway integrates with the corporate IdP and the SIEM.

ai-gateway-architectureai-gatewayai-securityinline-enforcementaudit-logs
Read post →

AI Gateway Performance Benchmark: What to Measure and How

AI gateway performance benchmarks compare proxy products on latency, throughput, and behavior under load. The benchmarks that matter for production deployment are p95 and p99 latency under realistic concurrency, tail-latency behavior when policy evaluation gets expensive, throughput ceiling per node, and behavior under upstream provider degradation. This piece walks through the benchmark methodology that produces production-actionable numbers and the comparison points worth tracking.

ai-gatewayperformancebenchmarklatencyengineering
Read post →

AI Agent Control Plane: Identity, Authorization, and Action Lineage

An AI agent control plane is the architectural layer that authorizes agent actions, enforces identity-bound policy on each action, and records action lineage for audit. The pattern emerged because the chatbot architecture (one prompt, one response, one log) does not cover the action surface autonomous agents produce. This piece walks through the control plane primitives, the integration points with the agent framework, and the performance characteristics the layer needs to maintain under production load.

ai-agentsagentic-aiai-control-planeauthorizationai-securityarchitecture
Read post →

AI Gateway Sub-50ms Latency: What the Number Actually Buys You

Sub-50ms latency on an AI gateway sets the per-request overhead below the noise floor of LLM inference (500ms to 5 seconds). The architectural property the number reflects is local policy evaluation, in-memory classification, and stateless horizontal scaling. This piece walks through how the budget is spent, where the latency typically hides, the benchmark methodology that produces production-actionable numbers, and how sub-50ms behavior changes the decision about inline versus out-of-band enforcement.

ai-gatewaylatencyperformanceinline-enforcementarchitectureengineering
Read post →