← All posts

Platform & Architecture

145 posts on platform & architecture.

Agent-to-Agent TLS: Mutual Authentication Between AI Agents in a Multi-Agent Workflow

A multi-agent workflow chains AI agents where each agent calls the next over an HTTP transport. The security posture of the chain depends on the mutual authentication between the agents at each hop. This piece walks through the mTLS pattern for agent-to-agent authentication, the certificate lifecycle, and the inspection-layer architecture that binds every agent-to-agent call to a verified identity pair.

agent-to-agentmtlsai-agent-securitymulti-agentai-engineering
Read post →

LLM gateway vs LLM router: what each component does and why the enforcement layer sits in only one of them

The LLM gateway and the LLM router occupy different layers in an AI stack even when a vendor bundles them under one label. The gateway is the identity-aware policy enforcement point that sits between authenticated users and any LLM. The router is the traffic-shaping component that decides which model handles a given request. Confusing the two produces predictable failure modes at audit time. This piece walks through the two components, the fields each layer records, and how the policy decision at the gateway constrains what the router is allowed to route to.

llm-gatewayllm-routerai-architectureai-policy-enforcementai-control-plane
Read post →

The LLM inference gateway: what sits between authenticated callers and the model, and what belongs somewhere else

The LLM inference gateway is the identity-aware policy enforcement point between authenticated users or agents and any model endpoint. It is the layer where authorization, data classification, and audit-record production live. This piece defines the term, walks through the four fields the gateway resolves per request, contrasts it with the inference server, model router, and API gateway it is often confused with, and shows why the audit-write path must be isolated from the caller. Applies to any deployment running an OpenAI-compatible or provider-native LLM API in production.

llm-gatewayllm-inferenceai-architectureai-control-planepolicy-enforcement
Read post →

LLM routing strategies: five patterns for production, and where the policy decision constrains each one

LLM routing strategies decide which model, provider, or endpoint handles a given request. Five patterns cover most production deployments: static routing, cost-optimized routing, quality-tiered routing, latency-budgeted routing, and fallback routing. Each pattern operates on request metadata after the policy decision at the gateway has authorized the request and produced the audit record. This piece walks through the five patterns, what each optimizes for, and the constraints the gateway places on all of them.

llm-routerllm-routingai-architectureai-gatewaymulti-model
Read post →

LLM fallback routing: the retry chain that survives provider outages without leaking policy

LLM fallback routing chains a primary model to a secondary and tertiary so provider outages, rate-limit errors, and quality regressions do not cause user-visible failures. The failure modes are usually not the fallback logic itself but the boundary between the fallback chain and the policy decision that authorized the request. This piece walks through the four common triggers for fallback, the retry semantics per trigger, the authorized-endpoint constraint, and the idempotency requirements for tool-calling workloads.

llm-routerllm-fallbackai-architectureai-gatewayreliability
Read post →

LLM multi-model routing: the invariants that hold when you serve traffic from more than one provider

LLM multi-model routing spreads traffic across two or more model providers so a single-vendor outage, price change, or policy shift does not stop production. The pattern is simple in principle and complicated in practice because different providers have different token formats, streaming semantics, tool-call schemas, and safety-refusal patterns. This piece walks through the six invariants that hold regardless of provider (identity resolution, classification, policy, audit, idempotency, and response normalization) and the three variances that do not (token accounting, streaming chunking, and tool-call format).

llm-routermulti-modelai-architectureai-gatewayanthropicopenaibedrock
Read post →

AI response tool-call validation: the five checks that run before a tool call reaches the executor

When an LLM response contains a tool call, the tool call sits between the model output and a side effect in a real system. Untouched tool calls execute whatever the model produced, including hallucinated tools, malformed arguments, and unauthorized parameters. Production deployments run five checks at the gateway before the tool call reaches the executor: schema validation, tool-allowlist check, argument authorization, idempotency-key attachment, and audit-record production. This piece walks through each check, the failure modes it catches, and how the checks compose across the OpenAI, Anthropic, and Bedrock tool-call formats.

tool-callingai-agentai-gatewayagent-securityllm-response
Read post →

AI tool-use authorization: what the caller can invoke, what the model is allowed to attempt, and where the line sits

AI tool-use authorization decides which tools an LLM caller can invoke, which arguments the caller can pass, and which tool calls the model is allowed to attempt on the caller behalf. Production deployments enforce three layers: caller-role authorization (what the identity is entitled to use), argument-value authorization (what values fall inside the caller scope), and model-behavior authorization (which tool call sequences the deployer permits). This piece walks through the three layers, the failure modes each one catches, and the evidence each layer produces on the per-decision audit record.

tool-callingai-agentauthorizationai-gatewayagent-security
Read post →

Policy as Code for AI: The Review Pipeline That Turns AI Policy From a Config Screen Into a Reviewed Artifact

AI policies expressed as configuration screens inside a gateway UI change without version control, without code review, and without a rollback plan. The same policies expressed as code (Rego, Cedar, or JSON schemas checked into git) inherit the review pipeline the rest of the codebase runs on. This covers the operational pattern, the language choices, and how policy-as-code maps to SOC 2 CC8.1, ISO 27001 8.28, and EU AI Act Article 26 change management.

policy-as-codeai-governanceai-securitydevopscomplianceai-gateway
Read post →

AI Agent Observability: The Signals That Turn Autonomous Behavior From a Black Box Into a Debuggable System

Application observability (metrics, logs, traces) misses the signals that matter for AI agents: which tools the agent called, which sub-agent it delegated to, which policy decision permitted or denied each step, how many tokens each decision cost. Autonomous behavior without per-step observability is an unauditable black box. This covers the signals a production agent has to emit, the OpenTelemetry semantic conventions taking shape, and where the AI request boundary fits in the telemetry pipeline.

ai-agent-observabilityai-agent-securityopentelemetryai-securityagentic-aitraces
Read post →

AI Request Authorization Model: The Four Predicates Every Production AI Call Has to Answer

Every production AI request has four authorization predicates the enforcement layer has to evaluate: which identity, which model, which data classification, and which organizational policy. Missing any one produces an audit gap the regulator, the SIEM, or the incident-response team surfaces later. This walks through the four predicates, the input attributes each requires, and the policy engine pattern that composes them.

ai-authorizationai-policy-enforcementai-securityauthorization-modelai-gatewayzero-trust-ai
Read post →

AI Bug-Hunting Drove ~1,500 High-Severity CVEs in June 2026: Prioritizing the Patch Window

AI models are now finding software flaws at scale. Epoch AI counted roughly 1,500 high-severity and critical CVEs reported in June 2026, about 3.5x the previous monthly record, while real exploitation still arrives within hours of disclosure. This walks through why patch prioritization by exploitability, plus identity-bound inline policy at the AI request boundary, is the control that holds between disclosure and patch.

ai-securityllm-securitycybersecurityinline-enforcementarchitecture
Read post →