← All posts

Platform & Architecture

227 posts on platform & architecture.

LLM multi-model routing: the invariants that hold when you serve traffic from more than one provider

LLM multi-model routing spreads traffic across two or more model providers so a single-vendor outage, price change, or policy shift does not stop production. The pattern is simple in principle and complicated in practice because different providers have different token formats, streaming semantics, tool-call schemas, and safety-refusal patterns. This piece walks through the six invariants that hold regardless of provider (identity resolution, classification, policy, audit, idempotency, and response normalization) and the three variances that do not (token accounting, streaming chunking, and tool-call format).

llm-routermulti-modelai-architectureai-gatewayanthropicopenaibedrock
Read post →

AI response tool-call validation: the five checks that run before a tool call reaches the executor

When an LLM response contains a tool call, the tool call sits between the model output and a side effect in a real system. Untouched tool calls execute whatever the model produced, including hallucinated tools, malformed arguments, and unauthorized parameters. Production deployments run five checks at the gateway before the tool call reaches the executor: schema validation, tool-allowlist check, argument authorization, idempotency-key attachment, and audit-record production. This piece walks through each check, the failure modes it catches, and how the checks compose across the OpenAI, Anthropic, and Bedrock tool-call formats.

tool-callingai-agentai-gatewayagent-securityllm-response
Read post →

AI tool-use authorization: what the caller can invoke, what the model is allowed to attempt, and where the line sits

AI tool-use authorization decides which tools an LLM caller can invoke, which arguments the caller can pass, and which tool calls the model is allowed to attempt on the caller behalf. Production deployments enforce three layers: caller-role authorization (what the identity is entitled to use), argument-value authorization (what values fall inside the caller scope), and model-behavior authorization (which tool call sequences the deployer permits). This piece walks through the three layers, the failure modes each one catches, and the evidence each layer produces on the per-decision audit record.

tool-callingai-agentauthorizationai-gatewayagent-security
Read post →

Policy as Code for AI: The Review Pipeline That Turns AI Policy From a Config Screen Into a Reviewed Artifact

AI policies expressed as configuration screens inside a gateway UI change without version control, without code review, and without a rollback plan. The same policies expressed as code (Rego, Cedar, or JSON schemas checked into git) inherit the review pipeline the rest of the codebase runs on. This covers the operational pattern, the language choices, and how policy-as-code maps to SOC 2 CC8.1, ISO 27001 8.28, and EU AI Act Article 26 change management.

policy-as-codeai-governanceai-securitydevopscomplianceai-gateway
Read post →

AI Request Authorization Model: The Four Predicates Every Production AI Call Has to Answer

Every production AI request has four authorization predicates the enforcement layer has to evaluate: which identity, which model, which data classification, and which organizational policy. Missing any one produces an audit gap the regulator, the SIEM, or the incident-response team surfaces later. This walks through the four predicates, the input attributes each requires, and the policy engine pattern that composes them.

ai-authorizationai-policy-enforcementai-securityauthorization-modelai-gatewayzero-trust-ai
Read post →

AI Bug-Hunting Drove ~1,500 High-Severity CVEs in June 2026: Prioritizing the Patch Window

AI models are now finding software flaws at scale. Epoch AI counted roughly 1,500 high-severity and critical CVEs reported in June 2026, about 3.5x the previous monthly record, while real exploitation still arrives within hours of disclosure. This walks through why patch prioritization by exploitability, plus identity-bound inline policy at the AI request boundary, is the control that holds between disclosure and patch.

ai-securityllm-securitycybersecurityinline-enforcementarchitecture
Read post →

Securing MCP Servers in Production: An Implementation Guide

A hands-on guide to running Model Context Protocol servers securely on the HTTP transport: stand up the server as an OAuth 2.1 resource server, validate token audience per request, authorize each tools/call against role and arguments, propagate caller identity downstream, constrain egress, and log every decision. Includes policy and validation snippets and a production checklist, plus the line where stdio servers leave the gateway.

ai-securityagentic-aillm-securityidentity-and-authorizationarchitecture
Read post →

MCP Gateway Security: Enforcing Policy on Remote Tool Traffic

A remote MCP server turns tool calls into HTTP traffic between an agent and an endpoint. That traffic is where an MCP gateway enforces identity-bound policy and records each tool invocation. This covers what an MCP gateway checks, why remote Streamable HTTP transport is the enforcement surface, and where local STDIO servers sit outside it.

mcpai-securityagentic-aipolicy-enforcementinline-enforcementarchitecture
Read post →

MCP OAuth 2.1 Security: Authorization Enforcement for Remote Servers

MCP''s authorization spec builds on OAuth 2.1, which sets the token rules but leaves the per-call decision to you. This walks through the MCP OAuth flow for remote servers, the confused-deputy risk it introduces, and why an enforcement point that evaluates every tool call against the acting identity is the control that OAuth alone does not provide.

mcpidentity-and-authorizationai-securityagentic-aipolicy-enforcementarchitecture
Read post →

LLM Gateway Security: What an Enforcement Layer Checks on Every Request

An LLM gateway that only routes and rate-limits leaves the security decision to the model. This walks through what a security-grade LLM gateway evaluates on every request: verified identity, model authorization, prompt-level data classification, a fail-closed decision, and a per-decision audit record committed before the response returns.

llm-securityai-securityinline-enforcementpolicy-enforcementarchitectureaudit
Read post →

Langflow CVE-2026-55255: Multi-Tenant Secret Isolation and AI Provider Keys

On July 8, 2026 Help Net Security reported active exploitation of CVE-2026-55255, a CVSS 9.9 cross-tenant IDOR in Langflow that lets one tenant read another tenant's embedded secrets: LLM provider keys, cloud credentials, database passwords. CISA set a July 10 federal mitigation deadline. The architectural lesson is about where AI orchestration tools keep long-lived provider keys, and why a stateless gateway that holds none of them gives a secret-harvest bug far less to reach.

ai-securityllm-securitycybersecurityai-gatewayarchitecture
Read post →

AI Agent Memory Security: What a Gateway Sees, and What Persists Where It Cannot Reach

Agent memory spans the context window, provider-hosted long-term memory, vector-store RAG memory, and local file state. Attacks like MINJA, SpAIware, the Gemini delayed-tool-invocation poisoning, and the July 2026 MemGhost paper corrupt that memory through prompt content. This article maps each memory type and attack to the HTTP AI request boundary, and is blunt about which steps a gateway can see and which persist where it cannot reach.

agentic-aillm-securityprompt-injectionai-securityarchitectureforensic-audit
Read post →