← All posts

Platform & Architecture

145 posts on platform & architecture.

AI Gateway Redaction for RAG Contexts: Stopping Cross-Tenant Data Leakage

A retrieval-augmented generation pipeline fetches documents from a vector store, concatenates them into the prompt context, and sends the assembled prompt to the LLM. The fetched chunks can carry data the requesting user is not authorized to see. The model has no way to distinguish authorized content from leaked content. An AI gateway that redacts at the context-assembly boundary, with identity-bound policy on each retrieved chunk, is the architectural pattern that stops cross-tenant data leakage in RAG.

ai-gatewayragredactionai-securitypolicy-enforcementdata-leakage
Read post →

AI Gateway Multi-Tenant Isolation: Identity, Policy, and Audit at the Tenant Boundary

Multi-tenant AI deployments share infrastructure across tenants and have to enforce isolation at the request boundary. Tenant context attached at authentication time has to flow through every policy decision, every tool invocation, every retrieval call, and every audit record. A gateway that maintains the tenant boundary at all four touch points is the architectural pattern that keeps multi-tenant AI safe under load. This piece walks through where the tenant context has to land and what the audit record looks like when isolation holds.

ai-gatewaymulti-tenantai-securityisolationpolicy-enforcementaudit
Read post →

AI Red Team Methodology: A Six-Phase Framework for Adversarial Testing of LLM Applications

Most AI red team engagements run as ad-hoc prompt-injection tests against a chat interface and call the result a red team. A defensible methodology runs through six phases: scope and threat modeling, identity-context attacks, content-vector attacks, agent-layer escalation, multi-turn and persistence attacks, and post-engagement reporting against a remediation owner. This article walks through each phase, the techniques each phase deploys, the evidence the red team should capture, the remediation owner each finding routes to, and the integration points with the rest of the security program.

ai-red-teamadversarial-testingprompt-injectionagent-securitysecurity-testing
Read post →

Mapping a Zero-Trust AI Gateway to NIST''s Upcoming COSAiS Single-Agent and Multi-Agent Overlays

NIST is teeing up the Concept of Operations for Securing AI Systems (COSAiS) overlays in two forms: a Single-Agent overlay and a Multi-Agent overlay, plus an AI RMF Profile for Critical Infrastructure. Federal contractors and critical infrastructure operators will be measured against these. The pre-map advantage is real: federal procurement reviews already reference the work in progress. This article walks the overlay structure, where a zero-trust AI gateway maps to each control family, and the evidence artifact each control consumes.

nistnist-ai-rmfzero-trustai-securitycompliancearchitecture
Read post →

Model Routing for Cost: What to Actually Measure Before Switching a Workload from GPT-4 to Haiku

Most "use the cheaper model" posts skip the rigor. Real model routing decisions have four layers: token cost, quality regression on an eval set, latency impact, and governance risk. This article walks through each layer with the questions a platform engineer should answer before flipping a workload from a frontier model to a smaller one, plus an example routing rule expressed at the gateway layer. The gateway is the right place to enforce routing because it has identity and policy context the application does not.

ai-securitypolicy-enforcementarchitecturellmllm-securitydevsecops
Read post →

Mapping the OWASP Top 10 for Agentic Applications 2026 to Control Points a Policy Gateway Enforces

OWASP GenAI published the Top 10 for Agentic Applications 2026 as a separate framework from the LLM Top 10. The framework adds the "agentic skills" intermediate behavior layer as a new vulnerable component and reorders the threat list around tool invocation, plan corruption, and identity propagation. This article maps each of the 10 categories to specific control points that a policy gateway at the AI request boundary actually enforces, with example policy rules and the audit fields each control writes.

ai-securityagentic-aillm-securitypolicy-enforcementarchitectureaudit
Read post →

NIST AI agent identity Pillars 2 and 3: authorization and audit at the request layer

NIST has framed AI agent identity and authorization around three pillars. Pillar 1 is identification at the request boundary. Pillar 2 is authorization tied to the resolved identity. Pillar 3 is audit and accountability across the request lifecycle. The public comment window on the NIST draft closed April 2, 2026. This piece walks through what Pillars 2 and 3 actually require at the architecture layer and where most enterprise AI deployments fall short.

nistai-agent-identityauthorizationauditzero-trust-ai
Read post →

22-Second Breach Windows: Why AI Enforcement Has to Be Inline

Google Mandiant M-Trends 2026 found median attacker handoff time collapsed from over 8 hours in 2022 to 22 seconds in 2025. Detect-and-respond runs after damage has occurred. For AI traffic specifically, an exfiltrated prompt is one-shot. Inline enforcement at under 50ms overhead is the architectural answer.

inline-enforcementai-securitymachine-speedmandiantprevention
Read post →

LLM Gateway Benchmarks: What to Measure, How to Measure It, and Where Most Vendor Numbers Mislead

Most vendor LLM gateway benchmarks publish a median latency figure under synthetic load and stop there. The numbers a platform team actually needs are the policy-decision tail latency, the policy-evaluation throughput under contention, the cold-cache impact, and the audit-write durability cost. This walkthrough shows the four measurement axes, the workload profiles that produce comparable numbers, and the failure modes that surface only at production traffic shape.

ai-gatewaybenchmarkslatencyperformancepolicy-enforcementengineering
Read post →

AI Rate Limiting by Identity: Why Per-Key Quotas Miss the Actual Risk

A per-API-key rate limit lets one runaway service consume the whole quota for its tenant. An identity-bound rate limit accumulates against the verified caller and produces a defensible refusal at the request layer. This walkthrough covers the four identity dimensions a useful rate limit accumulates against, the algorithms that hold under burst traffic, and the audit-record fields that make a refusal admissible.

rate-limitingai-gatewayidentityengineeringcost-governance
Read post →

AI Gateway Multi-Cloud: The Single Control Plane Across OpenAI, Anthropic, Bedrock, and Vertex

Enterprise AI traffic now spans OpenAI direct, Anthropic direct, AWS Bedrock, Azure OpenAI, and Google Vertex in the same week, often in the same application. Each provider has its own auth, its own request shape, its own error semantics, and its own audit emission. A multi-cloud AI gateway is the single control plane that normalizes identity, classification, policy, and audit across all of them. This walkthrough covers the normalization layer, the per-provider adapters, and the audit record that survives the regulator regardless of which provider the request hit.

ai-gatewaymulti-cloudopenaianthropicbedrockvertex
Read post →

Fail-Closed vs Fail-Open AI Gateway: The Decision That Survives the Post-Incident Review

A gateway whose policy plane is unreachable has to decide whether to forward the request anyway or refuse it. The decision is architectural and the cost of getting it wrong shows up as either a regulatory finding or a production outage. Fail-closed for policy decisions and fail-open for cached bundles is the pairing that survives the post-incident review. This walkthrough covers the failure modes, the per-route defaults, and the operational runbook for the brief window where the policy plane is unavailable.

fail-closedai-gatewayavailabilitypolicy-enforcementarchitecture
Read post →