← All posts

Platform & Architecture

338 posts on platform & architecture.

AI Gateway Blue-Green Deployment: How to Ship a Gateway Version Without Cutting Traffic

A blue-green deployment runs two full gateway environments in parallel, with traffic flipped at a load balancer from the current (blue) environment to the new (green) environment after the green environment has been verified. The pattern works for AI gateways with two differences from a standard API gateway: the policy and routing state has to be consistent across the cutover, and the audit log chain has to remain unbroken. This article walks the blue-green pattern at the AI gateway layer, the state-consistency requirements, the verification gates, and the fallback path.

ai-gatewayblue-green-deploymenthigh-availabilitydeploymentaudit-loggingdevops
Read post →

AI Gateway Multi-Region Failover: The Architecture That Survives a Regional LLM Outage

A regional LLM provider outage takes down every AI feature that depends on that region. The mitigation is a gateway architecture that routes around the failure within seconds. Multi-region failover at the AI gateway has three components: a gateway deployment in at least two regions, a policy and routing layer that supports per-region destinations, and a health-aware traffic director that promotes a region to active when the primary fails. This article walks the architecture, the failure modes that recur, the audit-log implications across regions, and the operational drill.

ai-gatewaymulti-regionfailoverhigh-availabilitydisaster-recoverydevops
Read post →

AI Incident Response Runbook: The Steps a SOC Actually Runs When an LLM Interaction Goes Wrong

AI incidents share their reporting timelines with the SEC 8-K 4-business-day rule, the EU AI Act Article 26.4 immediate reporting for serious incidents, and the HIPAA 60-day breach notification. The technical response has to run in parallel with the reporting clock. A working runbook covers the six-phase incident lifecycle: detect, triage, contain, investigate, report, and remediate. Each phase depends on the audit records the AI gateway produces. This piece walks through the runbook, the specific queries the SOC runs against the audit log at each phase, the roles that own each step, and the artifact pack the CISO hands to regulators when the reporting timers expire.

incident-responsesocai-securityrunbookir-playbookaudit-logs
Read post →

AI Gateway Latency Benchmark 2026: How to Measure the p95 Overhead of Every Enforcement Step

AI gateway latency budgets get argued about at architecture review and then never measured under production load. The gateway sits inline. Every millisecond of overhead compounds across every LLM call. This piece walks through a benchmark methodology that separates the connection, identity resolution, classification, policy evaluation, and audit-record steps, so an architecture team can defend a p95 budget against the 500 ms to 5 second baseline of LLM inference. The measurement pattern applies to any inline enforcement layer, not only DeepInspect.

ai-gatewaylatencybenchmarkperformanceinline-enforcementai-security
Read post →

AI Agent OAuth Scopes: Designing Per-Tool Authorization That Survives an Audit

OAuth scopes were designed for human users clicking through consent screens. AI agents call the same OAuth endpoints, but the agent's authorization is not the human's consent. The agent operates under an identity that persists across sessions, negotiates scopes at runtime, and combines multiple tool authorizations into a single execution graph. This piece walks through the OAuth scope design patterns that hold up when the caller is an agent, the audit records that prove the scope was enforced, and the failure modes that appear when scope design treats agents like humans.

ai-agentoauthauthorizationagentic-aiai-securityaudit-logs
Read post →

AI Gateway Per-Tenant Rate Limiting: The Buckets That Actually Contain a Runaway Workload

A rate limit on the AI gateway is not a single ceiling. Enterprise deployments run several rate-limit buckets in parallel: per model, per tenant, per user, per tool, per purpose. The buckets interact, and the interaction is where runaway workloads hurt most. This piece walks through the bucket design that contains a runaway agent loop, protects the model provider's shared quota, and produces the audit records the operator needs to explain a rate-limit event.

ai-gatewayrate-limitingmulti-tenantai-securityquota-managementai-cost
Read post →

LLM Vector Store Access Control: The Filters That Have to Run on Every RAG Query

The vector store holds embeddings the enterprise's users, tenants, and documents contributed. Every retrieval-augmented generation query has to run access-control filters against the store before the retrieval reaches the LLM context. This piece walks through the filter design that survives multi-tenant SaaS, cross-department access, and time-bounded document lifecycle: per-vector metadata, query-time filter injection, retrieval-response inspection, and the audit records that prove the filter held on every query.

ragvector-storeaccess-controlai-securityai-gatewayllm-dlp
Read post →

AI Cost Attribution Per Team: The Record Design That Turns AI Spend Into a Chargeback Line Item

Finance teams asking for AI cost attribution per team run into the same problem every time: the model provider's invoice shows the aggregate account spend, not the per-team consumption. Attribution has to happen at the gateway, where the enterprise can tag each request with the team, application, and user identity. This piece walks through the record design that produces attributable AI spend, the tagging pattern that survives multi-model deployments, and the reports the finance team accepts as chargeback evidence.

ai-costchargebackai-gatewayfinanceattributionai-governance
Read post →

A JSON Schema for AI Audit Logs: The Fields a Regulator, an Auditor, and a SIEM All Need in the Same Record

AI audit logs need one schema that satisfies a regulator during an Article 12 audit, an internal auditor during an ISO 42001 certification, and a SIEM during an active incident. Most deployments produce three log formats and reconcile them after the fact. This piece walks through a single JSON schema with the required and optional fields, the identity representation, the policy-state representation, and the storage-layer contract that makes the schema durable.

ai-audit-logsjson-schemasiemeu-ai-actiso-42001ai-engineering
Read post →

AI Gateway Cache Invalidation: When a Cached Prompt Response Becomes a Data Leak

AI gateways cache prompt responses to cut cost and latency. The cache lookup uses a hash of the prompt as the key, which means two callers with different authorization scopes can hit the same cache entry. This piece walks through the failure mode, the identity-scoped cache-key patterns that avoid it, and the inspection-layer architecture that makes cache lookup safe.

ai-gatewaycachingai-engineeringdata-leakai-security
Read post →

AI Agent OAuth Consent: The Permission Screen Users Never Read and the Blast Radius It Grants

An AI agent that authenticates to a SaaS application via OAuth requests a consent scope from the user. The scope grants the agent standing authorization to call APIs on the user behalf. Users grant scopes they do not read, and the standing authorization outlasts the interaction that produced it. This piece walks through the OAuth consent mechanism, the blast radius it creates, and the inspection-layer controls that constrain the scope after grant.

oauthai-agent-securityconsentnon-human-identityai-engineering
Read post →

AI Red Teaming Workflow: The Test-Fix-Prove Loop for Enterprise AI Deployments

AI red teaming discovers vulnerabilities in prompt handling, tool-call authorization, and response classification. The finding is one artifact. The fix is another. The evidence that the fix works is a third. This piece walks through a red-teaming workflow that produces all three artifacts inside the enterprise control boundary, and the inspection-layer architecture that turns findings into policy the enforcement layer executes.

red-teamingai-securitypenetration-testingai-engineeringai-governance
Read post →