← All posts

Platform & Architecture

145 posts on platform & architecture.

MCP Server Authentication for Enterprise Deployments: Identity, Authorization, and the Boundary Question

Model Context Protocol (MCP) servers have moved from developer-tool integrations to production agent backends inside enterprises. The authentication and authorization model for MCP traffic differs from REST-API authentication in two specific ways that matter at enterprise scale: the principal acting on the MCP server can be an agent on behalf of a human, and the tool invocations have to carry the propagation chain. This article walks through the MCP authentication model, the transport-boundary distinction, and the gateway-layer controls that produce a usable audit trail.

ai-securitymcpauthenticationagent-securityarchitecturepolicy-enforcement
Read post →

AI Gateway Canary Deployment: Patterns for Rolling Policy and Model Changes Safely

Canary deployment for an AI gateway covers two distinct change types: model routing changes (a new provider, a new model version, a new model entirely) and policy changes (a new redaction rule, a new tool allowlist, a new rate-limit threshold). Each change type has different risk characteristics and different rollback triggers. The canary pattern at the gateway differs from a classic application canary because the unit of traffic is identity-bound and the failure modes include silent drift in model behavior. This article walks through the canary architecture for an AI gateway, the metrics that drive the rollout, and the rollback conditions that have to be wired in before the canary starts.

ai-gatewaycanary-deploymentdeployment-patternsrelease-managementai-policy-enforcementplatform-engineering
Read post →

AI Gateway Rate Limiting by Identity: Why Per-Key Limits Fail in Production

AI gateway rate limiting that uses the API key as the limit boundary fails in three production patterns: shared service accounts, agent fan-out, and cost runaway from a single high-volume identity. The fix is to limit per verified identity, where identity is the authenticated principal extracted from the request context, not the API key in the header. This article walks the failure modes, the architecture that fixes them, the data model the gateway needs, and the operational tradeoffs of identity-bound limits versus simpler per-key approaches.

ai-gatewayrate-limitingidentityengineeringpolicy-enforcementagentic-ai
Read post →

AI Gateway Fail-Open vs Fail-Closed: The Decision That Shapes Your Audit Trail

An AI gateway that sits inline between authenticated callers and the LLMs they use has to answer a structural question. When the gateway cannot reach the policy decision (the policy engine is down, the identity service is unreachable, a configuration cannot be loaded), does the request go through (fail-open) or get refused (fail-closed)? The answer shapes the audit trail, the regulatory posture, and the production behavior under degraded conditions. This article walks the tradeoffs, the cases where each mode is appropriate, the data-driven defaults, and the operational patterns that hold up under audit.

ai-gatewayfail-closedreliabilityengineeringpolicy-enforcementaudit-logging
Read post →

AI Gateway Rollback Strategy: How to Revert a Policy or Model Change Without Breaking the Audit Trail

A bad policy change or a broken model upgrade at the AI gateway has to be reverted fast. The rollback is the high-availability move that prevents a small problem from becoming a service-wide outage. The rollback also has to preserve the audit trail, because the regulatory record of "what policy was in effect when" survives the rollback. This article walks the rollback patterns that work at the gateway layer, the failure modes that catch teams off guard, the integrity controls that keep the audit record consistent across the revert, and the operational drill that proves the rollback works before it has to.

ai-gatewayrollbackdeploymentpolicy-versioningaudit-logginghigh-availability
Read post →

AI Gateway Blue-Green Deployment: How to Ship a Gateway Version Without Cutting Traffic

A blue-green deployment runs two full gateway environments in parallel, with traffic flipped at a load balancer from the current (blue) environment to the new (green) environment after the green environment has been verified. The pattern works for AI gateways with two differences from a standard API gateway: the policy and routing state has to be consistent across the cutover, and the audit log chain has to remain unbroken. This article walks the blue-green pattern at the AI gateway layer, the state-consistency requirements, the verification gates, and the fallback path.

ai-gatewayblue-green-deploymenthigh-availabilitydeploymentaudit-loggingdevops
Read post →

AI Gateway Multi-Region Failover: The Architecture That Survives a Regional LLM Outage

A regional LLM provider outage takes down every AI feature that depends on that region. The mitigation is a gateway architecture that routes around the failure within seconds. Multi-region failover at the AI gateway has three components: a gateway deployment in at least two regions, a policy and routing layer that supports per-region destinations, and a health-aware traffic director that promotes a region to active when the primary fails. This article walks the architecture, the failure modes that recur, the audit-log implications across regions, and the operational drill.

ai-gatewaymulti-regionfailoverhigh-availabilitydisaster-recoverydevops
Read post →

AI Incident Response Runbook: The Steps a SOC Actually Runs When an LLM Interaction Goes Wrong

AI incidents share their reporting timelines with the SEC 8-K 4-business-day rule, the EU AI Act Article 26.4 immediate reporting for serious incidents, and the HIPAA 60-day breach notification. The technical response has to run in parallel with the reporting clock. A working runbook covers the six-phase incident lifecycle: detect, triage, contain, investigate, report, and remediate. Each phase depends on the audit records the AI gateway produces. This piece walks through the runbook, the specific queries the SOC runs against the audit log at each phase, the roles that own each step, and the artifact pack the CISO hands to regulators when the reporting timers expire.

incident-responsesocai-securityrunbookir-playbookaudit-logs
Read post →

AI Tenant Isolation: How Multi-Tenant SaaS Enforces Per-Customer Boundaries on LLM Traffic

Multi-tenant SaaS applications that add LLM features carry a new isolation obligation on top of the database and storage isolation the platform already enforces. Prompts flow through the LLM provider carrying tenant-specific data. Retrieval-augmented generation queries the vector store where tenant data lives. Agent tools call downstream systems that hold tenant data. Each of these paths introduces a way for tenant A's data to reach tenant B's context without a database join between them. This piece walks through the four isolation domains (prompt, retrieval, tool call, response), the enforcement patterns at the AI gateway, and the audit records that demonstrate the isolation held across the audit period.

multi-tenantai-securitytenant-isolationsaasragai-gateway
Read post →

AI Gateway Latency Benchmark 2026: How to Measure the p95 Overhead of Every Enforcement Step

AI gateway latency budgets get argued about at architecture review and then never measured under production load. The gateway sits inline. Every millisecond of overhead compounds across every LLM call. This piece walks through a benchmark methodology that separates the connection, identity resolution, classification, policy evaluation, and audit-record steps, so an architecture team can defend a p95 budget against the 500 ms to 5 second baseline of LLM inference. The measurement pattern applies to any inline enforcement layer, not only DeepInspect.

ai-gatewaylatencybenchmarkperformanceinline-enforcementai-security
Read post →

AI Agent OAuth Scopes: Designing Per-Tool Authorization That Survives an Audit

OAuth scopes were designed for human users clicking through consent screens. AI agents call the same OAuth endpoints, but the agent's authorization is not the human's consent. The agent operates under an identity that persists across sessions, negotiates scopes at runtime, and combines multiple tool authorizations into a single execution graph. This piece walks through the OAuth scope design patterns that hold up when the caller is an agent, the audit records that prove the scope was enforced, and the failure modes that appear when scope design treats agents like humans.

ai-agentoauthauthorizationagentic-aiai-securityaudit-logs
Read post →

MCP Server Authorization Patterns: Enforcing Who Can Call Which Tool Through the Model Context Protocol

The Model Context Protocol gave agents a common way to talk to tool servers. It did not give the enterprise a common way to authorize which agent can call which tool with which arguments. Authorization at the MCP server sits at three layers: the transport layer, the server layer, and the tool-invocation layer. Each layer answers a different question and produces a different audit record. This piece walks through the authorization patterns that survive an enterprise deployment across multiple MCP servers and multiple agent identities.

mcpmodel-context-protocolauthorizationai-agentai-securityai-gateway
Read post →