← All posts

Platform & Architecture

145 posts on platform & architecture.

AI Gateway Per-Tenant Rate Limiting: The Buckets That Actually Contain a Runaway Workload

A rate limit on the AI gateway is not a single ceiling. Enterprise deployments run several rate-limit buckets in parallel: per model, per tenant, per user, per tool, per purpose. The buckets interact, and the interaction is where runaway workloads hurt most. This piece walks through the bucket design that contains a runaway agent loop, protects the model provider's shared quota, and produces the audit records the operator needs to explain a rate-limit event.

ai-gatewayrate-limitingmulti-tenantai-securityquota-managementai-cost
Read post →

Anthropic MCP Server Security: The Enterprise Controls That Sit Around Claude's Tool Layer

Anthropic's Model Context Protocol implementation lets Claude call tool servers with a standard schema. The enterprise question is what security controls have to sit around the MCP server when the LLM behind the protocol is Claude. This piece walks through the controls: transport authentication to the MCP server, token exchange between Claude's session and the tool call, response inspection before the tool result reaches the model, and audit records tied to the human user who authorized the session.

anthropicmcpclaudeai-securityai-agentai-gateway
Read post →

LLM Vector Store Access Control: The Filters That Have to Run on Every RAG Query

The vector store holds embeddings the enterprise's users, tenants, and documents contributed. Every retrieval-augmented generation query has to run access-control filters against the store before the retrieval reaches the LLM context. This piece walks through the filter design that survives multi-tenant SaaS, cross-department access, and time-bounded document lifecycle: per-vector metadata, query-time filter injection, retrieval-response inspection, and the audit records that prove the filter held on every query.

ragvector-storeaccess-controlai-securityai-gatewayllm-dlp
Read post →

OpenAI Usage Tier Controls: How an Enterprise Enforces Per-Team Budgets on the Same API Key

OpenAI's account usage tiers describe the account-level rate ceiling. The tier is a single number the account holds, and the tier does not describe the enterprise's per-team, per-application, or per-user budgets. An enterprise that runs OpenAI at scale has to enforce a set of budget controls that sit above the account tier. This piece walks through the pattern set: per-team token budgets, per-application spend caps, per-user rate ceilings, and the audit records that tie every request back to the team accountable for it.

openaiusage-tierai-costai-gatewaybudget-controlai-security
Read post →

AI Cost Attribution Per Team: The Record Design That Turns AI Spend Into a Chargeback Line Item

Finance teams asking for AI cost attribution per team run into the same problem every time: the model provider's invoice shows the aggregate account spend, not the per-team consumption. Attribution has to happen at the gateway, where the enterprise can tag each request with the team, application, and user identity. This piece walks through the record design that produces attributable AI spend, the tagging pattern that survives multi-model deployments, and the reports the finance team accepts as chargeback evidence.

ai-costchargebackai-gatewayfinanceattributionai-governance
Read post →

A JSON Schema for AI Audit Logs: The Fields a Regulator, an Auditor, and a SIEM All Need in the Same Record

AI audit logs need one schema that satisfies a regulator during an Article 12 audit, an internal auditor during an ISO 42001 certification, and a SIEM during an active incident. Most deployments produce three log formats and reconcile them after the fact. This piece walks through a single JSON schema with the required and optional fields, the identity representation, the policy-state representation, and the storage-layer contract that makes the schema durable.

ai-audit-logsjson-schemasiemeu-ai-actiso-42001ai-engineering
Read post →

MCP Server Authentication: Identity Binding at the Model Context Protocol Boundary

The Model Context Protocol lets an LLM client discover and call tools exposed by an MCP server. Authentication at the MCP boundary determines which identity issues the tool calls, which policy applies, and which record ends up in the audit log. This piece walks through the OAuth 2.1 authorization flow the MCP spec adopted, the pitfalls in shared-secret patterns, and the inspection-layer architecture that binds every MCP tool call to a verified identity.

mcpauthenticationoauthai-agent-securityai-engineering
Read post →

AI Gateway Cache Invalidation: When a Cached Prompt Response Becomes a Data Leak

AI gateways cache prompt responses to cut cost and latency. The cache lookup uses a hash of the prompt as the key, which means two callers with different authorization scopes can hit the same cache entry. This piece walks through the failure mode, the identity-scoped cache-key patterns that avoid it, and the inspection-layer architecture that makes cache lookup safe.

ai-gatewaycachingai-engineeringdata-leakai-security
Read post →

AI Agent OAuth Consent: The Permission Screen Users Never Read and the Blast Radius It Grants

An AI agent that authenticates to a SaaS application via OAuth requests a consent scope from the user. The scope grants the agent standing authorization to call APIs on the user behalf. Users grant scopes they do not read, and the standing authorization outlasts the interaction that produced it. This piece walks through the OAuth consent mechanism, the blast radius it creates, and the inspection-layer controls that constrain the scope after grant.

oauthai-agent-securityconsentnon-human-identityai-engineering
Read post →

LLM Response Schema Validation: When JSON Mode Is Not Enough

JSON mode and structured output constrain the LLM to produce valid JSON, but the JSON can still contain values that violate business policy, personal data that violates data-classification policy, or tool-call arguments that violate authorization scope. This piece walks through what JSON mode covers, the semantic-validation gap it leaves, and the inspection-layer architecture that runs schema validation and semantic validation on the same response path.

llm-engineeringjson-schemaai-engineeringstructured-outputai-security
Read post →

AI Red Teaming Workflow: The Test-Fix-Prove Loop for Enterprise AI Deployments

AI red teaming discovers vulnerabilities in prompt handling, tool-call authorization, and response classification. The finding is one artifact. The fix is another. The evidence that the fix works is a third. This piece walks through a red-teaming workflow that produces all three artifacts inside the enterprise control boundary, and the inspection-layer architecture that turns findings into policy the enforcement layer executes.

red-teamingai-securitypenetration-testingai-engineeringai-governance
Read post →

AI Audit Log Immutability: Object Lock, WORM Storage, and the Storage-Layer Contract a Regulator Accepts

The reconstruction test a regulator applies during an AI audit assumes the log record has not been rewritten. The assumption fails when the log lives in a storage layer that permits modification by the same operator who runs the AI application. This piece walks through the immutability contract at the storage layer, S3 Object Lock and Azure Blob immutability policies as implementations, and the audit-record shape that verifies immutability by construction.

ai-audit-logsimmutabilityobject-lockcomplianceai-engineering
Read post →