← All posts

Platform & Architecture

338 posts on platform & architecture.

OpenAI Usage Tier Controls: How an Enterprise Enforces Per-Team Budgets on the Same API Key

OpenAI's account usage tiers describe the account-level rate ceiling. The tier is a single number the account holds, and the tier does not describe the enterprise's per-team, per-application, or per-user budgets. An enterprise that runs OpenAI at scale has to enforce a set of budget controls that sit above the account tier. This piece walks through the pattern set: per-team token budgets, per-application spend caps, per-user rate ceilings, and the audit records that tie every request back to the team accountable for it.

openaiusage-tierai-costai-gatewaybudget-controlai-security
Read post →

AI Tenant Isolation: How Multi-Tenant SaaS Enforces Per-Customer Boundaries on LLM Traffic

Multi-tenant SaaS applications that add LLM features carry a new isolation obligation on top of the database and storage isolation the platform already enforces. Prompts flow through the LLM provider carrying tenant-specific data. Retrieval-augmented generation queries the vector store where tenant data lives. Agent tools call downstream systems that hold tenant data. Each of these paths introduces a way for tenant A's data to reach tenant B's context without a database join between them. This piece walks through the four isolation domains (prompt, retrieval, tool call, response), the enforcement patterns at the AI gateway, and the audit records that demonstrate the isolation held across the audit period.

multi-tenantai-securitytenant-isolationsaasragai-gateway
Read post →

AI Agent Identity: NIST Pillar 1 in Production Deployments

NIST Pillar 1 names verified agent identity as the foundation of the AI agent identity and authorization framework. Per-agent identifiers, delegated authority from the authorizing user, and structured propagation to the model API call are the production requirements. Static service credentials fail the test.

agentic-aiidentity-and-authorizationnist-ai-rmfai-securityarchitecturezero-trust
Read post →

Zero Trust AI: Per-Request Evaluation at the Model Boundary

Zero trust applied to AI means evaluating every model request against verified identity, current policy, and prompt-level classification. The architectural pattern is an enforcement proxy at the HTTP AI request boundary. The post-authentication gap is the most common failure mode in current deployments.

zero-trustai-securityidentity-and-authorizationpolicy-enforcementinline-enforcementarchitecture
Read post →

Model Guardrails Are Probabilistic, Not Enforceable Controls

Model guardrails are trained behaviors inside the inference process. They degrade under fine-tuning, adversarial prompting, and role-play framing. External enforcement at the AI request boundary produces deterministic controls and identity-bound audit records that guardrails alone cannot.

ai-securityllm-securityprompt-injectionpolicy-enforcementarchitectureinline-enforcement
Read post →

22-Second Breach Windows: Why AI Enforcement Must Be Inline

Mandiant M-Trends 2026 measured median attack handoff at 22 seconds. At that tempo, log-and-alert fails as a control. Inline enforcement at the AI request boundary makes the policy decision before the request reaches the model. Under 50 ms enforcement overhead is invisible against 500 ms to 5 second model inference.

ai-securityinline-enforcementpolicy-enforcementcybersecurityarchitecturezero-trust
Read post →

Identity-Aware AI Gateway Architecture: How Inline Enforcement Binds Decisions to Users and Agents

An identity-aware AI gateway sits at the AI request boundary, attaches verified identity context to every model API call, evaluates per-route and per-role policies, and commits a per-decision audit record before the model response returns to the calling application. The architecture closes the post-authentication gap that most enterprise AI deployments have inherited from the credential-pooling pattern used by SDKs and proxy frameworks. This piece walks through the architectural building blocks, the call path, the audit primitives, and where the identity-aware gateway sits relative to existing IAM, API gateway, and DLP infrastructure.

ai-gatewayidentity-awareai-architectureenforcementauditzero-trust
Read post →

Tamper-Evident Audit Logs for AI: What Cryptographic Integrity Brings to Compliance Records

Tamper-evident audit logs make any post-hoc modification of a record detectable through cryptographic integrity. For AI compliance records, the property closes the self-attestation gap that application-controlled logs cannot. The technique combines per-record signing, hash chaining, and external anchoring. EU AI Act Article 12, Fannie Mae LL-2026-04, HIPAA, DORA, and NIST AI RMF all expect records that an auditor can rely on as evidence. Application logs that the application can modify do not meet that standard. This piece walks through the cryptographic mechanisms, the operational characteristics, and the architectural placement.

audit-logstamper-evidentcompliancecryptographyai-auditintegrity
Read post →

Signed Audit Logs for AI Requests: Per-Decision Signing and What Regulators Will Accept

A signed audit log binds a cryptographic signature to each record at the moment the record is committed. For AI requests, the signature ties the record to the inspection layer that produced it and lets a verifier confirm authenticity without trusting the storage layer. The technique is the cryptographic foundation under tamper-evident audit trails the EU AI Act Article 12, Fannie Mae LL-2026-04, HIPAA, DORA, and NIST AI agent identity framework all expect. This piece walks through the signing schemes, the key management, and the verification flow that auditors and regulators will accept.

signed-auditaudit-logscryptographyai-complianceintegritypki
Read post →

Per-Route AI Policies: How To Implement Endpoint-Specific Enforcement in Front of LLM APIs

Per-route AI policies attach a different enforcement rule to each LLM endpoint behind the inspection layer. A request to the customer-support route runs under one policy. A request to the developer-tooling route runs under another. The implementation lets a single inspection layer serve every team without the lowest common denominator policy that an organization-wide rule produces. This piece walks through the data model, the matching algorithm, the policy state that has to be present at decision time, and the operational characteristics that hold up at production scale across OpenAI, Anthropic, Azure OpenAI, and Bedrock endpoints.

ai-gatewayper-route-policiesinline-enforcementai-architectureidentity-awareaudit
Read post →

Stateless vs Stateful AI Proxy: Which Architecture Holds Up Under Production Load and Audit

A stateless AI proxy makes the policy decision on the contents of the current request and the per-decision audit record alone. A stateful AI proxy carries session memory, caches conversation history, or stores prompts across requests in its own storage. The choice has direct consequences for horizontal scaling, blast radius under compromise, the EU AI Act Article 12 record-keeping obligation, and the DORA third-party risk profile of the inspection layer. This piece walks through the architectural distinction, what each option requires from the deployment, and where most production teams settle once the trade-offs are visible.

ai-proxyai-gatewaystatelessai-architectureinline-enforcementaudit
Read post →

OpenAI API Gateway Patterns: How To Front api.openai.com with Inline Enforcement

An OpenAI API gateway sits between calling applications and api.openai.com, attaches identity context, evaluates per-request policy, and commits a per-decision audit record before the request reaches the model. The pattern replaces the direct calling convention that uses an organization-bound API key with an inspection layer that the application addresses instead. This piece walks through the request rewriting pattern, the SSE and streaming response handling, the function-calling and tool-use evaluation, and the audit record format that satisfies EU AI Act Article 12 and the deployer obligations under Article 26.

openaiai-gatewayapi-gatewayinline-enforcementstreaming-responsesaudit
Read post →