← All posts

Platform & Architecture

338 posts on platform & architecture.

LLM Streaming Trailer Allowlist for AI Gateways

HTTP trailers arrive after streamed content, which makes them useful for final digests, signatures, delivery metrics, and post-processing status. Their timing also limits what they can safely control. An AI gateway should keep trailers separate from initial headers, accept only fields whose definitions permit trailer use, reject security-sensitive names, and preserve approved metadata only across protocol paths that support it.

ai-securityllm-securityinline-enforcementpolicy-enforcementarchitecture
Read post →

Hop-by-Hop Header Stripping in AI Gateways

An AI gateway terminates one HTTP connection and creates another, so connection-specific fields from the inbound hop cannot pass through unchanged. RFC 9110 requires an intermediary to parse Connection, remove every field it names, and remove or replace Connection itself. A safe gateway also handles known connection-specific fields, protects authorization metadata from hostile nominations, and serializes a clean outbound LLM request.

ai-securityllm-securityzero-trustpolicy-enforcementarchitecture
Read post →

Trusted Forwarded Headers for AI Gateways

Forwarded and X-Forwarded headers can carry the client address, original host, and original scheme across an AI gateway proxy chain. Every client can also send those fields. A safe deployment removes untrusted values at the first controlled hop, rebuilds forwarding metadata from the connection it received, and gives each downstream service an explicit trust boundary. This guide defines that boundary, the parsing rules, and the acceptance tests for authenticated HTTP LLM routes.

ai-securityllm-securityzero-trustidentity-and-authorizationarchitecture
Read post →

LLM API Authority Validation: Pin Every Request to Its Intended Origin

LLM API authority validation binds each request to the intended scheme, host, port, TLS identity, route, tenant, and approved model destination. This guide applies RFC 9110 origin, Host, :authority, misdirection, and 421 semantics to gateways and provider adapters, then defines normalization rules, deny behavior, DNS and connection-reuse tests, and evidence for every routed decision.

ai-securityllm-securityzero-trustpolicy-enforcementarchitecturecloud-security
Read post →

LLM API Problem Details: A Stable Error Contract for Model Routes

RFC 9457 gives LLM APIs a common problem-details model for machine-readable HTTP errors. This guide maps type, status, title, detail, instance, and extensions to model-route failures; defines disclosure and versioning rules; and supplies an acceptance fixture for validation, policy denial, rate limits, payload limits, provider failures, streaming boundaries, and intermediary behavior.

ai-securityllm-securityarchitecturepolicy-enforcementdevsecopsaudit
Read post →

LLM API Authentication Challenges: A Correct 401 and 403 Contract

LLM API authentication challenges need a precise contract across 401, WWW-Authenticate, credential replacement, 403, proxy challenges, and machine-readable errors. This guide separates authentication from request-time authority, defines safe forwarding and logging behavior for an LLM gateway, and supplies negative tests for expired tokens, wrong audiences, multiple schemes, insufficient permissions, and stale credentials.

ai-securityllm-securityidentity-and-authorizationzero-trustpolicy-enforcementaudit
Read post →

LLM API Retry-After Policy: Safe Delays Without Duplicate Work

An LLM API Retry-After policy must combine the server's delay signal with method semantics, an idempotency contract, a bounded retry budget, jitter, and a final failure state. This guide defines handling for 503, temporary 413, and other documented responses, then turns the policy into a production acceptance fixture with evidence for every attempt and downstream effect.

ai-securityllm-securitypolicy-enforcementarchitecturedevsecopsaudit
Read post →

LLM API Payload Size Policy: Bound Requests Before Inference

An LLM API payload size policy needs separate limits for HTTP content, decoded fields, files, images, messages, tokens, and downstream expansion. This guide defines a deterministic rejection contract, a test matrix for every boundary, and the evidence an engineering team should retain. It also separates gateway controls on routed HTTP traffic from limits owned by model providers, storage services, and local runtimes.

ai-securityllm-securitypolicy-enforcementarchitecturedevsecopsaudit
Read post →

LLM API Header Size Limits at the Authenticated HTTP Boundary

LLM API header size limits need one aligned policy across the edge, gateway, application, and provider adapter. This guide applies RFC 9110 and RFC 9112 to authenticated user and agent traffic, covering per-field and aggregate limits, identity and tracing metadata, 4xx handling, request smuggling risk, test cases, and audit evidence. The target is deterministic rejection before prompt or model processing.

ai-securityllm-securitycybersecurityidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

Expect: 100-continue as an Authentication Gate for LLM Requests

Expect: 100-continue lets an HTTP client present an LLM request line and headers before transmitting a large body. This guide explains how an authenticated gateway can reject invalid credentials, methods, routes, media types, or declared sizes at that header stage; when to send 100, 401, 405, 413, 415, or 417; and how to test clients and proxies without treating the mechanism as a complete authorization control.

ai-securityllm-securitycybersecurityidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

LLM Endpoint HTTP Method Allowlists for Authenticated Requests

An LLM endpoint HTTP method allowlist binds each authenticated route to the small set of HTTP actions it actually implements. This guide maps RFC 9110 method semantics to inference, files, batches, and administration; explains 405 and Allow handling; and gives a test and audit pattern for gateways. The objective is to reject method drift before a request reaches model infrastructure.

ai-securityllm-securitycybersecurityidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

LLM Response Accept Negotiation for Authenticated API Clients

LLM response Accept negotiation turns response format into an explicit contract between an authenticated user or agent and an LLM endpoint. This guide covers media ranges, quality weights, streaming types, 406 handling, gateway validation, downgrade prevention, and audit evidence. It keeps JSON, event-stream, and other approved representations tied to route policy instead of client-library guesswork.

ai-securityllm-securitycybersecurityidentity-and-authorizationpolicy-enforcementarchitecture
Read post →