LLM API Problem Details: A Stable Error Contract for Model Routes
RFC 9457 gives LLM APIs a common problem-details model for machine-readable HTTP errors. This guide maps type, status, title, detail, instance, and extensions to model-route failures; defines disclosure and versioning rules; and supplies an acceptance fixture for validation, policy denial, rate limits, payload limits, provider failures, streaming boundaries, and intermediary behavior.

An LLM API error should give generic HTTP software a correct status and give the application a stable machine identifier for the specific problem. RFC 9457 supplies that division of responsibility. The transport status carries broad semantics. The problem body names the type, states the status, explains this occurrence, and can add structured extensions for corrective action.
The contract should remain readable during an incident. Picture a terminal showing a 413 response with application/problem+json, a stable type URI, an opaque instance ID, and limit, observed, and unit fields. An engineer can act without scraping a sentence or opening a provider dashboard.
TL;DR
- Use the actual HTTP status for generic behavior and a stable problem type URI for the API-specific condition.
- Keep title stable and make detail occurrence-specific. Treat instance as an opaque occurrence identifier.
- Add typed extensions for machine action, then require clients to ignore extensions they do not recognize.
- Vet every field for disclosure risk and keep status consistent across headers and body. Test intermediary plus streaming behavior.
RFC 9457 defines a small canonical model
RFC 9457 Section 3 defines the canonical problem details model as a JSON object and assigns application/problem+json to its JSON serialization. The standard members are type, status, title, detail, and instance. An API can also define extensions tied to a problem type.
The model complements HTTP semantics. RFC 9110 Section 15 defines status codes used by generic clients, proxies, caches, and load balancers. Problem details carry the API-specific explanation a status code alone cannot express. A policy denial and tenant restriction might share a broad 4xx class while a payload limit can require a different client action.
Start with an error inventory for each LLM route. Include invalid JSON and schema validation, unsupported media and model types, authentication and authorization, payload and token limits, rate limits and temporary unavailability, provider failures and timeouts, and policy decisions such as disallowed data or destination. Assign each condition an HTTP status and a stable problem type.
The OWASP API Security Top 10 for LLM endpoints provides the route-level failure categories around authentication, authorization, resource use, configuration, and upstream data.
Type is the machine identity of the problem
RFC 9457 Section 3.1.1 makes type the primary identifier for the problem. It is a URI reference, and an absent value means about:blank. Clients should compare the resolved URI rather than the title or detail text.
Use absolute HTTPS URIs under a domain the API owner controls. Publish human-readable documentation at each URI when practical. The page should explain the meaning, expected HTTP status, required extensions, corrective action, and compatibility policy. Treat changing the type URI as a contract change because clients may branch on it.
Reserve about:blank for conditions that add no semantics beyond the status code. LLM APIs usually benefit from specific types for payload-limit, model-not-allowed, context-limit, policy-denied, upstream-unavailable, and retry-budget-exhausted. Avoid a universal request-failed type. It saves the server designer a few decisions and transfers every decision to the client.
My blunt view is that an error code named invalid_request is a filing cabinet labelled "stuff." It looks organized until someone needs to automate recovery.
Version problem semantics through documented additive changes. A new optional extension can be additive because RFC 9457 requires clients to ignore unknown extensions. A changed meaning or unit, or a changed required recovery path, deserves a new type or an explicitly versioned contract.
Status, title, detail, and instance have different jobs
The status member repeats the HTTP status for convenience and is advisory. RFC 9457 requires generators to use the same status in the actual response. Generic software still follows the HTTP status. Log and alert on disagreement because it can indicate an intermediary rewrite or a server bug.
title is a short summary of the problem type. Keep it stable across occurrences except for localization. detail explains this occurrence and should help the client correct it. Put machine values in extensions rather than asking a client to parse the detail sentence.
instance identifies a specific occurrence. It may be dereferenceable, but it can also serve as an opaque identifier understood by the server. For an LLM API, an opaque path such as /problems/01J... can correlate the client response with a protected decision event. If dereferencing is supported, require authorization and minimize returned data. Sequential identifiers invite enumeration.
Use Content-Language when localized title or detail text matters. Keep the type URI and extension semantics stable across languages. Human text can change without breaking automation.
The AI audit log schema guide covers the internal decision record. A public problem response should reference that record through a safe occurrence ID rather than reproducing sensitive prompt content or internal policy state.
Extensions should carry typed corrective data
RFC 9457 Section 3.2 permits problem-specific extensions and requires clients to ignore extensions they do not recognize. Define each extension in the problem type documentation with its data type, unit, presence rules, and privacy classification.
Useful LLM API examples include:
limitandobserved, withunitfor payload, token, file, or concurrency ceilings.componentfor the field, attachment, message, or route that failed.retryableandretryAfterfor a normalized recovery signal when the HTTP contract allows it.policyCodefor a disclosure-approved authorization or data-policy reason.modelandregionwhen revealing them is safe and helps correction.errorsfor several validation findings of the same problem type, each with a JSON Pointer.
RFC 9457 recommends representing the most relevant or urgent problem when several problems have different types. Avoid a generic batch container that mixes authentication and validation with policy denial in one array. Choose the controlling outcome and make its precedence deterministic.
For field validation, define one validation problem type with an errors array. Each item can carry a pointer plus a stable code and a human detail. Keep submitted values out of the response unless the field is safe to echo. A pointer locates the fault without copying a secret or a block of prompt text.
Security review belongs inside the error design
RFC 9457 Section 5 warns that problem details can expose information useful for compromising a system or violating user privacy. Stack traces, internal hostnames, SQL text, file paths, raw provider bodies, token claims, and prompt fragments should stay out of public errors.
Set a disclosure class for each field. Public fields support client correction. Authenticated fields may reveal tenant-scoped limits or route details. Operator-only data belongs in protected telemetry joined by the instance or correlation ID. The same problem type can keep a stable public shape while the internal event holds richer diagnostics.
Provider errors need normalization. Record the raw upstream response only in an approved protected store, with redaction and retention controls. Return the enterprise API's own type and status to the client. Provider text changes over time and may reveal account or implementation details.
Cache behavior also needs review. Authentication and tenant-specific errors should carry appropriate cache controls. A shared cache serving one tenant's detailed error to another tenant is a data leak. Test the deployed CDN or reverse proxy rather than relying on framework defaults.
Streaming creates a hard response boundary
Problem details work cleanly before response headers and model output bytes have been committed. Once a 200 streaming response begins, that status remains fixed for the exchange. The stream protocol needs its own terminal event or abrupt-close semantics.
Define the streaming boundary explicitly in the route contract. Preflight authentication and authorization, then perform payload checks and route policy before starting the stream. For failures after commitment, send a typed stream event if the protocol supports one and mark the result partial. Link it with an internal occurrence. Avoid presenting that event as an HTTP problem response because the HTTP status has already been sent.
A gateway can also receive a provider failure after it has forwarded initial output. Decide which layer emits the terminal event and which fields are safe for the user. The AI audit log chain-of-custody guide covers preservation of the resulting event. Problem details still govern failures detected before stream commitment.
The acceptance fixture should treat errors as an API
Create a contract fixture containing one request for every problem type and every supported status. Validate the response media type and HTTP status. Check required members and each extension's data type, then check the documentation URL. Run it through the actual ingress, gateway, service, and cache path.
Include these cases:
- Missing or malformed authentication.
- Valid identity with insufficient model authority.
- Invalid JSON and several same-type field errors.
- Payload and token limits at the exact boundary and one unit above it.
- Temporary upstream unavailability with Retry-After.
- A provider timeout before stream commitment and a provider failure after stream commitment.
- An intermediary that changes the HTTP status, triggering mismatch detection.
- An unknown extension received by an older client.
- Two tenants requesting the same failing URL through the shared cache.
For each case, preserve the request fixture, response headers and body, internal occurrence event, and expected client action. The release fails when clients parse detail text or when a type URI lacks documentation. It also fails when an error leaks a fixture secret. Re-run the packet after changing frameworks, proxies, provider adapters, or localization.
DeepInspect
DeepInspect can produce policy decisions and normalized denial evidence for authenticated user or agent requests routed through its HTTP path to an LLM endpoint. At that boundary, a stable problem contract can tell the calling application that a request was denied or redacted, including when it was blocked by route policy, while an opaque occurrence value links to protected audit evidence.
Application validation, provider-native errors, cache configuration, and stream events outside that routed decision remain with their respective owners. Direct provider traffic and local model execution, including STDIO calls, also sit outside the HTTP enforcement record. Keep the problem type documentation honest about which component generated each error and which team owns correction.
Book a demo today.