← All posts

Platform & Architecture

338 posts on platform & architecture.

LLM API Content-Type Enforcement at the Authenticated Request Boundary

LLM API Content-Type enforcement gives each authenticated model route an explicit media-type contract. This guide maps RFC 9110 semantics to JSON inference requests, file-bearing calls, streaming responses, 415 handling, parser alignment, and identity-aware gateway policy. The goal is a small accepted type set, one parsing decision across every HTTP hop, and an audit record that captures the declared and effective request type.

ai-securityllm-securitycybersecurityidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

OWASP API Security Top 10 for LLM endpoints: a route-by-route control map

The OWASP API Security Top 10 applies to LLM endpoints at the HTTP API layer. Object and function authorization, authentication, resource limits, SSRF controls, configuration, inventory, and third-party response validation still govern the route. Prompt injection and model behavior require a separate LLM-security layer. This guide maps each API risk to a testable endpoint control.

ai-securityllm-securitycybersecurityidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

Vercel AI SDK Audit Logs: Turn Telemetry Into a Release Evidence Map

Vercel AI SDK v7 emits OpenTelemetry events for supported model operations after telemetry is registered, with inputs and outputs recorded by default. An audit-ready deployment must map those spans to caller identity, route policy, tool authorization, content-retention choices, and an independent request-boundary decision.

ai-securityagentic-aiauditforensic-auditarchitecture
Read post →

Smolagents Audit Logs: Build an Evidence Record Around OpenTelemetry Traces

Smolagents can export agent-run telemetry through OpenTelemetry, including run spans and token usage alongside inputs and outputs. An audit-ready deployment still needs verified caller identity, policy decisions, tool authorization, retention rules, and evidence that the final LLM request crossed the approved HTTP route.

ai-securityagentic-aiauditforensic-auditidentity-and-authorization
Read post →

Ray Serve Authentication: Test the Token, the Route, and the Bypass

Ray 2.60 can authenticate cluster APIs with a shared token, while Ray Serve exposes application ingress over HTTP. A production release test must prove token rejection, TLS protection, route authorization, caller-context handoff, and closure of every direct-port bypass before the endpoint carries LLM traffic.

ai-securityllm-securityidentity-and-authorizationarchitecturezero-trust
Read post →

LM Studio Authentication: Token Scope, Network Exposure, and Proxy Tests

LM Studio 0.4.0 and newer supports API tokens with configurable permissions, but API authentication is disabled by default. This guide separates token acceptance from end-user identity, maps the local and network server exposure, and defines production acceptance tests for teams routing authenticated HTTP inference through a policy proxy.

ai-securityidentity-and-authorizationllm-securitypolicy-enforcementarchitecture
Read post →

Presidio PII Detection Needs a Prompt-Path Acceptance Test

Presidio combines recognizers, context, scores, and anonymization operators to find and transform PII before an LLM request leaves an application. Its own documentation warns that automated detection cannot guarantee every sensitive value will be found. This guide treats Presidio as a community-owned project, builds a prompt-path acceptance corpus, tests threshold and transformation failures, and defines the evidence needed for production release.

ai-securityllm-securitydata-loss-preventiondlppolicy-enforcementdevsecops
Read post →

OpenAI Agents SDK Audit Logs Need a Trace Export Acceptance Test

The OpenAI Agents SDK traces agent runs, model generations, function calls, guardrails, handoffs, and audio operations by default. Those spans are useful diagnostic evidence, but their value depends on sensitive-data settings, exporter composition, buffer delivery, identity binding, and custody. This guide turns SDK tracing into a release acceptance test and keeps organization audit events separate from routed model-request policy records.

agentic-aiai-securityauditforensic-auditidentity-and-authorizationdevsecops
Read post →

Mastra Security: A Production Deployment Acceptance Guide

Mastra exposes agents, workflows, tools, and custom routes through an HTTP server, while agent model calls follow a separate provider route. A production review needs to test optional authentication, custom API prefixes, request context, provider credentials, identity propagation, and direct-provider bypass. This guide turns those checks into one deployment acceptance record with explicit control owners.

ai-securityagentic-aiidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

LM Studio Security: A Production API Server Acceptance Guide

LM Studio can expose local models through REST, SDK, OpenAI-compatible, Anthropic-compatible, and Jev-compatible APIs. A production review needs to prove the listener scope, token enforcement, permission boundaries, approved inference route, and direct-port denial. This guide turns those checks into a deployment acceptance record while keeping host, model-file, and management controls with their owners.

ai-securityllm-securityidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

LLM Request Signing: A Verification Profile for Model API Calls

HTTP Message Signatures can authenticate selected parts of an LLM API request, while Content-Digest can bind the transmitted prompt body to that signature. The useful security property comes from a strict deployment profile: covered components, signer resolution, algorithms, freshness, replay handling, proxy transformations, and negative tests all need explicit rules.

ai-securityidentity-and-authorizationllm-securityzero-trustarchitecture
Read post →

LLM-as-a-Judge Security: Treat Candidate Output as Untrusted Input

An LLM judge receives candidate answers inside the same inference context as its scoring instructions. That design lets attacker-controlled text target the evaluator rather than the application user. This guide maps the injection path, separates ranking integrity from access control, and turns judge security into a repeatable release test for authenticated HTTP evaluation traffic.

ai-securityllm-securityprompt-injectionpolicy-enforcementarchitecture
Read post →