← All posts

Platform & Architecture

338 posts on platform & architecture.

BentoML Authentication: Bind Every LLM Request to a Principal

BentoML clients can attach a bearer token, while BentoML Services leave credential validation and authorization design to the deployed application or its ingress. This guide builds an authentication path for a BentoML LLM endpoint, covering route inventory, JWT validation, workload and user identity, per-route authorization, health endpoints, direct-path tests, and request evidence.

ai-securityllm-securityidentity-and-authorizationpolicy-enforcementarchitectureaudit
Read post →

Wasm Plugins in an AI Gateway: Keep Policy Small, Signed, and Observable

A Wasm plugin can place request parsing, identity handoff, local denial, and telemetry inside an AI gateway without compiling those decisions into the proxy binary. The useful design keeps provider routing outside the plugin, pins the module artifact, limits host capabilities, pauses model traffic until policy finishes, and records the exact module version behind every outcome.

ai-securityllm-securityarchitecturepolicy-enforcementinline-enforcementaudit
Read post →

AI Gateway File Upload Scanning: Control the Bytes and the File ID

LLM file uploads create two policy moments: the upload that carries bytes and the later model request that carries only a file identifier. Secure handling needs authorization, type and size checks, quarantine, malware and parser isolation, prompt-injection analysis, tenant-bound file references, and a release record that survives reuse. This guide shows where an AI gateway can enforce each decision and where file security remains upstream.

ai-securityllm-securitydata-loss-preventionpolicy-enforcementarchitectureaudit
Read post →

Envoy ext_proc for LLM Policy: A Production Request-Path Guide

Envoy ext_proc can place an external policy service inside an LLM HTTP request and response path. The design depends on processing mode, body buffering, gRPC failure behavior, mutation limits, and a clean identity handoff. This guide maps those choices into a deployment pattern that can deny a model request before upstream transmission and preserve decision evidence without turning the processor into a routing engine.

ai-securityllm-securityarchitecturepolicy-enforcementinline-enforcementaudit
Read post →

AI Agent Honeytokens: Decoys for MCP and Agent Workspaces

AI agent honeytokens turn access to a decoy MCP configuration, URL, credential-shaped value, or workspace artifact into a high-signal alert. Their value depends on placement, unique ownership, and a rehearsed response. This guide separates token alerts from preventive controls and marks the exact point where routed LLM request evidence can help an investigation.

ai-securityagentic-aicybersecurityprompt-injectionforensic-audit
Read post →

Agno Security: The AgentOS Controls That Need Explicit Configuration

Agno AgentOS includes JWT verification, endpoint scopes, request copies, optional user isolation, and persistent approval records. Several protections depend on explicit settings or application-side authorization. This guide turns the current Agno documentation into a security acceptance review and keeps LLM request enforcement inside its real HTTP boundary.

ai-securityagentic-aiidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

Agno Audit Logs: Separate Traces from Runs and Approval Records

Agno spreads operational evidence across OpenTelemetry traces, run and session records, approval records, and authentication context. This guide maps each record to the question it can answer, identifies the configuration that preserves user attribution, and shows where an independent LLM request decision record still belongs.

ai-securityauditforensic-auditagentic-aiidentity-and-authorization
Read post →

OpenAI Agents SDK Prompt Injection: Guard Every Workflow Boundary

OpenAI Agents SDK guardrails run at defined workflow boundaries. Input guardrails cover the first agent, output guardrails cover the final agent, and tool guardrails cover guarded function calls. Prompt-injection controls need to follow those boundaries and govern each routed model request.

ai-securityagentic-aiprompt-injectionllm-securitypolicy-enforcement
Read post →

MCP Tool Shadowing Lets One Server Reframe Another Tool

MCP tool shadowing occurs when malicious tool metadata influences how a model uses another tool. Secure clients pin tool identities and schemas, show metadata changes, separate server trust, authorize every invocation, and correlate the resulting LLM request with tool decisions.

mcpai-securityagentic-aillm-securityidentity-and-authorization
Read post →

MCP Sampling Attacks: Control the Nested Model Call

MCP sampling lets a server ask the client to run another model completion. That nested call carries server-supplied prompt content into an LLM route, so the client needs prompt review, route policy, token limits, and a decision record before it approves the request.

mcpai-securityllm-securitypolicy-enforcementagentic-ai
Read post →

AP2 Agent Payments Protocol: An Implementation Guide to Mandates

The AP2 Agent Payments Protocol gives commerce and payment parties cryptographic evidence of an agent’s purchasing authority. The April 2026 specification replaces the launch article’s broad Intent and Cart vocabulary with open and closed Checkout and Payment Mandates, role-specific verification, signed receipts, plus exact schema versions. This guide maps those artifacts, the direct and autonomous flows, and the controls that remain outside AP2.

agentic-aiagent-paymentsap2identity-and-authorizationpayments-securityengineering
Read post →

vLLM Authentication Needs an Endpoint Exposure Record

vLLM can require bearer credentials on selected HTTP path prefixes, but the same server exposes other inference and non-inference routes outside that check. This guide builds an endpoint exposure record that separates bearer authentication, natural-person identity, route authorization and deployment perimeter controls, plus independent request evidence.

ai-securityllm-securityidentity-and-authorizationarchitecturepolicy-enforcementaudit
Read post →