← All posts

Platform & Architecture

338 posts on platform & architecture.

Letta Security: A Production Deployment Acceptance Guide

Letta security spans model-provider routing and API credentials, plus App Server exposure, WebSocket authentication, Agent SDK permission modes, and direct paths around an approved gateway. This guide turns those controls into a production acceptance record while keeping memory and local tools with their owners. Permissions, session transport, and route enforcement stay separate.

ai-securityagentic-aiidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

Letta Audit Logs: Map Typed Events and Persistent State to Evidence

Letta exposes typed stream events, persistent agent memory, conversation history, session identifiers, and run identifiers. This guide maps those records to the questions they can answer, separates connection-scoped activity from durable state, and defines the extra evidence needed when an authenticated Letta application calls a remote LLM over a controlled HTTP route.

ai-securityauditforensic-auditagentic-aiarchitecture
Read post →

Hugging Face TGI Authentication: API Keys, Exposure, and Proxy Enforcement

Hugging Face Text Generation Inference exposes an API-key setting alongside host, port, CORS, request-limit, and telemetry controls. This guide separates credential possession from end-user identity, maps the current launcher settings to production checks, and defines the routing condition required for an identity-aware proxy to govern TGI traffic.

ai-securityidentity-and-authorizationllm-securitypolicy-enforcementarchitecture
Read post →

Haystack Audit Logs: Separate Pipeline Diagnostics from Audit Evidence

Haystack provides Python logs, structured logs, LoggingTracer output, and traces for understanding pipeline execution. This guide separates those diagnostic records from immutable audit evidence, maps each source to the question it can answer, and defines a correlation test for authenticated LLM requests deliberately routed through an inspectable HTTP policy point.

ai-securityauditforensic-auditllmarchitecture
Read post →

Granite Guardian: Govern Remote Risk Classification over HTTP

Granite Guardian can judge prompts, model responses, RAG groundedness, function calls, and custom criteria. This guide defines a production pattern for calling a remotely hosted Guardian endpoint through authenticated HTTP, testing its yes or no verdicts, and keeping the original model route under separate policy. Local or in-process Guardian inference sits outside this scope.

ai-securityllm-securitypolicy-enforcementidentity-and-authorizationarchitecture
Read post →

Google ADK Security: A Production Deployment Acceptance Guide

Google ADK exposes security controls across identity, tool authentication, callbacks, plugins, Model Armor, sandboxing, network controls, session handling, and observability. This guide turns those separate mechanisms into a production acceptance record with failed-request tests, named owners, and a precise boundary for routed LLM traffic.

ai-securityagentic-aiidentity-and-authorizationpolicy-enforcementarchitecture
Read post →

DSPy Security: A Deployment Acceptance Test for Production Programs

DSPy security depends on the program artifact you load, the cache locations you trust, the LM route you configure, and the application controls around tools and callbacks. This guide turns current DSPy behavior into a production acceptance record with explicit trust boundaries, negative tests, and a narrow role for policy enforcement on routed HTTP model calls.

ai-securityapplication-securitydeployment-securitypolicy-enforcementagentic-ai
Read post →

DSPy Audit Logs: Build an Evidence Map for Every LM Request

DSPy exposes LM history, MLflow traces, callbacks, cache behavior, and provider records at different layers. This guide maps each source to the audit question it can answer, identifies the identifiers needed to correlate a production prediction, and defines the independent policy record required for deliberately routed HTTP model calls.

ai-securityauditforensic-auditobservabilitypolicy-enforcement
Read post →

DPoP Tokens for AI Agents: Bind OAuth Access to a Signing Credential

Demonstrating Proof of Possession binds an OAuth access token to a client-held cryptographic credential and adds a signed proof to each protected HTTP request. For AI agents, the useful release artifact is a negative-test pack covering cryptographic binding, method and URI binding, token hashing, proof freshness, nonce handling, and replay rejection.

ai-securityagentic-aiidentity-and-authorizationzero-trustcybersecurity
Read post →

CIBA for AI Agents: Human Approval on a Separate Device

Client-Initiated Backchannel Authentication gives an AI agent workflow a standards-based way to request authentication and consent on a separate device. A defensible deployment records the initiating agent and authorizing user, requested action and auth_req_id, expiry and delivery mode, plus the final result while keeping tool authorization and execution as separate controls.

ai-securityagentic-aiidentity-and-authorizationzero-trustarchitecture
Read post →

AI Gateway on Istio Ambient: Route Model Traffic Through a Verifiable Egress Path

Istio Ambient gives Kubernetes teams a sidecar-free L4 overlay and optional L7 waypoints. An AI gateway deployment needs both a verifiable egress route and an HTTP inspection point that can read model requests. This guide separates ztunnel, waypoint, TLS, identity, bypass prevention, and AI policy into an implementation plan with acceptance tests.

ai-securityinline-enforcementpolicy-enforcementkubernetesarchitecture
Read post →

BentoML Security: A Release Gate for Self-Hosted LLM Services

A BentoML LLM Service packages model code, dependencies, artifacts, configuration, and an HTTP serving layer into one deployable system. Its security review needs separate evidence for the built artifact, network exposure, request schema, resource limits, secrets, observability, patch state, and routed LLM policy. This guide turns those decisions into a release gate for a self-hosted BentoML endpoint.

ai-securityllm-securityarchitecturedevsecopspolicy-enforcementaudit
Read post →