← All posts

Platform & Architecture

338 posts on platform & architecture.

MCP Gateway Setup: Putting a Governed Proxy in Front of Model Context Protocol Traffic

An MCP gateway sits between agents and the MCP servers they call, centralizing authentication, per-tool-call authorization, egress control, and audit. This guide walks the setup on the streamable HTTP transport: terminate the connection, validate the OAuth token audience, authorize each tools/call against role and arguments, propagate identity to downstream servers, and log every decision. It marks where a local stdio server leaves the reach of a network gateway.

ai-securityagentic-aillm-securityidentity-and-authorizationarchitecture
Read post →

LangChain Security Patterns: Governing Tools, Egress, and Output in a Chain

LangChain hands the model tools, memory, and retrieved context, and each one is a trust boundary the framework does not police for you. This article walks the security patterns that matter in a LangChain app: constrain what tools can do, propagate caller identity through the chain, validate output before it drives an action, and constrain egress. It marks which patterns live in your code and which need an enforcement layer on the HTTP path.

ai-securityagentic-aillm-securityarchitectureidentity-and-authorization
Read post →

MCP Security vs API Security: What Changes When Tools Are Discovered at Runtime

MCP looks like an API, so teams reach for their API security playbook and inherit its blind spots. This article walks four places where MCP security departs from classic API security: tools are discovered at runtime rather than fixed at design time, tool descriptions are an input the model trusts, the caller is often a non-human identity, and authorization is per tool call rather than per endpoint. It shows which API controls carry over and which do not.

ai-securityagentic-aillm-securityidentity-and-authorizationarchitecture
Read post →

RAG Security Architecture: The Four Trust Boundaries in a Retrieval Pipeline

A RAG request crosses four trust boundaries before the answer comes back: what gets indexed, who can retrieve which chunks, what the assembled prompt carries into the model, and what the response returns. This article lays out the reference architecture for each boundary, marks which two are data-plane controls and which two sit on the HTTP path to the model, and shows where identity-bound policy and a per-decision audit record belong.

ai-securityarchitectureragllm-securityinline-enforcementidentity-and-authorization
Read post →

AI Policy Enforcement at the HTTP Layer: Why the Request Boundary Is the Control Point

AI policy can be enforced at four layers: the model, the application, the network, and the HTTP request boundary. This piece walks each layer, shows why the model layer is probabilistic, the application layer self-attesting, and the network layer blind under TLS, and argues that the HTTP request boundary is the only point that sees decrypted prompt content plus identity and can make a deterministic, recordable decision.

policy-enforcementarchitectureai-securityinline-enforcementzero-trust
Read post →

AutoGen Security Patterns for Multi-Agent LLM Traffic

AutoGen makes it quick to stand up multi-agent systems, but each agent opens outbound LLM calls that need identity-aware authorization and an audit trail. This walks through where the AI-traffic boundary sits and the security patterns that hold at machine speed.

agentic-aiai-securityllm-securityidentity-and-authorizationinline-enforcementai-governance
Read post →

Cohere API Gateway Patterns: Governing Chat, Embed, and Rerank Traffic

Cohere exposes three distinct request shapes: v2 chat generation with the Command models, v2 embed, and v2 rerank, each a separate route with a different data-exposure profile. Routing Cohere traffic through a policy gateway lets a platform bind every call to a natural-person identity, apply per-route policy, constrain egress to the Cohere host, and record each decision. This walks the gateway patterns for the three endpoints, including the private-deployment case.

ai-securityllm-securitypolicy-enforcementidentity-and-authorizationarchitecture
Read post →

CrewAI Security Patterns: Governing What Your Agents Send to the Model

A CrewAI crew is Python: agents with roles, tasks, tools, and a process that runs in your own process space. Most of that orchestration sits outside a network gateway. The part a gateway governs is the outbound HTTP traffic, when an agent calls an LLM through LiteLLM or reaches an external API through an HTTP tool. This walks the security patterns for that boundary: per-agent identity on model calls, prompt classification, egress control, and an audit record of what each agent asked the model.

agentic-aiai-securityllm-securityidentity-and-authorizationpolicy-enforcement
Read post →

Azure OpenAI Gateway Setup: A Governed Proxy in Front of Your Deployments

Azure OpenAI addresses models by deployment name on a resource-specific host, authenticates with either an api-key header or a Microsoft Entra ID token, and carries an api-version query parameter on every call. This guide walks the setup for a governed gateway in front of those deployments: terminate the connection, validate the Entra ID token, bind a natural-person identity, apply per-deployment policy, constrain egress to the Azure host, and commit an audit record for each decision.

ai-securityllm-securityidentity-and-authorizationpolicy-enforcementcloud-security
Read post →

Groq API Gateway Patterns: Identity-Bound Policy at Inference Speed

Groq serves an OpenAI-compatible API on its LPU hardware, which returns completions fast enough that a slow, out-of-band control has no time to act before the response lands. That speed is the argument for inline enforcement, not against it. This walks the gateway patterns for Groq traffic: point the OpenAI-compatible base URL at the gateway, bind each call to a natural-person identity, apply per-model policy, constrain egress to the Groq host, and record each decision.

ai-securityllm-securityinline-enforcementpolicy-enforcementarchitecture
Read post →

Mistral API Gateway Patterns: One Policy Layer for Hosted and Self-Hosted

Mistral is two deployment stories under one brand: the hosted La Plateforme API at api.mistral.ai with a Bearer key, and open-weight models a team runs itself behind vLLM, TGI, or Ollama. A single identity-bound gateway can govern both, because both are HTTP endpoints. This walks the Mistral gateway patterns: bind each call to a natural-person identity, apply per-model policy, constrain egress for the hosted case, cover the self-hosted endpoint the same way, and record every decision.

ai-securityllm-securitypolicy-enforcementidentity-and-authorizationarchitecture
Read post →

Vertex AI Gateway Setup: A Governed Proxy in Front of generateContent

Vertex AI addresses models by a long resource path on a regional host, authenticates with a short-lived Google OAuth bearer token rather than a static key, and offers both the native generateContent shape and an OpenAI-compatible endpoint. This guide walks the setup for a governed gateway in front of Vertex: terminate the request, validate the Google identity, bind a natural-person identity, apply per-model policy, constrain egress to the regional host, and commit an audit record for each decision.

ai-securityllm-securityidentity-and-authorizationpolicy-enforcementcloud-security
Read post →