AI Governance for Security Architects: Put Policy on the Model-Call Path

An AI governance policy reaches architecture at one specific moment: an authenticated user or agent sends an HTTP request to an LLM endpoint. The security architect has to decide which identity travels with that request and where data gets classified. The architect also decides which component evaluates policy and how failure behaves. A separate decision determines where the evidence is written. Every box around that moment either carries a control or creates an assumption.

I would start the design review by drawing one request across a whiteboard, including every header and trust boundary. A polished reference diagram can wait.

TL;DR

• Security architects should place an explicit policy decision point on the HTTP path between authenticated callers and LLM endpoints.
• Identity and role need to travel through one reviewable decision. That decision also carries data classification and model authorization, then records policy version and outcome.
• The design should fail closed and separate policy administration from enforcement. It should also preserve audit records outside the application write path.
• MCP tools and local processes retain their own control boundaries, while destination-system permissions remain with the destination system.

The model call is a resource request

NIST SP 800-207 defines zero trust around users and assets in relation to resources, with authentication and authorization performed as discrete functions before a session to an enterprise resource is established. An LLM route is one of those resources. Authentication establishes the caller before authorization evaluates the request. Authorization decides whether that caller may send this classified content to this model under the active policy.

That distinction exposes the post-authentication gap. A valid JWT can identify an employee or service. It can also identify an agent while saying nothing about the data inside the prompt. A security architect should specify how identity context reaches the model-call boundary and how the policy point verifies that context. The architect should also specify what happens when attributes are missing.

The request path also needs a defined resource name. "Claude" or "OpenAI" is too broad when one deployment permits an internal summarization route and blocks a public endpoint. Bind policy to the application route, model destination, business role, and data classification. The resulting decision is specific enough to test.

Separate the policy plane from the data path

The architecture needs two related planes. The policy plane is where authorized administrators define rules and approve versions. It is also where they set exception conditions. The data path is where a request encounters the current rule and receives a permit or deny outcome before reaching the model. The request can also be redacted before permitted delivery.

NIST AI 100-1 places documented policies and controls in GOVERN 1. GOVERN and MAP cover documented controls and context. MEASURE and MANAGE cover monitoring and treatment. The architect's job is to preserve that separation in components. A policy repository should identify the approved version. A stateless enforcement point should evaluate the live request. An independent evidence sink should retain the decision.

AI governance operating models often assign a committee authority to approve policy. Production traffic cannot wait for a committee meeting. The approved rule has to compile into a deterministic decision at request time.

My opinion is that a governance design with no named enforcement point is still a document-management design. It has architecture-shaped boxes and no control surface.

Identity and classification converge at one boundary

The application owns the first identity binding. It authenticates the human or workload and passes verified context downstream. The enforcement layer should validate the issuer and audience. It should check expiry and role, then validate tenant and route-relevant claims before using them in policy. Shared service credentials collapse users into one principal and weaken every later record.

Classification happens on the decrypted HTTP payload. The security architect should define classes that map to existing obligations, such as PII, PHI, privileged matter, source code, credentials, or pre-announcement financial data. An AI security proxy can evaluate those classes where the application-to-model request is visible, rather than asking a network control beneath TLS to infer prompt contents.

The decision contract should remain compact: identity reference, role, tenant, route, destination model, classifications, policy version, outcome, reason code, exception identifier, and timestamp. A line of JSON containing those fields is more useful during review than a green shield icon in an architecture deck.

Failure and evidence are architectural choices

Fail-closed behavior needs an explicit scope. If identity verification fails or policy cannot load, the protected route should stop. The same applies if the evidence write cannot complete. A health endpoint and operational runbook should distinguish that controlled denial from a model-provider outage. Product owners can then design user messages and retry behavior without weakening the security decision.

Write-path independence keeps evidence outside the application under review. The application under review should never have sole custody of the record used to prove its own behavior. AI governance audit explains the evidence an assessor requests. At the architecture level, preserve a correlation ID across the application event, policy decision, model call, and response. Sign the decision record or send it to a tamper-evident store before returning the response.

A local MCP server communicating over STDIO and a stolen credential used outside the governed route sit outside an HTTP AI proxy. A vulnerable destination tool sits outside that boundary as well. Endpoint controls and credential protection own the first two surfaces, while tool authorization owns the destination surface. The architecture should mark them rather than shade the whole diagram as covered.

DeepInspect

DeepInspect is a stateless proxy on routed HTTP AI traffic between authenticated users or agents and LLM endpoints. It evaluates identity, role, data classification, destination, and organizational policy inline, then produces a signed, tamper-evident record for each decision outside the application's write path.

That gives the security architect a defined policy decision point and evidence path without claiming ownership of local execution, STDIO, credential storage, or downstream tool authorization. The application supplies identity context, and the network design keeps protected model traffic on the governed route. Book a technical deep dive at deepinspect.ai.