The Security Layer in Agentic AI Architecture Sits at the Model-Call Boundary
Most agentic AI architectures diagram the planner, memory, tools, and model, then add security as a wrapper around the application. That places enforcement above the layer where agent decisions become actions. The load-bearing security layer is the model-call boundary itself, where identity, policy, and audit apply to every request an agent makes to a model.
Draw a typical agentic AI architecture and you get four boxes: a planner that decides what to do, a memory that holds context, a set of tools the agent can call, and the model it reasons with. Security usually shows up as a fifth box wrapped around the application, labeled with the SSO provider and the WAF. That placement is the problem. It puts enforcement at the perimeter of the app, above the layer where an agent's decisions turn into requests. The security layer that carries weight in an agentic system sits lower, at the boundary where the agent calls the model. This piece argues for that placement. For the non-security architecture patterns, see agentic AI architecture patterns.
The reference shape of an agentic system
An agent runs a loop. The planner proposes a next step, the agent may call a tool, the result feeds back into context, and the model is consulted again to decide what follows. Every meaningful step routes through a model call. The planner asks the model to plan. The tool selection asks the model to choose. The summarization asks the model to compress. This means the model-call boundary is not one edge of the diagram. It is the recurring chokepoint the whole loop passes through, dozens of times per task.
Where security is usually bolted on
Perimeter security authenticates the human who started the session and inspects inbound web traffic. Both are necessary and neither reaches the agent loop. Once the session is open, the agent makes its own model calls carrying whatever service credential the process holds, and the perimeter has no view of those calls. A WAF does not see the prompt an agent sends to a model API. An SSO token proves a human logged in three hours ago; it says nothing about whether this specific model call, right now, should be allowed. Enforcement at the perimeter governs entry to the application. It leaves the agent's own traffic ungoverned.
The AI request boundary as the enforcement layer
Put the security layer where the traffic actually is. Every model call an agent makes crosses a boundary as it leaves the process for the model API. A control point on that boundary sees each request individually and can decide on it before it executes. That is the difference between governing a session and governing a decision. The AI control plane idea names this layer: a place where AI traffic is authenticated, authorized, and logged, independent of the model and independent of the application code that generated the call. Because the boundary is common to every model call, one layer covers the whole loop.
What the enforcement layer evaluates
At the model-call boundary, four things are checkable that the perimeter cannot see. The calling identity, resolved to a specific agent or user rather than a shared key. The data in the request, classified so that sensitive content triggers policy. The destination model, checked against which models this identity may use. And the organizational policy that ties those together into a pass or block decision. Evaluating these inline, before the call reaches the model, is what makes the layer an enforcement layer rather than a monitor. The overhead for that decision measures under 50 ms in internal DeepInspect testing, against model inference that runs from 500 ms to several seconds, so the check fits inside the latency the call already spends.
The audit plane
The same boundary is the right place to write the record. A decision made at the request layer can be logged with full context: which identity, which model, what policy applied, what the outcome was. That record is the audit plane of the architecture, and it belongs at the same layer as enforcement because both need the same context. Application logs, written a layer up, capture the app's own state and miss the identity and policy detail of each model call. Google Mandiant's M-Trends 2026 report recorded a median 22-second window from initial access to handoff in 2025, so the audit plane also has to be structured enough to reconstruct quickly.
DeepInspect
This is the problem DeepInspect was built to solve. DeepInspect is a model-agnostic control plane that sits inline at the AI request boundary. For every agent-to-model call it evaluates identity, data classification, model authorization, and organizational policy, then makes a pass or block decision before the traffic reaches the model. It runs independent of the model and independent of the agent's application code, so it governs the model-call layer that perimeter controls never reach.
The same layer produces the audit plane. Every decision is written as an event naming the identity, the model, and the outcome, which gives you evidence at the granularity of individual agent decisions rather than whole sessions. Place it once at the request boundary and it covers every model call in the loop.
If you are designing the security layer for an agentic system, let's talk today.
Frequently asked questions
- Why is the model-call boundary the right place for security?
Because every step in an agent loop routes through a model call. The planner, the tool selection, and the summarization all consult the model, so the model-call boundary is the recurring chokepoint the whole loop passes through. A control there sees each decision individually, which perimeter controls like SSO and a WAF cannot do once the session is open.
- Isn't perimeter security enough?
Perimeter security authenticates the human who started the session and inspects inbound web traffic. Neither reaches the agent's own model calls, which carry a service credential and never touch the perimeter again. You need both: perimeter controls for entry, and a control at the model-call boundary for the agent's decisions.
- What does the enforcement layer actually check?
Four things the perimeter cannot see: the calling identity resolved to a specific agent, the data in the request, the destination model against an allowlist for that identity, and organizational policy tying them together. It evaluates these inline and decides before the call reaches the model, which is what makes it enforcement rather than monitoring.
- Where does the audit trail belong?
At the same boundary as enforcement, because both need the same context. A decision made at the request layer can be logged with the identity, model, policy, and outcome for each call. Application logs sit a layer up and record the app's own state, missing the per-call identity and policy detail an audit needs.