How does an AI control plane differ from an AI gateway?

The terms overlap. "AI gateway" describes the deployment topology of an inspection layer on the request path. "AI control plane" describes the policy and audit surface that the gateway evaluates against. A working AI gateway is one mechanism for implementing an AI control plane. Other mechanisms (posture products, MLOps catalogs, in-process guardrails) implement different control planes at different layers of the stack. The Article 12 record only emerges when the control plane sits inline on the HTTP path. A glossary entry for ai-gateway is in the DeepInspect glossary.

Does an AI control plane replace our DLP?

The two solve adjacent problems. A traditional DLP inspects files and email at the network or endpoint layer. An AI control plane inspects prompts and responses at the HTTP request layer between the caller and the model API. The DLP cannot see the prompt because the prompt sits inside the encrypted POST body to the model endpoint. The AI control plane sees the prompt because it terminates the TLS connection. For most enterprises the answer is to keep the DLP for file and email surfaces and add an AI control plane for the LLM request surface.

What happens to control-plane records when a model vendor changes its API?

The records the inspection layer commits are independent of the model vendor's response schema. The record carries the identity, the classification, the policy state, and the decision outcome. The model vendor's response is logged alongside but does not drive the record format. If the model vendor changes its API tomorrow, the audit record series stays valid against the regulator's review.

Does the inspection layer add latency?

The end-to-end overhead measures under 50 ms in DeepInspect's internal testing. LLM inference itself takes 500 ms to 5 seconds depending on the model and the request size. The inspection-layer overhead is well inside the variance of the inference round-trip, which means the user-perceived latency does not move when the control plane is added inline.

Where does the control plane sit when the AI is embedded inside a SaaS vendor?

When the AI runs inside a vendor SaaS product, the deployer's control plane sits at the vendor procurement and contractual layer, not at the HTTP request layer. The architectural answer is to require vendor-side audit records that match the deployer's Article 12 obligation, retrievable on demand. This is the under-reported exposure of embedded-AI vendor usage and is covered in the Article 12 architecture piece.

AI Control Plane: What Sits at the Request Boundary and What an Auditor Reviews

The phrase "AI control plane" reaches the buyer with four very different meanings. Some vendors apply it to a CSPM-style posture product that scans cloud accounts for AI service usage. Some apply it to an MLOps catalog that tracks model versions in the registry. Some apply it to a model-side guardrails library sitting inside the application process. And some apply it to an inspection layer on the HTTP path between authenticated users or agents and the LLM endpoint. Only the last shape produces a per-decision record that satisfies EU AI Act Article 12, and the architectural reason is mechanical.

I want to walk through the four shapes the term currently covers, where each one sits in the request path, what each one can and cannot inspect, and the record series the inspection layer commits at decision time.

The four product shapes the term covers

The first shape is a posture product. It runs out-of-band against cloud accounts and discovers OpenAI, Azure OpenAI, and Bedrock endpoints in use. The record series is asset-shaped: this account uses these AI services. The shape never sees a prompt.

The second shape is an MLOps catalog. It tracks model artifacts, training data lineage, and version metadata. The record series is artifact-shaped: this model was trained on this data and promoted to production at this timestamp. The shape never sees a runtime request.

The third shape is a model-side guardrails library that runs inside the application process. It calls a classifier on the prompt before the request reaches the model SDK. The record series is application-shaped: the application that made the request also wrote the log. That makes the record self-attested and rejected by an auditor at the granularity Article 12 requires.

The fourth shape is an inspection layer on the HTTP path between the authenticated caller and the LLM endpoint. The record series is request-shaped: identity, data classification, policy version, decision, integrity signature, and timestamp.

Where each shape sits in the request path

The mechanical question for any AI control plane is where in the request path the inspection happens. Posture products sit out-of-band against the cloud control plane. They see service-level usage. MLOps catalogs sit inside the model registry. They see artifact lineage. Application-embedded guardrails sit inside the calling process. They see the prompt at the point the application chose to call the classifier, which means the same process that holds the inspection can also bypass it.

The HTTP inspection layer sits on the wire between the caller and the model API. Every request that addresses the LLM endpoint passes through it. The application cannot route around it without exposing the LLM endpoint directly, and that exposure is what the layer is replacing. The inspection sees the prompt, the response, the calling identity that the gateway authenticates against the corporate IdP, and the data classification the classifier returns on the prompt content.

What each shape can and cannot inspect

A posture product can answer "which accounts use AI." It cannot answer "what was in this prompt." An MLOps catalog can answer "which model version was promoted to production." It cannot answer "who was the natural person behind this request." An application-embedded guardrail can answer "what did the classifier say about this prompt." It cannot produce an immutable record once the application that called it has access to the storage layer.

The HTTP inspection layer answers the questions the regulator asks during a review. Which natural person initiated this request. What identity context applied at the moment of decision. What data classification the prompt fell into. Which policy version governed the decision. What the outcome was. The record is committed before the application receives the model's response, which means the application cannot suppress it.

The audit record the inspection layer commits

For each request the inspection layer evaluates, the record carries the identity context the gateway authenticated, the data classification the prompt mapped to, the policy version that governed the decision, the decision outcome (permit, redact, deny), a high-precision timestamp, and an integrity signature that prevents post-hoc modification. The record is signed and tamper-evident. The record persists regardless of the application's runtime state.

The fields map directly to Article 19 of the EU AI Act, which is where the regulation gets specific about what the automatically generated log carries: period of use, reference data checked, input data leading to a match, and identification of natural persons involved. Articles 12 and 19 together produce the field list any defensible AI control plane has to commit per request. The format spec is covered in AI Audit Logs: The Format Spec That Survives EU AI Act, DORA, and Fannie Mae Review.

DeepInspect

DeepInspect is the inspection-layer shape of an AI control plane. It sits inline on the HTTP path between authenticated users or agents and the LLM, evaluates identity-bound policies per request, and commits a tamper-evident audit record before the model response returns to the application. The proxy is stateless. The audit write path is independent of the application. The integrity signature on each record is verifiable by an auditor.

For organizations preparing for the EU AI Act Article 12 deadline on August 2, 2026, the choice of which product shape sits at the control-plane boundary determines whether the deployment can produce the records the regulator expects. If your current AI control plane is shaped as posture, catalog, or in-process guardrails, the request boundary is unmanaged at the granularity the mandate requires.

Book a demo today.