Does a gateway replace vector database access controls?

No. Vector database access controls enforce retrieval-time authorization, which is boundary two in this architecture, and they run inside the data store where the query executes. A gateway on the AI request path operates at boundaries three and four, on the assembled prompt and the model response. The two are complementary. Metadata filtering in the vector store decides which chunks a caller retrieves, and the gateway governs what happens to those chunks once they are serialized into a prompt bound for the model, including detecting injected instructions and redacting regulated data the retriever surfaced.

Where does prompt injection through retrieved context get caught?

At boundary three, the assembled prompt. Injected instructions planted in a source document survive ingestion and retrieval as ordinary text, then travel into the model inside the retrieved context. A proxy reading the decrypted prompt before it reaches the provider can flag context segments that carry imperative instructions inconsistent with the caller's query. Ingestion-time scanning at boundary one helps, and it cannot see content added after indexing, so the runtime check on the outbound prompt is the reliable catch point.

What audit record does a RAG pipeline need?

A per-request record that names the authenticated caller, the retrieved sources that entered the prompt, the policy decision applied, and the response returned. Application logs usually record the question and the final answer and omit which documents were retrieved and whether any were redacted. A control on the request path captures the decision at the moment it happens, which is what a regulator or a customer data-subject request needs when the question is whether a specific person's data was sent to a third-party model.

Is RAG security different from general LLM security?

RAG adds the retrieval corpus as an attack surface and a data-exposure surface that a bare LLM call does not have. A direct prompt carries only what the user typed. A RAG prompt carries whatever the retriever pulled, which can include regulated data the user never saw and instructions an attacker planted. The general controls still apply, and RAG layers the four-boundary model on top, with the retrieval boundaries in the data plane and the prompt and response boundaries on the request path.

RAG Security Architecture: The Four Trust Boundaries in a Retrieval Pipeline

A retrieval-augmented generation request looks like one API call from the outside, and it is four security decisions on the inside. A user question arrives, an embedding turns it into a vector, the vector database returns the nearest chunks, the application assembles those chunks into a prompt, and that prompt goes to the model. Each hop crosses a trust boundary where content changes hands and where a wrong answer to "who is allowed to see this" becomes a data-exposure event. I want to lay out the reference architecture boundary by boundary, mark which ones a gateway on the AI request path can enforce, and be exact about which two sit in the data plane instead.

Boundary one: index ingestion trust

The first boundary is what enters the index. A RAG system is only as trustworthy as the corpus behind it, and most corpora are assembled from wikis, ticket systems, shared drives, and PDFs that no one classified for sensitivity. Two failures start here. Poisoned content planted in a source document becomes retrievable instruction the moment it is embedded, the pattern documented in RAG poisoning research. Over-broad ingestion pulls HR records or unredacted contracts into the same index that a customer-support bot queries. This boundary is a data-plane control. It lives in the ingestion pipeline, the document ACL model, and the classification step that runs before embedding, and a gateway on the HTTP path to the model has no visibility into it.

Boundary two: retrieval-time authorization

The second boundary is who can retrieve which chunks. A single shared index with no per-query filtering means every caller retrieves against every document, so an intern's question and a CFO's question search the same vectors. The correct pattern attaches the caller's identity and entitlements to the retrieval query and filters candidate chunks by document-level ACL before they are ranked, so retrieval returns only what the caller is cleared to see. This is metadata filtering in the vector store keyed to the authenticated principal, and Pinecone, Weaviate, and pgvector all support namespace or metadata scoping for exactly this. Retrieval-time authorization is also a data-plane control, enforced at the vector database, not on the model request. It has to be built where the query runs.

Boundary three: the assembled prompt

The third boundary is the prompt that carries retrieved context into the model, and this one sits on the HTTP path. Whatever the retriever returned, correct or poisoned, scoped or over-broad, is now serialized into the messages payload heading for the LLM endpoint. A proxy on that path reads the assembled prompt after TLS termination and before it reaches the provider. It can detect retrieved context that carries injected instructions, redact PII or PHI that the retriever surfaced from a document the caller technically could open but should not send to a third-party model, and bind the whole request to the identity the application passed through. This is the control point that catches a retrieval mistake made two boundaries earlier, because the mistake still has to travel through the prompt.

Boundary four: the model response

The fourth boundary is what comes back. The model's completion can echo retrieved context verbatim, reconstruct a redacted value from surrounding tokens, or return source passages the caller was never entitled to read. A control on the response path inspects the completion on the same request the prompt went out on, applies output policy, and records what was returned. The context redaction pattern belongs here and at boundary three together, because redacting the inbound context and inspecting the outbound answer are two halves of the same decision. Response inspection also produces the record an auditor asks for: which caller, which retrieved sources, which answer.

Where the four boundaries leave the enforcement gap

Boundaries one and two are data-plane controls at the ingestion pipeline and the vector store. Boundaries three and four sit on the decrypted HTTP path between the application and the model, and that is where a request-layer proxy operates. Most RAG deployments invest in boundary one classification and skip the runtime boundaries, so a poisoned chunk or an over-broad retrieval reaches the model with nothing between it and the provider. The broader defense-layer view is in securing RAG systems, and the injection-specific angle is in RAG prompt injection. The architecture point is that retrieval correctness and prompt-payload governance are separate jobs, and a system that only does the first ships regulated context to a third party on every query it gets wrong.

DeepInspect

This is the runtime half of RAG security. DeepInspect is a stateless proxy on the AI request path, sitting between your application and the model endpoint your RAG pipeline calls. It reads the assembled prompt after TLS termination, including the retrieved context the application concatenated in, classifies that content across PII, PHI, source code, and jurisdictional categories, binds the request to the caller identity your application forwards, and applies a pass, block, or redact decision before the prompt reaches the provider.

It inspects the response on the same request and commits a per-decision audit record naming the caller, the policy applied, and the outcome. It runs inline and can fail closed, so a poisoned or over-broad retrieval is stopped at the model boundary rather than reconstructed afterward from logs. It does not replace your document ACLs or your ingestion classification, and it is the control that catches what those miss once retrieved content is in flight to the model. Book a demo today.