Retrieval-Augmented Generation (RAG)
Retrieval-augmented generation (RAG) is a pattern that improves a language model's answers by retrieving relevant documents from an external knowledge base and inserting them into the prompt before the model generates a response. Instead of relying only on what the model learned during training, a RAG system searches a document store (often a vector database) for passages related to the user's query, then asks the model to answer using that retrieved context. RAG is how most enterprise AI applications ground responses in current, private data, which also means untrusted or sensitive content routinely enters the prompt.
How a RAG pipeline works
A RAG system converts documents into embeddings and stores them in a vector database. When a user asks a question, the system embeds the query, retrieves the closest-matching passages, and assembles a prompt that combines those passages with the user's question. The model then generates an answer conditioned on the retrieved text. The quality of the answer depends on retrieval, and the safety of the answer depends on what the retrieval step pulled into the context window.
Where the security questions land
RAG moves two kinds of risk onto the prompt path. Retrieved passages can carry sensitive data the requesting user is not entitled to see, and retrieved passages from untrusted sources can carry injected instructions. The vector database and the retrieval logic are infrastructure that sits outside the AI request boundary, so they need their own access controls. What DeepInspect governs is the request that carries the assembled prompt to the model: it classifies the retrieved context on the HTTP path, can redact disallowed content before the call reaches the endpoint, and records the decision.
Related reading
- RAG Security Architecture: The Four Trust Boundaries in a Retrieval Pipeline
A RAG request crosses four trust boundaries before the answer comes back: what gets indexed, who can retrieve which chunks, what the assembled prompt carries into the model, and what the response returns. This article lays out the reference architecture for each boundary, marks which two are data-plane controls and which two sit on the HTTP path to the model, and shows where identity-bound policy and a per-decision audit record belong.
- AI Gateway Redaction for RAG Contexts: Stopping Cross-Tenant Data Leakage
A retrieval-augmented generation pipeline fetches documents from a vector store, concatenates them into the prompt context, and sends the assembled prompt to the LLM. The fetched chunks can carry data the requesting user is not authorized to see. The model has no way to distinguish authorized content from leaked content. An AI gateway that redacts at the context-assembly boundary, with identity-bound policy on each retrieved chunk, is the architectural pattern that stops cross-tenant data leakage in RAG.