Document-Level Security in RAG Must Filter Before Retrieval

A RAG system leaks a document before generation when retrieval places an unauthorized chunk inside the model prompt. Output filters arrive too late. Document security has to constrain the search query using trusted caller identity and current source permissions.

TL;DR

• Attach tenant and access metadata to indexed documents and filter retrieval before prompt assembly.
• Derive identity and group membership from trusted authentication, rather than client-supplied filter fields.
• Test permission removal, stale indexes, shared links, and citations with two accounts that have different access.

Security trimming belongs in retrieval

Microsoft documents security trimming for Azure AI Search using identity or group information stored with indexed documents and applied as a query filter. Pinecone documents metadata filtering for limiting vector search by structured fields.

The products differ. The control point is the same. The retriever narrows candidate chunks before the application builds the LLM context. A response filter can catch certain output patterns; it cannot undo disclosure to a provider after an unauthorized chunk entered the prompt.

The order of operations should be visible in the request path: authentication establishes the subject, the application resolves current entitlements, and the retriever combines those entitlements with the semantic query. Only the permitted result set should reach reranking, prompt assembly, or a model call. If an unfiltered candidate list is logged, cached, or sent to another service first, the filter sits too late in the path.

The access metadata model

Each indexed unit needs enough metadata to enforce the source permission model. Common fields include tenant, document ID, classification, owner, allowed users, allowed groups, and source version. Chunk records should retain the parent document identity so a partial chunk cannot lose its access rule.

Avoid accepting an allowed_groups value directly from the browser. Resolve the authenticated subject and groups on the server, then construct the filter. A handwritten query in a developer console should fail when it tries to claim another tenant.

Metadata also needs a consistent rule for inherited access. A document may receive permissions from a folder, workspace, or tenant boundary rather than from its own record. The indexing job should materialize the effective rule or retain enough source context to calculate it. Otherwise, two chunks from the same document can produce different access decisions after a partial update.

Permission synchronization creates a window

Source permissions change after indexing: an employee leaves a project, a Drive document loses domain sharing, or a case folder becomes restricted. Define how the index receives that change and how long stale access can persist. High-risk sources may require live authorization after retrieval and before prompt assembly.

Run a physical two-browser test: one account keeps access while the second account is removed from the source group. Record the source change time, index update time, query result, and first denied result. My opinion is that a vendor promise to "respect permissions" has little value without this timed revocation test.

Deletion and access removal should travel through the same operational path as additions. A successful ingestion run proves that new text arrived, but it says nothing about revoked text leaving the searchable set. Track the last permission update separately from the last content update, then test both timestamps during the revocation exercise.

Evidence for every model request

Store the caller identity, group or entitlement version, filter policy, retrieved document IDs, source versions, model route, and final decision. Citations help users inspect an answer. The audit record helps security reconstruct which source text entered the prompt.

The identity-aware gateway guide covers verified caller context. The LLM DLP guide covers classification of the assembled prompt before it crosses the model boundary. Both controls matter because retrieval authorization and outbound content policy answer different questions.

Failure tests

Test a missing identity, an unknown group, a stale document, a deleted source, a cross-tenant query, and a shared-link document. Confirm that failures return no restricted chunks and that the system fails closed when the authorization service is unavailable.

Also inspect caches. A response or retrieval cache keyed only by prompt text can return content across users even when the original query was filtered correctly.

Include reranking and fallback paths in the same test set. A primary retriever may apply the correct tenant filter while a keyword fallback, citation lookup, or retry path omits it. Compare the document IDs at each stage and verify that every branch uses the same authenticated subject and permission version.

DeepInspect

DeepInspect can evaluate authenticated HTTP AI requests after retrieval and before the assembled prompt reaches an LLM. It can enforce data-class and route policy and preserve decision evidence. Source ACL indexing and retrieval filtering remain responsibilities of the RAG system.

Book a technical deep dive at deepinspect.ai.