How to preserve RAG access controls end to end

RAG access controls fail at every content copy

RAG preserves access control only when every retained representation of a protected document enforces the same authorization decision. Checking a user's permissions before retrieval is insufficient because the index, embedding store, result cache, prompt logs, and evaluation corpus each retain content or a recoverable representation of it.

This is an architectural problem in on-premise RAG, not a cloud-provider problem. Moving inference into a private network changes where these copies reside. It does not make their ACLs converge.

A document permission is usually evaluated against the system of record: a SharePoint item, a file repository object, a database row, or an internal knowledge service. Retrieval then fans that decision into several systems with different data models:

The design target is therefore not "authorize the query." It is "authorize every read path to every copy, and remove or invalidate copies when the source permission changes."

That distinction is why an apparently correct demo can fail after deployment. The demo asks one user one question through one retrieval endpoint. Production introduces retries, batch jobs, direct index access, debugging logs, offline evaluation, and caches that outlive the original authorization decision.

A paralegal testing the system during the demo submits one query against one matter file she is cleared to see, and the answer is correct. Weeks later, an overnight reindex job touches the whole document store to rebuild stale embeddings, running under a service identity that was never scoped to any single matter, because nobody expected a batch job to need the same authorization context as a user's query.

A vector filter is an enforcement point, not a security boundary

Most shared-collection vector designs encode tenancy or document permissions as payload metadata and attach a filter to each search. Qdrant documents this exact pattern: add a tenant payload field to each point and filter on it so each tenant sees only its own data. ¹

That design is useful, but its security property depends on every vector query carrying the required predicate. The collection contains all tenants' vectors. A direct query path, a maintenance script, an administrative endpoint, or a new retrieval service that does not add the filter changes the result from authorization failure to disclosure.

Treat the filter as mandatory infrastructure, not application convention. The service identity that can query the vector store should expose a retrieval interface that receives a signed authorization context and constructs the filter internally. It should not accept an arbitrary JSON filter from every caller. A caller that can submit unrestricted vector queries has access to the corpus regardless of what the chat UI checks.

Separate collections improve blast-radius containment because a query against one collection cannot accidentally return another collection's points. Qdrant also documents the cost: it recommends multiple collections only for a limited number of tenants needing strict isolation. ¹ That is the trade-off in concrete terms. Physical separation creates more index lifecycle work, more schema migrations, more backup and restore targets, and more collection-level credentials.

For a small set of hard boundaries, such as regulated business units or isolated customer deployments, that cost can be justified. For a large tenant population, shared collections are often unavoidable. In that case, the design must assume logical isolation can fail at every query-capable interface.

Metadata filtering also does not automatically become source-of-truth authorization. AWS describes metadata filtering and identity synchronisation in vector databases as a baseline capability that does not provide strong authorization using the source system as the authority. ²

Retrieval semantics can turn an ACL filter into a recall defect

Permission filters are frequently selective. A user may have access to a narrow project, a single matter, or documents shared through a group exception. Applying that filter after approximate nearest-neighbour search produces an operational conflict: the system must choose candidates before it knows which candidates survive authorization.

Weaviate defines post-filtering as running vector search first and removing results that fail the filter afterwards. ³ Its documentation also identifies the consequence of a restrictive filter: the initial unfiltered search can contain no matching candidates at all. ³

This is not merely answer-quality degradation. A team often reacts to empty results by widening the retrieval pool, falling back to another corpus, or asking an LLM to answer from general context. Those fallbacks must preserve the same authorization context or they create a second, less reviewed policy path.

A support engineer debugging why a customer's ticket search returns nothing has an easy diagnostic: drop the permission filter and search every collection to confirm the document exists at all. That diagnostic query, left in a shared debugging script instead of a personal terminal session, is how a widened filter becomes the default retrieval path for the next engineer who reuses the script.

pgvector makes the effect measurable for one common configuration. Its README states that when a condition matches 10% of rows, HNSW with the default hnsw.ef_search value of 40 returns roughly 4 matching rows on average. ⁴ That is a retrieval recall property, not proof that pgvector is unsuitable. It means an ACL filter and approximate-index settings must be tested together with the access distributions that exist in the deployment.

The retrieval contract should record two outcomes separately:

1. The system found no semantically relevant documents the subject may read.
2. The system found candidates, but none survived authorization.

Those cases require different tuning and different operator responses. Combining them into one empty result makes an authorization filter look like a relevance defect.

Embeddings require the same classification as source text

An embedding store is often described as an index, which encourages teams to classify it below the source corpus. That classification is unsafe when the embedding can reveal source information.

The Vec2Text research team trained models to decode text embeddings and reported recovery of full names from a clinical notes dataset. ⁵ An embedding is therefore a retained representation with its own disclosure risk. It needs access controls, encryption policy, backup handling, retention treatment, and incident scope appropriate to the source text it represents.

This changes a common on-premise architecture decision. Running an embedding model locally prevents the embedding request from leaving the environment, but it does not reduce the sensitivity of the resulting vector database. The vectors, their payload metadata, their snapshots, and any replica used for evaluation remain governed copies.

Every chunk should carry immutable provenance sufficient to reconstruct its authorization filter: source object identifier, source revision, chunk identifier, policy revision, and the ACL attributes used for retrieval. The point is not to duplicate an entire source ACL into every vector payload without thought. The point is to make the exact authorization basis visible and invalidatable.

A source ACL update must produce one of two outcomes for every derived chunk: recompute its authorization attributes before it can be served, or remove it from retrieval until recomputation completes. Serving a chunk under its prior policy because ingestion is delayed turns eventual consistency into a permission grant.

Consider a contractor whose access is revoked on her last afternoon on the project. If the retrieval service still uses cached group membership or stale permission metadata, she may continue to retrieve former project passages until that state is refreshed or invalidated. Test the revoked identity, not merely a colleague who retains access.

Caches, logs, and evaluations create the copies teams forget to govern

A retrieval cache is useful only if its key expresses the security context of the result. A cache keyed by query text and collection identifier can return passages selected for one subject to another subject with the same or similar query.

The cache key should include the corpus scope, subject or permission-equivalence class, policy revision, retrieval configuration revision, and query representation. The cached value should retain the source chunk identifiers and policy revision used to create it. On an ACL change, invalidate by source object and policy revision. A time-to-live alone leaves a known interval in which revoked access can continue through the cache.

This is a demonstrated class of failure. ConfusedPilot researchers reported a vulnerability that leaked secret data through a retrieval caching mechanism in Copilot for Microsoft 365. ⁶ The lesson is narrower and more useful than "avoid caching": cache entries are protected data objects, not neutral performance artifacts.

Logs have the same property. Amazon Bedrock states that model invocation logging can collect invocation logs, model input data, and model output data for supported calls. ⁷ In a RAG flow, model input often includes retrieved passages. The log destination has therefore become another content system with its own readers, retention, backups, and export paths.

Inspect retention and deletion at the actual log destination, rather than assuming that disabling a logging configuration removes stored records. The on-premise equivalent deserves the same scrutiny: determine whether observability captures prompts, retrieved chunks, tool output, embedding input, traces, or error payloads, then place those records behind the same data classification and retention rules as their source.

Evaluation data fails differently. It is usually copied deliberately to make offline testing convenient, then granted broad access to developers, analysts, and CI jobs. An evaluation corpus containing real chunks is a second retrieval corpus. Its loaders, notebooks, artifact store, test reports, and model outputs need explicit scope. De-identification can reduce exposure, but it is not a substitute for access control unless it is established for the exact data and threat model in use.

Apply a copy-by-copy authorization procedure

Use this sequence before treating a RAG system as ready for protected content.

1. Define the source authorization record. Specify the source object identifier, subject attributes, group resolution, deny rules, and policy revision that decide access. Do not make the vector database the authority for permissions.

2. Map every retained representation. Include chunks, vector payloads, embeddings, backups, replicas, caches, prompts, traces, invocation logs, evaluation datasets, and generated answer stores. If a component can be restored or exported, it is a copy.

3. Assign an enforcement mechanism to each copy. For example, vector search requires mandatory server-side filtering; a cache requires authorization-aware keys and invalidation; logs require restricted readers and retention deletion; evaluation data requires scoped identities and source-derived permissions.

4. Carry provenance through ingestion. Store source object ID and policy revision with each chunk. Build a revocation queue keyed by source object so an ACL change identifies all derived points, cached results, and evaluation fixtures that require deletion or recomputation.

5. Test revocation rather than only allowed access. Remove a user's permission after content has entered the index, populated the cache, appeared in a prompt trace, and entered an evaluation workflow. Verify each copy becomes inaccessible or is deleted according to its retention policy.

6. Instrument authorization outcomes. Record that filtering occurred and which policy revision was used. Avoid placing protected passages in the authorization telemetry itself. An audit record should prove the decision without recreating the document in the logging system.

For systems containing electronic protected health information, this procedure maps directly to the scope of HIPAA Security Rule 45 CFR 164.312(a)(1). The provision requires technical policies and procedures for electronic information systems that maintain electronic protected health information so access is limited to persons or software programs granted rights. ⁸ A vector index, log store, and evaluation repository that maintain that information do not fall outside the duty because they are derived systems.

The broader deployment questions still matter. How enterprise RAG works across two pipelines is useful when mapping ingestion and query paths, while Local LLMs are not automatically HIPAA compliant explains why local inference alone does not close the compliance gap.

What this approach does not solve

Copy-by-copy enforcement cannot repair a source system with incorrect permissions. If group membership is wrong, the RAG system will consistently apply the wrong decision. It also cannot provide immediate revocation where a source system cannot publish permission changes promptly. The correct response in that condition is to define the exposure window explicitly and block retrieval when policy freshness cannot be established for sensitive content.

It does not eliminate inference risk from answers a properly authorized user receives. Once a user can read a source document, a RAG answer can summarize it. Authorization controls who receives that summary; they do not prevent an authorized recipient from repeating it elsewhere.

It also does not make shared vector infrastructure equivalent to physical isolation. Shared collections can be operated safely only if the retrieval API, credentials, filters, backups, and administration paths all preserve the same tenant scope. When that assurance is too difficult to sustain, the cost of separate collections or separate deployments is the cost of a clearer boundary.

The final design review should therefore ask for a revocation path for each copy, not a single screenshot of a filtered search result.