One Missing RAG Filter Can Expose Another Customer’s Documents
A practical design for tenant-aware retrieval that prevents document permissions from disappearing during ingestion, chunking, search, and caching.
Retrieval is part of your authorization system
RAG search is often treated as a relevance feature. In a multi-tenant product, it is also an access-control decision. If a vector query returns a document the current user cannot read, the model may quote or summarize it even if the interface never links to the original file. The result can be a cross-customer disclosure that looks like an AI hallucination to the user and a serious incident to the business.
Permissions can be lost at several points: during ingestion, when documents are split into chunks, when group membership changes, in a cached search result, or when a query filter is constructed. A tenant ID stored only at the document level may not be enough if chunks are retrieved independently. A correct filter applied after ranking may still expose sensitive content to an intermediate model or log.
Treat every retrieval result as protected data. The system should prove that the user may access each returned chunk before it enters the model context, not merely before displaying a citation.
Carry identity and policy through the pipeline
Resolve identity and tenant membership in trusted application code. Do not ask the language model which organization the user belongs to. Derive the allowed scope from the authenticated session and pass it to the retrieval layer as a mandatory filter. Make it impossible for a user-supplied query or model-generated filter to remove that restriction.
Attach security metadata to every chunk: tenant, source document, access-control list or policy reference, version, and deletion state. Validate metadata during ingestion and fail closed when required fields are missing. If one source document has sections with different access rules, represent those rules at a granularity that matches the actual permissions.
At query time, intersect the user’s authorized scope with the semantic search. Do not retrieve broadly and rely on the model to ignore forbidden results. If the vector database cannot enforce the needed policy efficiently, use a trusted service layer that performs authorization before content is exposed to downstream components. Evaluate the cost of the chosen architecture, but do not trade isolation for a small latency gain without an explicit risk decision.
Treat caches and updates as security boundaries
A cached answer can outlive the permissions that produced it. Key caches by the complete authorization scope, document version, relevant query, and model/prompt configuration. When membership or document access changes, invalidate affected entries. Do not use a global semantic cache that returns one user's grounded answer to another user without rechecking its source set.
Deletion must propagate. When a document is removed or access is revoked, mark its chunks unavailable and update indexes, caches, summaries, and derived artifacts. Define a maximum revocation delay and test it. A user should not continue receiving a cached excerpt after losing access to its source.
Consider logs too. Retrieval traces often contain snippets, document names, or user queries. Apply access controls and retention limits to logs, and redact content where possible. Make sure analytics vendors and debugging systems do not receive data that the product itself is forbidden to expose.
Test adversarial tenant boundaries
Create separate test tenants with unique canary phrases. Ask questions in one tenant that semantically match documents in another. Try direct document IDs, guessed names, filters supplied in natural language, cache hits, and users whose group membership changes during a session. Verify that restricted content never enters the model request, not just that the final answer omits it.
Test ingestion paths independently: uploaded files, connectors, imported email, shared folders, and documents with inherited permissions. Include malformed metadata and permission service outages. The safe behavior during an authorization outage is to return no protected evidence, not to query without a filter.
Log the IDs and authorization decisions needed to investigate an incident, without keeping full sensitive text unnecessarily. Alert on impossible tenant combinations, missing scope metadata, and retrieval results that fail a post-query authorization check. Use a second authorization check before citations or downloadable snippets are returned.
Make the security contract explicit
Document which source systems are connected, how frequently permissions sync, what happens on revocation, and which derived data is retained. Establish an owner for each connector and test it after vendor permission changes. A security review should cover the full path from source to chunk to model context to answer, including backups and observability.
RAG cannot preserve permissions by intuition. The index must carry enough policy information, the query must enforce trusted identity, and every cache and output path must respect the same scope. A useful search result is not useful if the wrong person can see it. Build tenant isolation as a required invariant and test it with data designed to expose cross-boundary mistakes.