Preserve Citation IDs When RAG Results Are Reranked

Rank positions are not identities

Many citation bugs come from treating “result number two” as a durable source identifier. Retrieval returns ranked chunks, a reranker reorders them, and prompt formatting may truncate or merge content. If the model cites [2] but the UI maps [2] against the original list, the displayed source can be wrong even though the answer used a different chunk.

Assign every chunk a stable ID and carry it with the text, document ID, section path, version, and access scope. After reranking, create the prompt's citation labels from the final ordered list and preserve a direct mapping from each label to that stable ID. Never reconstruct the source from rank alone.

Test reorder and truncation explicitly

Write a unit test in which reranking reverses two results. Assert that each rendered citation resolves to the text actually supplied under that prompt label. Add a case where a chunk is dropped because of the token budget and confirm that the model cannot cite it.

If chunks are merged for context, keep a list of contributing source IDs and show citations at the narrowest useful level. Do not attach a citation to an entire document merely because one chunk supported one sentence.

Log citation IDs, source versions, and prompt item ordering for a sampled request with privacy controls. That lets you reproduce a mismatch without retaining sensitive text. A citation link should be resolved against the same authorized source snapshot that retrieval used; otherwise the user may see an updated or inaccessible document that does not support the generated answer.

Preserve identity through every transformation

Create a retrieval record with a stable chunk ID, source document ID, version, page or section location, authorization scope, and retrieval metadata. Pass that record through filtering, deduplication, reranking, context packing, and generation. Never regenerate a citation number from the final list without retaining the underlying stable identity. Display labels such as [1] are presentation values; the source ID is the contract used for verification.

Test common transformations explicitly. A reranker may reorder candidates, a deduplicator may remove one chunk, and a context budget may drop a candidate. The answer renderer must reject a citation that points to a dropped or unauthorized record. Include a test where two chunks share a title but have different versions, and another where a source ID contains characters that need escaping in a URL. These cases catch mappings that appear correct in simple examples.

Citation resolution should use the same authorization context as retrieval. If the user cannot open a cited document, either provide an authorized source representation or omit the citation and abstain from the unsupported claim. Do not “repair” a broken link by resolving the ID against the newest public document; that can change the evidence after generation.

Monitor unknown citation IDs, duplicate citation labels, and citations whose source version differs from the prompt trace. These are pipeline integrity signals, not merely formatting defects. A sampled trace with stable IDs and redacted text gives reviewers enough information to diagnose the problem while respecting document privacy.

Add renderer tests for dropped candidates, duplicate display labels, escaped URLs, deleted sources, and permission changes between retrieval and citation display. The renderer should fail closed when it cannot resolve an ID. A citation that looks plausible but points to different evidence is worse than a visible missing citation because it creates false confidence.

Also test citations after deduplication and reranking reorder the list. The display label may change, but the underlying source ID must remain stable. Verify that a source removed for authorization or context budget cannot be cited by stale model output. Reject unknown labels at the application boundary and record the failure without exposing private source text.

Use a source snapshot or version when resolving a citation. A live document can change after generation and make a previously correct answer appear unsupported. The citation service should either show the retrieved snapshot or clearly identify that the source has changed and require a new answer.

Keep a human-reviewed sample of threshold decisions and label both false confidence and unnecessary refusal. These errors have different product costs. A threshold change should include the score distribution, affected query classes, and the expected change in escalation volume, not only a new average answer score.