Order RAG Context by Evidence Strength, Not Just Retrieval Rank

Retrieval rank is only one signal

The first result may be semantically close but old, low-authority, or duplicative. If prompt construction simply concatenates chunks in rank order, the model sees no explicit distinction between a current policy and an informal discussion. It may quote the wrong source even when a stronger source was retrieved.

Carry source type, version, effective date, owner, and stable citation ID with each chunk. Apply deterministic rules for authoritative sources and current versions before prompt packing. Keep those rules separate from the model's prose instruction so they can be tested.

Preserve disagreement rather than blending it away

If two documents conflict, do not silently choose whichever appears first unless product policy says it is authoritative. Label the conflict, present both citations, or abstain and escalate. Duplicates should be collapsed or identified so repeated copies do not create artificial confidence.

Use a fixed prompt format with bounded chunks and explicit source boundaries. Then test how answers change when result order is permuted. If harmless ordering changes produce different claims, the system may be relying on prompt position rather than robust evidence selection.

Measure token usage and citation accuracy after packing. Truncation should not cut off the source title, date, or the qualifying sentence that changes the meaning. Keep the final prompt's chunk IDs in traces so an answer can be reviewed against exactly what the model saw. Context ordering is part of retrieval quality because it shapes the evidence available to generation.

Define an evidence packing policy

Order chunks by the task's evidence needs, not by an unexplained universal rule. A direct answer may benefit from the strongest passage first; a comparison may need one current source from each side; a procedure may need steps in source order. Keep source title, effective date, and chunk ID adjacent to the text they describe. Separate documents clearly so a model cannot mistake a heading from one source for a fact in another.

Reserve space for instructions and citations before packing chunks. Estimate tokens with the same tokenizer family used by the generation model and enforce a hard maximum. If a chunk is too long, use deterministic truncation or summarization that preserves the qualifying sentence and source identity. Never split the citation marker away from the evidence it references.

Evaluate ordering with an obsolete policy, two conflicting sources, a later exception, and a long irrelevant document. Inspect the final prompt trace under privacy controls and compare claim support before and after reordering. If the answer changes because relevant evidence moved beyond the context limit, the issue is a packing regression even when retrieval recall is unchanged.

Record selected IDs, dropped IDs, token estimate, ordering policy version, and truncation reason. This allows a reviewer to distinguish a retrieval miss from a context-budget decision and gives the team a concrete metric for improving the pipeline.

Use the same packing policy in evaluation and production. A benchmark that measures retrieved chunks before truncation can overstate answer quality. Review both the selected set and final prompt under redaction, and test conflicting sources where ordering matters. Context construction is a security and correctness boundary, not just string formatting.

Keep source identity adjacent to every chunk after formatting, and test the prompt serializer with an empty result and a maximum-length result. The serializer should preserve delimiters and citation metadata even when one chunk is dropped. This makes context failures visible instead of allowing the model to infer missing structure.

Use the same packing policy in evaluation and production. A benchmark that measures retrieved chunks before truncation can overstate answer quality. Review both the selected set and final prompt under redaction, and test conflicting sources where ordering matters. Context construction is a security and correctness boundary, not just string formatting.

Use the same packing policy in evaluation and production. A benchmark that measures retrieved chunks before truncation can overstate answer quality. Review both the selected set and the final prompt under redaction, and test conflicting sources where ordering matters. Context construction is a security and correctness boundary, not just string formatting.

Record whether a source was selected, dropped for budget, or truncated. A claim missing from the answer may be a retrieval failure or a packing failure, and the remediation differs. Prompt traces should be privacy-safe but sufficiently structured to reconstruct the evidence set.