Keep RAG Answers Fresh With Document Version and Effective-Date Metadata

Stale evidence can look highly relevant

An embedding index can retrieve an obsolete policy because its wording matches the query perfectly. Deleting old content may be wrong when users need historical answers, but mixing every version into one undifferentiated search creates ambiguity. The pipeline needs a way to identify current and historical material.

At ingestion, preserve the source's stable document ID, version, publication date, effective-from and effective-to dates where available, status, owner, and canonical URL. Do not infer current status from the file's modification time alone; a reformat can change that timestamp without changing policy meaning.

Decide what “current” means for each task

For operational questions, filter to the current effective version and include the date in the citation. For audit or historical questions, allow explicit version selection and label the result. If the user asks a date-sensitive question, retrieve versions that cover that date rather than silently returning today's policy.

During reindexing, make updates idempotent. Replace or supersede chunks using stable identifiers so a renamed document does not leave duplicate active copies. Keep deletion and permission propagation in the same ingestion contract; stale access metadata can be as serious as stale content.

Test with a query whose answer changed between two versions. Assert that a current query cites the current version and a historical query can retrieve the correct older one. Freshness is not simply “prefer newest”: it is a version-selection rule tied to the user's intent and the evidence required.

Model version state explicitly

Store stable document identity separately from version identity. A source URL, filename, or database primary key may remain constant while its policy changes. Chunks should carry the version, effective dates, publication status, owner, and a canonical reference to the source snapshot. During ingestion, make upserts idempotent and mark superseded versions rather than leaving duplicate active chunks whose metadata is indistinguishable.

Define behavior for documents with missing dates, overlapping effective windows, or a deletion event that arrives after a content update. Do not infer current status from file modification time alone. If the business rule cannot choose between two active versions, abstain or route the record for review instead of silently ranking one as newer.

Test freshness with near-identical documents where only one sentence changes. Ask current, historical, and date-specific questions and inspect both retrieved IDs and citations. Include a permission change between versions; an old document that was once visible may no longer be authorized. Refreshing embeddings is not enough if stale metadata remains in the index or cache.

Expose freshness in the answer when it affects meaning. A citation should show the relevant version or effective date, and a cache key should include the freshness policy. This prevents a cached answer generated for today's policy from being returned to a user asking what applied last year.

During reindexing, publish a new index version and switch traffic only after freshness and permission checks pass. Retain the old version for rollback, but do not let it remain the default through an opaque cache. Record which index produced each answer so a stale result can be traced to a deployment boundary.

When extraction confidence is low, preserve the source page for human review and prevent the row from being treated as authoritative automatically. Store parser version and table identity in metadata. This makes a bad numeric answer traceable to extraction rather than incorrectly blamed on embeddings or prompting.

Run a clean re-extraction on a representative sample after changing the parser. Compare row count, headers, units, and footnotes before updating the index. Keep old and new extraction versions distinguishable so a citation can be traced to the parser that produced it.

Test tables with merged cells, repeated headers, OCR substitutions, and a footnote that changes the meaning of a value. Store a source locator that can identify the page and table region. A whole-document citation is weak evidence for a numeric answer because it does not prove which row, unit, or exception was used.

Represent each extracted row with the table title, all relevant column headers, row label, normalized value, original value, and unit. For a wide table, split logical column groups rather than flattening cells in an order that loses relationships. Keep page number and source version in metadata and reject extraction that cannot establish header context. Evaluate table retrieval separately from prose retrieval before changing embeddings or prompts.

When a table spans pages, repeat headers on every page and verify that extraction has not attached a continuation row to the next section. Preserve footnotes that qualify exceptions, thresholds, or date ranges. Store the parser version with each indexed row so a numeric answer can be traced to the exact extraction run and corrected without guessing which representation the model saw. Test a row lookup, a column comparison, a unit conversion, and an answer that should be rejected because the table lacks enough context.

For wide tables, split logical column groups rather than flattening cells into an ambiguous sequence. Keep the table title, row label, unit, page number, source version, and parser version with every row. Compare extraction output against representative source pages before indexing. If a value cannot be tied to its header and qualifying footnote, route it for review instead of allowing a fluent model answer to give it false authority.

When a table spans pages, repeat headers on every page and verify that extraction has not attached a continuation row to the next section. Preserve footnotes that qualify exceptions, thresholds, or date ranges. Store the parser version with each indexed row so a numeric answer can be traced to the exact extraction run and corrected without guessing which representation the model saw.

For calculations, keep normalized values alongside displayed units and retain the original cell text for audit. Test percentages, currencies, dates, and blank cells separately. A parser that returns a plausible number with the wrong unit is a semantic failure even when retrieval and generation are technically successful.