Keep RAG Query Rewrites From Dropping Identifiers or Negation
Query expansion can help with shorthand, but careless rewrites erase error codes, version numbers, and negation. Preserve literal tokens and evaluate rewritten queries alongside originals.
A fluent rewrite can be a worse search query
An LLM may rewrite “why does 2.4.1 not connect to ERR-18?” into a smooth sentence that omits the version or flips the negation. The rewrite reads better but retrieves the wrong evidence. Exact identifiers, quoted strings, dates, and words such as “not” often carry more search value than the surrounding grammar.
Preserve the original query as a parallel retrieval path or explicitly extract literal terms before rewriting. Compare retrieval results for the original, rewritten, and combined query on a labeled evaluation set. Do not overwrite the user text in logs or in the interface; the rewrite is an internal retrieval aid.
Constrain what the rewrite may change
Ask the rewrite stage to retain identifiers verbatim, preserve negation and temporal qualifiers, and avoid inventing product names. Validate these invariants programmatically where possible. If the query is short and already precise, bypass rewriting to avoid adding latency and semantic drift.
Measure success by evidence recall and answer quality, not by how natural the rewritten string looks. Include adversarial examples with a product version, an error code, and a negated requirement. If the rewrite helps conceptual questions but harms exact-term questions, route by query class or use hybrid search rather than applying one rewrite policy universally.
Keep a trace of the original and rewritten query under appropriate privacy controls. When retrieval changes unexpectedly, that trace reveals whether the index or the rewrite caused it. Query rewriting should add context without changing the user's facts.
Keep the original query available to the final answer policy and citation trace. If a rewrite produces no useful candidates, fall back to the original or ask a clarification question according to the workflow. Measure rewrite latency and token cost as well as recall. A rewrite that improves a benchmark while increasing ambiguity or hiding the user's literal constraints is not an improvement in production.
Add tests for quoted identifiers, version ranges, negated requirements, date windows, and tenant filters. Keep the rewrite typed and validate it before sending it to search. If validation fails, use the original query or abstain rather than issuing a fluent but semantically altered request. The rewrite stage should make retrieval more useful without becoming an unreviewed source of facts.
Measure the rewrite path against a no-rewrite control with the same filters and budgets. Report latency, cost, recall, and unsupported-answer rate by query class. If rewriting helps only broad conceptual questions, route exact-term requests directly to lexical or hybrid retrieval instead of imposing semantic drift on every user.
Keep the original query in the user-facing audit trail when policy permits, while redacting sensitive content from operational logs. A rewrite should never expand a user's permission scope or invent an entity that the original did not name. Validate structured fields before search and make fallback behavior explicit so failures are safe and debuggable.
Add a no-rewrite control to every evaluation and compare the original and rewritten candidate sets under the same filters and budget. Report unsupported-answer rate, not only retrieval recall. If a rewrite helps broad conceptual questions but harms exact-term requests, route by query class or keep the original query as a parallel lexical path. The rewrite is an optimization, never the authority for facts or access scope.
Preserve identifiers, dates, and negation
Use structured rewrite output with fields for the original query, expanded terms, preserved identifiers, date constraints, negation, and filters. Validate that the rewritten query still contains exact package names, ticket numbers, quoted phrases, and version strings. A rewrite that turns “does not support” into “supports” is not a relevance improvement; it changes the task.
Compare retrieval for the original and rewritten query on a labeled set. Rewriting should recover synonyms and implied context while retaining the original evidence target. For an exact identifier, lexical retrieval may be more reliable than a paraphrase. For a short ambiguous question, ask for clarification or keep the rewrite conservative rather than inventing an entity.
Do not let a model rewrite away permission filters or tenant scope. Keep authorization and date constraints outside free-form rewriting, and merge them into the backend query through typed fields. Cache rewritten queries only with the relevant user and policy context when the rewrite contains private information.
Log rewrite decisions with redaction and a policy version. Measure whether rewriting improves recall, answer support, and latency separately by query class. A rewrite feature is healthy when it provides measurable retrieval benefit without changing identifiers, negation, scope, or the user's intended time period.