Recency helps only where newer correlates with correct

Recency, as a ranking signal, means multiplying a document's relevance score by a factor that is larger for documents published recently and shrinks as they age. It is one of the cheapest interventions available to a retrieval team: it needs no new metadata, no ingestion work and no supersession relation, and it can be added to a scoring function in a single expression. That cheapness is why it gets reached for, usually by a team that has just watched an assistant answer from a document replaced years ago.

What it actually measures is worth separating from what it is being asked to measure. The quantity the team wants is whether a document's content still matches the world. The quantity the term computes is how long ago a file was written. Those two coincide inside a set of documents that supersede one another in an ordered line and diverge everywhere else, and the whole of this chapter is the consequence of applying a term derived from the second quantity to a corpus that contains both kinds of document.

What the multiplier actually multiplies#

The change is one expression, and every symbol in it needs naming before the behaviour follows:

score = cosine_score * (1 + 0.15 * math.exp(-age_days / 365.0))  # scoring.py

cosine_score is the relevance score the retriever already produced, age_days is how many days ago the document was published, and math.exp is the exponential function, which for a negative argument returns a number between zero and one that falls as the argument grows more negative. The whole parenthesised expression is therefore one plus a fraction of 0.15, and it multiplies the existing score rather than replacing it.

Two endpoints follow directly. For a document published today, age_days is zero, so the exponent is zero and the exponential is one, leaving a multiplier of one plus 0.15 — the boost tops out at 15% for a document published today. For a document one year old, age_days divided by 365.0 is one, so the exponent is minus one and the exponential is one over Euler's number. The boost is therefore the maximum divided by that constant, which is where the stated 5.5% at one year comes from. Those two figures are not independent measurements; the second is the first divided by e, exactly, with no residual, and checking that is how you confirm you have read the expression correctly.

The constant in the denominator is where a word rather than a digit goes wrong, and it is worth pausing on because the chapter's own replacement configuration depends on the distinction. A reader meets 365.0 next to a decaying exponential and hears "it halves every year". It does not. In an expression of this shape the constant is a mean lifetime: after one of them the term has fallen to one over e of its starting value, which is a bit under two-fifths rather than a half. The interval over which it actually halves is that constant multiplied by the natural logarithm of two, a bit under three-quarters of a year. The configuration later in this chapter is written in explicit half-lives, so the two decay laws in front of you are parameterised differently, and a reader who carries the number across from one to the other will set the replacement too slow.

The effect on an ordering follows as an inequality rather than as arithmetic, which is the more useful form because it applies to every pair rather than to one. Two documents change places when the relative advantage the newer one gains from the multiplier exceeds the relevance gap separating it from the older one. Everything else about the two documents cancels: the multiplier is applied to both scores, so what decides the ordering is the ratio of the two multipliers against the ratio of the two relevance scores. A boost cannot flip a pair separated by a gap larger than the boost, and it necessarily flips every pair separated by a smaller one.

Freshness is a proxy, and proxies have domains#

Recency is not a fact about correctness; it is a bet that documents in a corpus form ordered revision lines where the later entry replaces the earlier one. That bet is true for some document classes and false for others, and applying it corpus-wide applies it to every class at once, including the ones where it is not merely useless but inverted.

The column to read is the third, and the thing to read in it is not the strength but the sign. Four classes take a positive term, three take none, and two take a term pointing the other way. A single global multiplier cannot express that: it has one sign, and it applies it everywhere. The rows where the sign is wrong are not rows where the signal is weak — they are rows where using it makes the ordering worse than using nothing.

The middle three rows are where the global boost does its damage, and they are also the rows most likely to be numerous. A wiki accumulates meeting notes and proposals far faster than it accumulates standards, so a corpus-wide freshness term systematically transfers ranking mass from the small durable set to the large ephemeral one. That transfer follows from the two halves of one comparison rather than from an assumption. On the 60,000-page corpus, meeting_notes and draft together were 31% of documents and had a median age of 90 days, against policy and security_standard at 4% of documents with a median age of 3.2 years. A term that rises with newness therefore favours the numerous young class over the rare old one on nearly every query where both are candidates, and the size of the effect grows with the age gap, which here is measured in years against months.

Scoping the term to the classes that have revision lines#

Scoping the signal means the boost is computed against a class configuration, and a class with no revision line gets no term at all. It also means the comparison is within a document family where one exists, since the question recency is trying to answer — is there a newer version of this thing — is only meaningful between revisions of the same thing.

RECENCY = {
    # class            half_life_days, max_boost, family_scoped
    "api_reference":  (180,  0.25, True),
    "runbook":        (365,  0.20, True),
    "pricing":        (120,  0.30, True),
    "onboarding":     (540,  0.10, True),
    "security_standard": None,
    "policy":            None,
    "postmortem":        None,
    "meeting_notes":  (365, -0.20, False),   # explicit demotion
    "draft":          (365, -0.30, False),
}

def recency_factor(doc, now):
    cfg = RECENCY.get(doc.doc_class)
    if cfg is None:
        return 1.0
    half_life, max_boost, family_scoped = cfg
    if family_scoped and doc.version_state != "current":
        return 1.0            # version state already handled it; do not double-count
    decay = 0.5 ** (age_days(doc, now) / half_life)
    return 1.0 + max_boost * decay

The decay in that function is written the other way round from the shipped expression: 0.5 raised to the power of age divided by a half-life is, by construction, exactly one half after one half-life, so the constant in each tuple means what a reader expects it to mean. That is a deliberate change of parameterisation and not a change of shape — both are exponentials — and it is worth making because the numbers in a configuration file are read and edited by people who will not derive them.

Two decisions in that function are worth stating. Classes with no revision line return exactly 1.0 rather than a small positive term, because "a little bit of recency everywhere" is the global boost with a smaller constant and reproduces both regressions at reduced amplitude. And meeting_notes and draft carry negative maxima, which is the direction the evidence actually supports: a recent draft is more dangerous than an old one, since the old one has usually been superseded by something approved while the new one is still floating. That inversion is the piece teams find counterintuitive and it is the piece that fixed the classification-tiers answer.

The first of those two is where the stumble lives, and it is a belief about a defence rather than a slip. Halving the constant feels like halving the risk, and it is tempting because it appears to trade the benefit against the harm on a single dial. What it actually does is leave the multiplier constant across every document class while the property that decides whether a multiplier should be applied at all — whether the class has an ordered revision line — continues to vary completely. A quantity held fixed while the dangerous property varies is not a smaller version of the right intervention; it is the same wrong intervention at reduced amplitude, and it produces the same two regressions in a form that is harder to notice.

The dependency on doc_class is real and it is the reason this cannot be shipped on Tuesday afternoon either. Class labels have to exist on every document, and on an inherited corpus they do not. Deriving them from space, path prefix, title pattern and template ID covered 71% of the 60,000 pages at an audited accuracy of 0.93; the remaining 29% got unknown, which returns 1.0 and therefore behaves exactly as the system did before the change. Shipping a scoped boost that covers 71% of the corpus and is neutral on the rest is a strictly smaller intervention than a global boost that touches everything, which is the opposite of how the two options usually get compared in planning.

The word doing the work in that comparison is neutral. An unlabelled document is not scored wrongly; it is scored exactly as before, because the function returns one and multiplying by one changes nothing. The residual risk therefore does not live in the uncovered fraction at all — it lives in the covered fraction that was labelled wrongly, which the audited accuracy bounds without eliminating, and those errors are worth sampling by class rather than in aggregate, since a security standard mislabelled as a draft is a different event from a runbook mislabelled as a guide.

The regression that nobody reports#

A ranking change of this shape generates no user reports, and understanding why is more useful than the fix.

A user asking about classification tiers has no independent copy of the answer. The response was fluent, cited an internal document with a plausible title, and described four tiers where the truth is three — a difference that reads as detail rather than as error. The failure mode of a promoted draft is that it is 80% correct, because the draft was written by someone who understood the existing scheme and was proposing a modification to it. An answer that is wrong in the way a draft is wrong is precisely the answer least likely to trip anyone's suspicion.

The dashboards were equally quiet, for reasons that are structural rather than accidental. The evaluation set was 220 queries built nine months earlier, concentrated on the top intents — password reset, VPN access, expense submission, PTO — because that is what a query-log sample gives you, and none of those intents touch a document class where the boost changed anything. Faithfulness and citation accuracy score each answer against the passages that were retrieved for it, and the draft passage genuinely supports the four-tier answer, so both stayed at their pre-change values. The thumbs-down rate moved from 1.18% to 1.23% over six weeks, which is inside the week-to-week variance of 0.09 points. There is no single metric on the board that a ranking change of this shape can move.

Detecting it before it ships is possible and costs more than the change itself. It requires a judged query set with coverage per document class rather than per intent, which for 40 classes at ten queries each is a 400-query labelling exercise, and it requires re-labelling whenever the class taxonomy shifts. The cheaper approximation is a shadow comparison: run the new scoring alongside the old over a week of live traffic without serving it, and diff the top-1 document class distribution.

A shadow diff works because it changes what is being measured from answer quality to composition. Judging an answer requires a person to know the right answer, which is the scarce thing here. Counting which class of document the top-ranked citation came from requires nobody to know anything, because the class label is already on the document and the comparison is between two runs of the same query set. It cannot tell you whether an answer got worse; it can tell you that ranking mass moved from one class to another, which is the quantity this change was always going to move.

Recency is not staleness, and mtime is not either#

The deeper problem with the whole framing is that updated_at measures when a file was touched, which is a fact about editor behaviour rather than about the content. The 2019 onboarding guide from the previous chapters has a 2024 modification date because somebody fixed a link. A document that has been correct and untouched for six years and a document that has been wrong and untouched for six years have identical timestamps. Recency is a signal about revision events, and staleness is a claim about whether the content still matches the world, and no timestamp in a wiki distinguishes them.

Those two documents are the cleanest available statement of the limit. Every observable a timestamp exposes is identical across the pair, and the property that matters differs completely. No function of timestamps alone can separate them, however it is scoped, because the inputs do not contain the distinction. Scoping the term by class narrows the population in which the confusion does damage; it does not make a timestamp into a statement about content.

What does distinguish them is review state — a reviewed_at set by an owner attesting that the content is current, and an approval_state separating draft from approved from retired. SEC-2018-03 with a reviewed_at of 2026-02-11 is fresher, in the only sense that matters, than a 2024 proposal with approval_state: draft, and a scoring term built on those two fields produces the correct ordering without any class configuration at all. Both fields work because they record a human decision about the content rather than an event in a file system, which is the property updated_at lacks.

The honest limitation is that almost no corpus has them. Confluence does not model approval; SharePoint does, inconsistently; a Git-based docs repo can carry both in frontmatter and usually carries neither. Where the fields exist, use them and delete most of the class configuration above. Where they do not, the class-scoped term is the approximation, and the thing worth lobbying for is not a better ranking function but a reviewed_at on the four hundred documents that matter.

All of which leaves the original motivation intact and unexplained. The team that shipped the eleven-character change was not careless; they were looking at a dashboard that said the system was healthy while users were being handed last year's policies, and reached for the only lever that did not require an ingestion project. The dashboard is the part that needs accounting for, because every number on it was accurate.