A fact your system of record already holds does not belong in memory

Some facts about a customer are already held, correctly and currently, by a service you can call. The subscription tier is a column in a billing system. Open tickets are rows in a ticketing system. The connected data warehouse is a field on an integration record. Each of those is something a user might mention in conversation, which means each of them is something a write gate will happily propose as a memory candidate, having passed every test the previous chapters set: true, durable, user-stated, in scope.

This chapter is about why passing those tests is not enough, and the reason is a property none of them examines. A stored record's relationship to the world is fixed at the moment it is written. The world's relationship to itself is not. Where somebody else is already tracking the change, copying the value is taking on an obligation that the original owner is discharging for you anyway.

The decision runs on ownership first, latency second#

The general rule answers the question of when re-retrieval is simpler than memory: whenever the fact lives in a system of record you can query at read time, a system of record being the service that owns a fact and is authoritative for it, so that when it disagrees with any copy the copy is what is wrong. The write policy's job is to notice that before the record exists, not to manage the staleness afterwards.

It is worth being exact about why a copy cannot be rescued by tuning, because every team tries. Suppose you shorten the expiry on plan records. The expiry is a number you chose at write time and it does not move; when the source changes is a fact about the customer and it varies without limit. So the two are independent, and no setting of the first bounds the error introduced by the second — it only bounds how long the error lasts, at the price of re-eliciting the fact that often. A nightly refresh job, or a webhook that invalidates on plan change, does better, and both are work whose entire purpose is to make a copy behave like the thing it was copied from. At the end of that work you have re-implemented a lookup, with more moving parts and a window in which it is wrong.

The first branch does most of the filtering, and running it over a support agent's realistic candidate list is sobering.

Read that table for what varies between the rows that get written and the rows that do not, because the answer is only one column. The candidates are not separated by durability: a subscription tier and a preferred name are both durable. They are not separated by how often they come up, since the plan tier comes up far more often than the naming preference. They are not separated by how confidently the user stated them. The only column that moves across the boundary is Owner, and it moves from a named service to the word "Nobody". Everything the earlier chapters taught you to check is held constant across this table, which is why a gate built on those checks admits every row.

One row of that table takes a third path and it is worth separating from the other two. What the product does on plan X is owned by a docs corpus, which is an owner in the sense that somebody maintains it, and it is not a service you call with an identifier to get one value back. So the verdict is retrieval rather than lookup: you search a body of text and read the answer out of passages. That distinction matters because retrieval and lookup fail differently — a lookup returns the wrong value or nothing, and a retrieval returns something plausible from a neighbouring topic — and because a docs corpus is versioned by publication rather than by row, so the freshness question is about which version you indexed rather than about when you copied a field.

The no path out of the latency question deserves its own note, because it is where the confusion between memory and caching usually gets resolved in the wrong direction. A fact that has an owner but is too slow to fetch on every turn is a caching problem, and it should be solved with the machinery caches have: a TTL, meaning a time to live after which the copy is discarded and fetched again, derived from how fast the source changes, an invalidation hook on the source's change events, and a key that identifies the source row rather than a sentence about it. What it should not become is a memory record, because a memory record has no refresh path, no invalidation hook and no relationship to the source at all beyond the moment it was copied.

The two look similar enough that the comparison is worth stating as a comparison and then broken. Both hold a value, both carry a clock, both are read instead of the source. Where it breaks is on recovery: a cache entry can always be thrown away safely, because the source can rebuild it, and a memory record cannot, because there is nothing behind it. That single difference is what makes a TTL a performance knob in one case and a data-loss policy in the other.

Auditing an existing store against this branch is a short exercise with an uncomfortable result. Take the fact records, and for each one ask whether its claim corresponds to a field in a system you can query. Arbor's audit over 4,100 sampled fact records found 44% of them were copies of data owned elsewhere — plan, seats, region, contacts, ticket history — which is a little under two thousand records in the sample, and that subset accounted for three of the four memory-related incidents in the previous six months. Fewer than half the records produced three quarters of the incidents, which is the clearest available statement that the class is not merely useless but disproportionately dangerous.

Searching what you already keep#

The second branch is the one people skip, and it is where the strongest version of the argument lives. Past conversations are already stored — Arbor keeps 18 months of transcripts for support and compliance reasons that have nothing to do with the agent — and searching them at read time is retrieval, with retrieval's properties.

Those properties follow from one fact about a transcript, which is that it is a record of what was said rather than a claim about what is true. Because it is a record of an event, it is immutable, so it never needs invalidating; nothing that happens later can make it false that the sentence was uttered. Because it carries no interpretation, interpretation happens per request against the current question, so a sentence that meant one thing in February can be read correctly in April. And because nothing was derived from it, there is no copy for a deletion request to chase — deleting the conversation deletes the evidence.

TOOLS = [
    {
        "name": "get_account_facts",
        "description": (
            "Current billing and account state for the customer. Authoritative. "
            "Always prefer this over anything you believe about their plan or seats."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"account_id": {"type": "string"}},
            "required": ["account_id"],
        },
    },
    {
        "name": "search_past_sessions",
        "description": (
            "Full-text and semantic search over this user's previous conversations. "
            "Use when the user refers to something established earlier."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {"type": "string"},
                "months_back": {"type": "integer", "default": 18},
            },
            "required": ["query"],
        },
    },
]

{"tool": "get_account_facts", "result": {
  "plan": "starter", "plan_since": "2026-04-02", "seats": 12,
  "features": ["exports_manual", "api_read"], "sso": false}}

That response is right on 9 April and right on 9 May, with no expiry, no sweeper, no read log entry to chase later and no deletion obligation. It costs 40 ms and about 90 tokens.

Making the case that a workload needs no memory at all#

Put those pieces together for Arbor's support agent and the position that follows is that the product should have no persistent memory. Every fact that changes what the agent does — whether a feature is available, whether a refund is authorised, who the billing contact is, which region to name in a runbook — has an owner system that is more current than any record could be, and the failures in chapters 5, 6 and 7 were all facts of this kind being copied. The SLA belonged to contracts, the SSO entitlement to billing, the region to the customer's connected integration, and the refund authorisation to nobody at all, which is why believing it was so dangerous.

The facts with no owner are mostly presentation preferences, and those are cheap on both sides of the ledger: cheap to re-elicit, because a user who wants shorter answers says so in three words, and cheap to be wrong about, because the cost of a verbose answer is a verbose answer. Spending an invalidation, provenance, repair and deletion regime on them is a poor trade. Anything else the user established in an earlier conversation is in the transcripts, which are kept anyway, and searching them costs 120 ms and roughly 1,800 tokens of context for the retrieved passages.

Set that against memory's numbers — an 18 ms read and 240 tokens — and the comparison shows exactly what a store buys. It is about a tenth of a second and rather more than a thousand tokens per turn, and it is nothing at all on the correctness side, since both arms are reading the same underlying material. On a turn budget of 2.5 seconds, 120 ms is affordable, being under a twentieth of the time available. The honest form of the argument is therefore that for this workload memory is an optimisation with a liability attached, and the liability is measured in the eleven hours from chapter 6 rather than in dollars of storage.

It is worth being explicit that this is a conclusion about a workload rather than about memory. A support agent talks to people about an account whose state is held in four systems the company already runs, which is close to the best possible case for lookups and close to the worst for a store. Move the same argument to a product with no billing system, no ticketing system and no CRM — a coding assistant working inside somebody's repository, say — and the first branch of the flowchart stops filtering anything, because there is no service to call about how this team likes its tests written.

This is also the answer to give when someone says memory is fine as long as you evaluate it. Evaluation is what settles the question at the end; it is not a substitute for having a position going in, and a team that cannot say which fact classes it intends to store and why those classes have no owner has not designed a write policy, it has enabled a feature.

The workload that defeats the argument#

The argument has a boundary, and it is sharp. It fails for facts with three properties at once: no system of record, learned from the user, and expensive to re-elicit or to find again. All three are needed. Drop the first and a lookup wins; drop the second and a document holds it; drop the third and re-asking is cheaper than storing.

Test the conjunction on the plan tier and it fails on the first property, since billing owns it. Test it on what the product does on a given plan and it fails on the second, since the docs hold it and the user never established it. Test it on a request for shorter answers and it fails on the third, since the user restates it in three words. Only a candidate that survives all three is left, and running the three tests in that order is cheap because the first is a schema question that needs no judgement at all.

A data analyst working with Arbor over months establishes that in her company's warehouse rev_net excludes refunds but includes chargebacks, that the orders_v2 table is authoritative while orders is a deprecated view nobody has dropped, and that fiscal quarters start in February. None of this is in any system Arbor can query; it is in her head and in the corrections she has made to the assistant's SQL. It is durable, since a warehouse convention outlives most employees. And it is expensive to recover: reconstructing those three facts took a measured 6 to 8 turns and about nine minutes when Arbor tested a memory-off variant with the same analyst, and every subsequent session pays it again.

What makes this class different from a plan tier is that the fact does not exist anywhere until somebody writes it down, and the person best placed to write it down is the system that watched it being established. There is no service to call, no document to retrieve, and no owner who will keep it current, which also means the invalidation problem is genuinely different in kind: orders_v2 stops being authoritative only when the analyst says so, and she is the same person who will be in the conversation when it happens. The maintenance path and the source are the same human, which is the situation memory is actually for.

Transcript search is the part of the counter-argument that fails hardest here, for a reason worth understanding. The convention was never stated in a single quotable sentence. It was established over about forty turns of the assistant getting a number wrong and the analyst saying "no, that's including refunds" three times in different contexts. A semantic search for "rev_net definition" returns passages that argue about a specific query on a specific day, and the model reading them has to redo the synthesis every time, at a cost of 4,000 to 6,000 tokens and a nontrivial chance of getting it wrong. Compare that with the extraction — one sentence, written once, retrieved for 240 tokens — and the ratio is better than an order of magnitude in tokens alone, before counting the chance of a wrong synthesis. That is precisely the value no read-time mechanism provides, because the synthesis is the thing being stored and the transcript never contained it.

Tenure sharpens this, and it is the property a short evaluation is least able to see. Searching 40 sessions for a fact established in session 3 works acceptably. Searching 2,400 sessions accumulated over eighteen months works much less well, because the relevant passage competes with hundreds of superficially similar ones and the retrieval budget does not grow with the archive. A memory record is roughly constant-cost to retrieve regardless of how long the relationship has run, so the two curves move in opposite directions: the case for memory strengthens with tenure while the case for transcript search weakens. A three-week evaluation will therefore understate memory's value for a two-year customer, and it will do so systematically rather than noisily.

Re-retrieval has its own failure modes, and pretending otherwise makes the argument brittle. Read-time lookups put your agent's availability behind the billing service's availability, so an outage that would have been invisible becomes an agent that cannot answer basic questions. Several lookups on the same turn serialise into real latency, and an agent that needs plan, tickets and integrations is 200 ms in before it starts thinking. Permissioning is harder than it looks when the person asking is not the person the fact is about. None of these overturn the ownership rule, because none of them is about freshness; they mean the alternative is engineering work rather than a free lunch.

The rule and its exception can be stated together in one line, and the pairing is what makes it usable rather than dogmatic. Where a fact has an owner, the owner is fresher than any copy on every turn after the first, so the copy earns nothing and costs a maintenance regime. Where a fact has no owner and was expensive to establish, the store is the owner, and everything the earlier chapters described — the expiry, the provenance, the repair path — is what owning it responsibly costs.

Everything in this chapter is an argument from properties: ownership, freshness, cost of re-elicitation. It is a good argument and it is still an argument, made in advance, about a system nobody has measured. Whether a write policy actually earns its keep on a specific workload is an empirical question with a specific experiment attached, and running that experiment — memory on against a re-retrieval baseline, on held-out tasks, with costs in the same table — is the last chapter.