A dated log shows whether your reading is paying for itself

The question came from the CTO in a one-to-one, and it was not hostile: you spend a fair amount of time on this — what have we got out of it? The honest answer available at that moment was a feeling of being well-informed and two examples that came to mind, one of which turned out on checking to have been someone else's find. Six months of attention had produced no record of what it changed, which meant it could not be defended and could not be improved, and the only available responses were to assert its value or to quietly cut it.

The instrument that answers the question is a text file with five columns, written during an hour that is on the calendar. It costs about four minutes a week to maintain and it answers two questions that are otherwise unanswerable: whether the twelve sources from the first chapter earn their slots, and whether a long run of rejected releases reflects a bar that is still being maintained.

An hour, not a stream#

Continuous attention is expensive in a way that does not show up as hours. Reading a release note when it appears means switching context, forming a half-opinion without checking anything, and carrying that opinion into the next thing you do. Batching the same reading into one slot means it happens once, with the week's items next to each other, which is also the only arrangement in which duplicates are visible — the same claim from four sources looks like corroboration on Tuesday and like an echo on Friday when the four are stacked in one list.

The mechanics that make an hour sufficient are a strict separation between capture and evaluation. During the week, anything that looks relevant costs ten seconds: the link and one line saying why it caught your eye, appended to a file. No reading, no judgement, no opening a second tab. On Friday morning the accumulated list gets worked through in one pass, and the pass is timeboxed rather than completion-based, so a week with 40 items gets the same hour as a week with 6 and the surplus is simply dropped. Dropping the surplus is safe under the inclusion criteria, because anything that genuinely matters and comes from a primary source will still be there next week, usually with an independent run attached.

Capture has to cost almost nothing or it will not happen at the moment something appears, which is the only moment the context is free. A single append-only text file opened by a keyboard shortcut, a saved-messages thread in whatever chat tool is already open, or a shell alias that appends a line with a timestamp all work equally well; what does not work is anything that asks for a category, a tag or a title at capture time, because being asked to classify something is what makes people defer the capture and then lose it. The one line of why it caught my eye matters more than the link, since three days later the link alone regularly fails to explain itself and the item gets dropped for the wrong reason.

The hour has an order that matters. Vendor-primary items first, since deprecations and price changes have deadlines. Then anything touching a decision currently open. Then independent runs and papers, which is where the reading discipline from reading research applies. Commentary last, or not at all if the hour runs out.

Five fields, one of which does all the work#

{"date":"2026-03-06","item":"Provider X: batch API, 50% discount, 24h SLA","source":"vendor-changelog","class":"backlog","artefact":"OPS-812 — move nightly reindex to batch; est. $2.1k/mo"}
{"date":"2026-03-06","item":"Long-context retrieval paper, 128k needle results","source":"arxiv-via-simonw","class":"watch","artefact":"revisit 2026-06 if we exceed 32k prompts"}
{"date":"2026-03-13","item":"Candidate model GA","source":"vendor-release-notes","class":"decision","artefact":"slice run 2026-03-14; no — TTFT p95 5.1s vs 3.0s gate"}
{"date":"2026-03-20","item":"Structured-output failure thread, 3 independent repros","source":"simonwillison.net","class":"threshold","artefact":"added validity gate >= 99.5%/1000 calls (2026-03-30)"}
{"date":"2026-04-10","item":"Competitor migrated to candidate","source":"industry-press","class":"interesting","artefact":null}
{"date":"2026-04-17","item":"Deprecation: embeddings-v2 EOL 2026-10-01","source":"vendor-deprecations","class":"backlog","artefact":"OPS-844 — re-embed 4.1M docs, 3wk lead time"}
{"date":"2026-05-08","item":"Ops agent multi-turn complaints (internal)","source":"support-escalations","class":"backlog","artefact":"22 cases added, suite@v10"}
{"date":"2026-06-05","item":"Candidate price cut 30%","source":"vendor-changelog","class":"decision","artefact":"slice re-run; cost $0.0182/task; still no on incident vetoes"}

The class field is the whole point, and it has four values with hard meanings. A decision entry is one where something was chosen or rejected, and it names the artefact — a slice run, a verdict sentence. A backlog entry produced a ticket with an ID, so its value is auditable later by looking at whether the ticket shipped. A threshold entry changed a gate, with the date of the change. Everything else is interesting, and watch is a subtype of interesting with a review date attached, for the developments that are clearly going to matter and do not yet.

The discipline is that artefact cannot be prose. A ticket number, a gate change with a date, or null. An entry classified backlog with a description instead of a ticket ID is an entry that felt actionable and was not acted on, and allowing that one exception is how the log turns into a reading diary within a month.

What two quarters of it said#

Twenty-six review hours across 26 weeks produced 148 logged items. Eleven changed something: four became tickets that shipped, two moved a threshold, three fed cases into the suite, one triggered a slice run that ended in a rejection, and one — the embeddings deprecation — surfaced a three-week migration with five months of lead time rather than five days. An actionable rate of 7.4% sounds low and is worth putting against the cost, which is 26 hours; the deprecation entry alone was worth more than that in avoided emergency work.

The per-source view is where the log feeds back into the source list. It counts, for each of the twelve slots, how many items it produced, how many of those changed something — its hit rate — and how often it was the first place an item appeared rather than the third.

The first to surface column matters as much as the actionable count, because a source that reports things you already know from a primary feed adds latency and volume without adding information. Industry press produced 31 items over two quarters, was first on two of them, and changed nothing, which is the eviction memo written for you — and it is exactly the argument the first chapter's evicted block needs in order to make dropping a familiar feed feel like a decision rather than a risk.

The internal escalation feed having the highest hit rate is the finding that surprises people, and it recurs across teams that run this. The most decision-changing information about your models tends to come from your own production, which nobody thinks of as a source to be reviewed on a schedule until they see it in a table next to the ones they were spending their attention on.

The second question the log answers#

Three quarters of rejections, from the previous chapter, comes with five diagnostics, and three of them are queries against this log rather than against the suite.

-- were the gates ever revisited?
SELECT date, item, artefact FROM log
 WHERE class = 'threshold' AND date >= '2025-10-01';
-- 1 row: 2026-03-30, structured-output validity gate added

-- what did the rejections fail on?
SELECT artefact FROM log WHERE class = 'decision' AND artefact LIKE '%no%';
-- 5 rows: 4 cite the TTFT p95 ceiling, 1 cites incident vetoes

One threshold change in three quarters, and four of five rejections citing the same ceiling, is a specific and uncomfortable finding. It does not prove the ceiling is wrong; the chat SLO may genuinely still be three seconds. It does establish that one number has been rejecting almost everything while nobody has examined whether it still describes the product, which is precisely the shape of rot the previous chapter's counterfactual re-run is designed to test. Without the log, the same three quarters can be described as consistent application of a bar, and there is no way to check that description against anything.

The log also turns out to have a second audience, which was not the reason for keeping it. Pasting the quarter's decision, backlog and threshold rows into an engineering update takes five minutes and is a more convincing account of a team's technical currency than any narrative — eleven dated entries with ticket numbers, two gate changes, one avoided migration, and a rejection with a number attached. It also makes the review hour defensible when someone is looking for time to reclaim, and it gives other engineers a way in, because the entries are specific enough that someone can pick up a watch item and take it further. The original question from the CTO now has a two-minute answer that ends with the embeddings deprecation and the five months of lead time it bought, and the answer does not depend on anyone's recollection.

Where the log misleads#

Attribution is the weak part. The log records the proximate cause of a change, and real decisions are usually the fourth encounter with an idea rather than the first, so the source that gets credit is the one that happened to arrive when the conditions were right. That makes hit rates a reasonable guide for eviction between similar feeds and a poor guide for anything requiring precision.

The stronger warning is against optimising the actionable rate. It is a ratio, and it goes up fastest by reading only vendor changelogs, which produce tickets reliably and teach nothing about where the field is going. A source whose entire two-quarter output is watch entries may be the most valuable thing on the list, because the developments that eventually restructure how you build are visible eighteen months before they are actionable, and a log that punishes them will have removed them by then. The watch class exists to keep those entries countable without forcing them to justify themselves on a quarterly horizon, and the review dates on them are what makes the class more than a bin.

The log is also not a knowledge base. It records that an item changed something and where the change lives; it does not hold summaries or notes, and it will not help you remember what a paper said. Keeping it small is what keeps it maintained at four minutes a week, and a log that gets maintained beats a beautifully organised system that stops in April.

Everything in this topic runs on a calendar, which is the only form in which it survives a busy quarter. One hour a week for the review and the log. One monthly pass for the intake quotas and the coverage audit, an hour, usually finding one gap worth writing three cases for. One quarterly session for the gate review and the counterfactual re-run of last year's model against today's suite, half a day. On top of that, an unscheduled eleven-minute slice run whenever something ships that clears the source criteria. The place to start is not the source list, which is the enjoyable part, but the counterfactual: run whatever cases you already have against a model you stopped using, and see whether the number moves.