Blame suppresses the near-misses that are worth the most

In the April review of FM-2025-014, a director asks how a schema change shipped without a backfill, and somebody answers that it was MR-4471, which was Priya's. The sentence is accurate and takes four seconds. Priya is in the room; she says the migration was reviewed and approved by two people, which is also true, and the conversation moves on to the corrective actions. Nothing dramatic happens, nobody is disciplined, and by every account of the meeting it was a normal review.

The register tells a different story about the following months. Near-miss entries — write-ups of something that could have caused an incident and did not, filed voluntarily by whoever spotted it — had been running at eleven a month across the platform teams. In May there were six, in June three, and in July two. Nothing else about the process changed; the template was the same, the tooling was the same, and the sweep job kept running. What changed was that everyone in that room learned that a write-up can end with a name and a merge request number, and the cheapest way to avoid being in one is to fix the thing quietly.

The cost of that shows up in September. In July an engineer noticed that the archival pipeline had the same delete-by-filter pattern as the ingestion pipeline, with the same silent no-op when the filter field was absent, and fixed it in his own service without filing anything, because filing would have meant explaining why it had been that way for a year. Nobody else learned that the pattern was general. In September the same construct in the summarisation ingest path served superseded documents to four tenants for nine days, producing about 1,100 conversations with stale context before a customer noticed. A near-miss entry in July would have cost forty minutes to write and would have produced a single query across the codebase for the pattern.

Near-misses carry the mechanism without the harm#

The value of a near-miss is that it contains everything the register needs — the mechanism, the components involved, the conditions that would trigger it — and none of the cost. It arrives before the customer conversation, before the scope replay, before the twenty-day detection interval, and the engineer reporting it usually understands the mechanism better than a post-incident investigator will, because they found it by reading the code rather than by working backwards from a symptom. Near-misses are also more numerous by a wide margin; in this register they outnumbered incidents by roughly seven to one over the year, so a register that captures only incidents is working from an eighth of the available information.

Aviation worked this out through several decades of trying the other approach. The Aviation Safety Reporting System, run by NASA since 1976 rather than by the regulator, exists because pilots would not report to the body that could revoke their licences, and it offers confidentiality and limited immunity from enforcement action in exchange for reports. The design principle underneath it transfers directly: the organisation that most needs to hear about a near-miss has to be visibly unable to punish the person reporting it. What does not transfer is the immunity, since a software team cannot promise anything about employment consequences, so the available lever is narrower — what the write-up says, who reads it, and whether anyone has ever been embarrassed by one.

A blameless post-mortem, which is one whose write-up and review are conducted under a standing rule that no statement names a person as the cause, does not mean nobody is accountable, and the distinction is worth being precise about because it is where the practice gets discredited. Accountability in a post-mortem attaches forward, to corrective actions with named owners and dates, which is exactly what chapter 5's register enforces. What is dropped is the backward-looking judgement about whose fault the past was, on the grounds that it is expensive to establish, almost never actionable, and reliably produces a system where information stops flowing upward. Repeated negligence is a management problem with its own process, and running it through the incident review contaminates the one channel that depends on people volunteering things.

Making near-miss reporting cheap matters as much as making it safe, because the competing option is always to fix the thing and say nothing. A near-miss entry in this register is the same row as an incident with status: near_miss, no scope numbers, no timeline and no review meeting, and the expected effort is under an hour. Two practices keep the flow up. Reports are filed by whoever noticed, including when what they noticed is their own mistake, and the monthly engineering note names two or three of them by mechanism with what they prevented, so the visible consequence of filing is that your finding gets used rather than that your judgement gets discussed. Neither is expensive; both are easy to stop doing during a bad quarter, which is exactly when the reporting rate is worth the most.

The review meeting is where blame usually enters, and it responds well to structure. Establishing the timeline and the mechanisms before anyone discusses judgement keeps the discussion on conditions while the facts are still being assembled, and having the session facilitated by someone outside the owning team removes the incentive to arrive with a defence prepared. One useful convention is to treat "should have" as a signal to stop and rephrase: whenever a sentence in the room contains it, the missing content is a control that did not exist, and asking what would have had to be true for the correct action to be the easy one usually produces it within a minute.

The same fact, written two ways#

Each right-hand statement contains the left-hand one's information and points at something buildable. The rewriting is not a euphemism exercise; it is the same discipline as chapter 2, applied to the part of the causal chain that happens to involve a person, and the test is the same — does the statement name a condition that can be changed by a control, or does it name a person who can be asked to be more careful.

What a stakeholder needs in order to decide#

The account team facing Calderwood, and the executive who has to approve or refuse a rollback, need something different from the engineering write-up. Their decisions are whether to roll back, whether to add a guardrail that costs product quality, whether to notify affected users, and whether to accept the residual risk, meaning the exposure that is still there once the fixes that have actually shipped are counted. Vague assurance blocks all four. "We've fixed the issue and improved our monitoring" gives an account manager nothing to say to a customer who wants to know whether any other answer their employees received was wrong, and gives an executive no basis for deciding between notifying twelve tenants and notifying three.

## Calderwood: stale policy answers (FM-2025-014)

**What happened.** From 13 Feb to 5 Mar, the assistant could retrieve the
2024 benefits handbook alongside the 2025 one and answer from the older
version. Cause: a re-index removed no superseded content, because the
delete filter did not match content written before we recorded versions.

**Impact, measured.** 63 conversations across 12 tenants had superseded
content in context; 19 answers contradict current policy on a figure or
a date. Calderwood: 7 conversations, 3 contradicting answers, all on
parental leave. Conversation ids and dates attached.

**Corrective actions.**
- Removed 1,910 superseded chunks across 12 tenants — done 5 Mar
- Ingestion fails when a corpus change deletes nothing — shipped 6 Mar
- Per-tenant staleness alert, fires within ~5 h — shipped 11 Mar
- Version-aware retrieval ranking — owner R. Okafor, due 30 Apr

**Residual risk.** Documents ingested before Nov 2024 that were never
re-indexed are not covered by the new assertion; a sweep is running and
completes 18 Mar. Until then, the alert is the control, not prevention.

**What we cannot tell you.** Whether any employee acted on a wrong
answer. That happens outside our systems; the 19 conversations and
their timestamps are attached so you can check against leave requests.

The last two sections are what make this brief usable, and they are the ones most often cut. Naming the residual risk lets the customer decide whether to do their own check in the interim, and naming what you cannot determine stops the reader from assuming a number that does not exist. The section is also the honest boundary of the post-mortem, since impact on the outside world is nearly always partly unmeasurable from inside the system.

When your own evaluators disagree and the customer is certain#

A harder version arrives when the incident is not a figure that can be checked against a document. Suppose two reviewers read the same answer and one calls it wrong while the other calls it acceptable, and the customer is certain it was wrong. The tempting resolution is to side with the customer, log it as a defect, and add the case to the eval set with the customer's expected answer as ground truth. That closes the ticket and does two bad things: it treats a disagreement between trained reviewers as noise, and it writes one customer's preference into a shared eval set, where it will fail future outputs that are correct for everyone else.

What the post-mortem records is the disagreement itself, as a finding with its own cause, because two competent reviewers reaching opposite verdicts against the same rubric means the rubric does not decide this case. That is a defect in the specification of correct behaviour, and it is usually the more valuable finding, since an underspecified definition produces disagreement across every future incident in the same area and quietly corrupts the eval scores in between.

id: FM-2025-021
title: Reviewer disagreement on partial-eligibility answers
customer_position: >
  Answer was wrong; employee was told to "check with HR" when the
  handbook states the eligibility rule directly.
reviewer_a: defect - handbook states the rule; deflecting is a failure
reviewer_b: acceptable - eligibility depends on tenure at leave start,
  which the assistant cannot know from the conversation
rubric_clause: "Answers must be grounded in the tenant's current policy"
gap: >
  Clause does not say whether a conditional rule must be stated with its
  conditions or may be deferred to a human when an input is unknown.
customer_impact: real and independent of the classification -
  employee delayed a leave request by 6 days
actions:
  - amend rubric: conditional rules must be stated with conditions and
    the unknown input named; deferral alone is a defect - owner S. Ihejirika
  - reclassify this case and re-score the 40 prior eligibility cases
    under the amended clause - due with the next eval run
  - add eval case only after the rubric lands, with expected behaviour
    derived from the amended clause rather than from this customer

Two things stay separate in that record. The customer's experience is a fact, and it is written down as harm with its measured consequence regardless of how the classification resolves, so nothing here is a way of telling a customer they were not really harmed. The classification of the output is a question about your own standard, and it stays open until the standard says something. Where the amended rubric ends up agreeing with the customer, the case joins the eval set with an expectation derived from the rubric, which means it now generalises to every tenant instead of encoding one.

Where this stops working#

None of this survives a context where the write-up is discoverable in litigation or demanded by a regulator, and pretending otherwise is unfair to the people asked to write candidly. Teams under that exposure usually keep a factual, narrow incident record for the outside world and run the learning conversation separately, and the register in that setup holds mechanisms, checks and owners rather than narrative. It is a real cost, and the failure mode is a register that becomes so careful it stops containing anything.

The deeper limit is that everything in these six chapters depends on somebody choosing to write. The trace retention from chapter 1, the mechanical causes from chapter 2, the detection interval from chapter 3, the regression case from chapter 4 and the register from chapter 5 are all downstream of a person deciding that a thing that went wrong is worth an afternoon. The next incident is the test of that: pull the trace before the window closes, write the causes as mechanisms with their evidence, put the time-to-notice at the top with the reason no monitor fired, turn the finding into a case with variants, and file it with an owner and a check that runs.