Types see the paths tests miss, and miss the logic tests see

The ticket says that EU enterprise customers are billed against a negotiated contract and should not be priced from the rate card. The agent adds four lines to lookup_rate, runs the suite, and gets 412 passed in 31 seconds with line coverage steady at 88%. The patch is this:

def lookup_rate(plan_id: str) -> dict[str, Decimal] | None:
    row = db.fetch_one(
        "select unit_price, region, tier from rate_cards where plan_id = ?",
        plan_id,
    )
    if row is None:
        return None
    if row["region"] == "eu" and row["tier"] == "enterprise":
        return None          # <- the four lines the agent added
    return {"unit_price": Decimal(row["unit_price"])}

The caller, untouched by the patch, has been in the repository for two years:

def line_total(plan_id: str, units: int) -> Decimal:
    rate = lookup_rate(plan_id)
    return rate["unit_price"] * units

The suite is green because no test in it constructs an EU enterprise plan. There are eleven tests over line_total, all of them using the standard_plan fixture, which is US and mid-tier. Coverage did not fall because coverage counts lines that were executed, and the two new lines were executed — every test runs the if and takes the false branch. The metric that would have moved is branch coverage — which asks whether each conditional was taken in both directions rather than whether its lines ran at all — and the repository does not collect it; even that would have reported a gap rather than a defect. What is waiting in production is a TypeError: 'NoneType' object is not subscriptable in the invoice path, for the customers with the largest invoices.

The type checker refuses the same patch in 4 seconds:

$ mypy --output=json billing/rates.py
{"file": "billing/rates.py", "line": 34, "column": 11, "severity": "error",
 "message": "Value of type \"dict[str, Decimal] | None\" is not indexable",
 "code": "index",
 "hint": "rate may be None; the call on line 33 can return None"}

The two tools are answering different questions. The suite reports on the paths some test happened to construct, and there is no test that constructs this one; a branch nothing executes is invisible to a suite by definition, and adding a branch is exactly the operation that creates paths no existing test constructs. The checker reports on every path the code can take, because it works over the function's type rather than over any particular run of it. The moment lookup_rate was declared as returning dict[str, Decimal] | None, every subscript of its result anywhere in the repository became a claim the checker was obliged to verify, whether or not anything ever calls it.

Whether the checker actually says this depends on configuration to a degree that surprises people who have only used one setup. With strict_optional disabled, which was the default in mypy for years and is still how a good deal of legacy configuration reads, None is compatible with every type and the diagnostic above does not exist. In TypeScript the same defect is invisible without strictNullChecks, and the closely related index case — reading rates[planId] from a Record<string, Rate> that has no such key — is invisible without noUncheckedIndexedAccess, which is off by default even under strict. The check's coverage is a property of the flags, so a repository that reports a low detection rate for its type checker is often reporting a configuration rather than a limit of static analysis. Before measuring anything, it is worth knowing which of these are on, because turning one of them on can move the number more than any change to the loop.

What each one can and cannot see#

That asymmetry is not a general ranking, and the second half of it is the half that gets skipped. Here is a patch to the same file that satisfies mypy, tsc, ESLint with type-aware rules, and any other static check you care to add:

def proration_days(start: date, end: date) -> int:
    return (end - start).days

Billing wants inclusive days, so a plan changed on the first and cancelled on the first of the next month is thirty-one days rather than thirty. The types are impeccable: two date values subtract to a timedelta, .days is an int, the declared return type is int. The defect is one character. The same is true of a condition the agent inverted, a comparison against plan.updated_at where the requirement said plan.created_at — both are datetime, so both type-check — and a currency conversion applied twice. Static checks constrain the shape of the data flowing between expressions. They have no opinion about whether the number that comes out is the number the business asked for, because nothing in the program states what that number should be.

That class of defect is reachable only by executing the code against an expectation, which is what topic 13.6 was about. It is also, on most measurements, the larger class in agent-generated patches: the model is fluent enough to produce code that is well-formed and plausibly shaped, and its errors skew towards the semantic rather than the syntactic. The value of the type checker is that it removes an entire category from the loop's plate cheaply, early, and with a precise location, leaving the expensive verifier to do the work only it can do.

There is direct evidence about how large that removed category is during generation. Type-Constrained Code Generation with Language Models (PLDI 2025) applied constrained decoding, a technique that filters the model's candidate next tokens as it writes so that a forbidden continuation is never available to choose, and here the filter allowed only continuations that were well-typed. The paper reported compile errors on generated code falling by more than half, along with a corresponding rise in functional correctness. The mechanism is worth separating from the headline: the improvement in functional correctness comes from the fact that a program which does not compile is functionally wrong for free, so eliminating uncompilable candidates raises the average of what remains. The paper does not claim that types make logic correct, and neither should the loop.

Two checks, one budget#

The practical version of this appears when a TypeScript loop has two static checks available and has to decide which one runs in the inner loop — the check, patch and re-check cycle that repeats on every attempt, as against the slower pipeline that runs once before a change is merged. Say tsc --noEmit on the changed package takes about 6 seconds, and ESLint with type-aware rules — lint rules that consult the type checker's answers rather than reading the syntax alone — across the whole repository takes about 45. They overlap less than their names suggest, because ESLint does not report compile errors at all — it will happily lint a file that tsc rejects — and tsc has no opinion about the rules that make up most of a type-aware lint configuration.

The rows where tsc is alone are assignability, arity and nullability — the structural claims the compiler exists to check. The rows where the linter is alone are the ones where the code is type-correct and still means something the author did not intend: a promise that nobody waits for compiles perfectly and silently drops the write. The rows where both are silent are the ones that need a suite, and there are more of them in a real repository than in this table.

For the inner loop, the 6-second check goes in and the 45-second one does not, and the reason is only partly the wall clock. Pricing a check by the duration of a single attempt is the mistake that makes 51 seconds sound affordable; the loop runs the check on every attempt, so the honest unit is cost per attempt multiplied by attempts per task, and at a median of a bit over three attempts the difference is roughly 19 seconds against 145. The second reason is scope. tsc --noEmit on the changed package reports on code inside the blast radius of the edit, meaning the code whose meaning the change could plausibly have altered. ESLint across the repository reports on 40 packages, and the overwhelming majority of what it says is about files the patch never opened, which is a problem that survives any amount of speed improvement and is the subject of chapter 4.

The linter still runs — scoped to the changed files, on a later rung, and with its output attributed to the patch before the agent sees it. Chapter 7 puts it in a specific place. What it should not be is the thing standing between the agent and the six-second answer to "does this even compile".

What a green static run has actually established#

A patch that clears both checks has established that the program is well-formed under the type system, that the promises are awaited, that no any is leaking across a boundary in an unguarded way, and that the switch statements are exhaustive. That is a real and useful set of properties, and it is entirely a set of properties about shape.

Still entirely unaddressed is the behaviour: whether the function computes the right value, whether the condition tests the thing the requirement described, whether the boundary is inclusive, whether the rounding happens before or after the tax, whether the query filters on the tenant. Those are the defects an agent produces most often, because the model's failure mode is plausible-and-wrong rather than malformed. They are also the defects that reach production, since a program that does not compile does not ship.

There is a second, quieter gap. Static checks can be satisfied by silencing them, and the silencing operations are one token long: as any and @ts-expect-error in TypeScript, # type: ignore in Python, #[allow(...)] in Rust, an inline eslint-disable-next-line. An agent under pressure from a red check and a retry budget will find these, because they are the shortest path from a diagnostic to no diagnostic, and they appear throughout the training data in legitimate contexts. A loop that treats "no diagnostics" as the acceptance criterion has an exploit sitting in plain sight; the harness needs to read the diff for suppression tokens and treat their appearance as a failed attempt rather than a pass. This is the same shape as the rewritten assertion from 13.6, transposed onto a different verifier.

The two checks in this chapter have been discussed as if each produced a handful of diagnostics that the agent then addresses. That holds for a patch inside one function. It stops holding the moment a patch touches something shared, at which point one edit to one line produces dozens of errors across a dozen files, nearly all of them the same error wearing different filenames — and the loop that hands all of them back is the subject of the next chapter.