Build, lint and type errors as a feedback signal

Read these first

• Test execution as the verifier

What you will know

• Feed compiler and linter output back into the loop

  Preserve structured file, line, and column locations so the agent can patch the relevant code rather than consume an undifferentiated log.

• Show type errors catching what tests missed

  Use a fixture with an unexecuted path or missing test case where static checking rejects the defect before runtime.

• Measure inner-loop feedback latency and failure-detection coverage against the full build

  Track p50 and p95 runtimes separately from defects found, since a check that is rarely affordable may not improve iteration throughput.

• Build a staged validation loop that runs cheap checks first and escalates on demand

  Keep lint and targeted type checks local while reserving dependency resolution, generation, and integration builds for changes that pass earlier gates.

• Rank the signals by cost and value

  Account for diagnostic precision, false-positive rate, runtime, and whether the signal leads to an actionable patch rather than merely adding context.

Chapters

1. 1

   A diagnostic is a record with a location, not a line of a log

   When build output is piped wholesale into the model, diagnostics arrive interleaved with progress lines, recompilation notices and a linker summary, and the agent patches whichever file it saw last. Switch the loop to the compiler's machine-readable output and each diagnostic becomes a path, a line, a column and a severity the harness can route, count and attribute. Everything later in this topic depends on that structure existing.

   Read chapter
2. 2

   Types see the paths tests miss, and miss the logic tests see

   Eighty-eight percent line coverage and a fully passing run say nothing about the path an agent just added. Work through a fixture where a new branch returns None and the caller subscripts the result: the type checker rejects it because it reasons over every path the code can take, while a suite can only observe the paths some test happens to construct. The asymmetry runs the other way too — a patch that satisfies both tsc and type-aware lint can still invert a condition, be off by one, or compare the wrong field, because static checks constrain shape and not behaviour. That well-typed-but-wrong class is reachable only by executing the code, which is why the staircase later in this topic still has to climb as far as tests.

   Read chapter
3. 3

   Forty-seven errors are usually one root cause and forty-six echoes

   Rename a field on a shared type and the checker returns dozens of errors across a dozen files, nearly all of them the same edit propagating downstream. Feed them all back and the agent makes eleven speculative patches from symptoms, each generating fresh errors that look like new problems. You will learn to order diagnostics by dependency, return only the first few, and re-run — and why the model cannot do this ordering itself.

   Read chapter
4. 4

   Warnings a patch did not cause are noise the agent will still obey

   Three hundred standing lint warnings hand the loop three hundred problems the patch cannot be blamed for, and an agent given a list of problems works through it. Measured against the task at hand, that signal's false-positive rate is effectively total. The remedy is a baseline: run the check before the edit, diff the two runs, and return only what the patch introduced — an attribution the harness can compute and the model cannot.

   Read chapter
5. 5

   A detection rate is a measurement you made, not a property a check has

   Every claim of the form "this check finds 31% of defects" is a number somebody produced, and you cannot price checks until you can produce it yourself. Seed a corpus with defects that resemble the ones agents actually generate — mutation-style operators, or real defects replayed from your own history — and keep adding seeds until the rate stops moving. Then run the checks in sequence over that single corpus, so what you read off is a marginal column rather than four standalone rates: most of what the full build catches was already caught upstream. Record latency as a distribution over the same runs, because a p95 thirteen times the p50 is a diagnosis — a cold or evicted cache, no incremental state, a codegen or dependency change invalidating everything downstream — with fixes to match: warm keyed caches, a persistent checker, project references, scope pinned to the changed package.

   Read chapter
6. 6

   Inner-loop value is defects found per second of iteration time

   Lint at 2s catching 8%, a type check at 6s catching 31%, unit tests at 90s catching 44%, an integration build at 11 minutes catching 61% — the headline detection rate is the wrong column to read. Price each check as cost per attempt multiplied by attempts per task, record p50 and p95 latency separately, and ask whether a failure names a line the agent can patch or a service it cannot. A marginal gain of six points is not a rate of sixty-one.

   Read chapter
7. 7

   Cheap checks first, and a change earns its way onto the expensive rung

   Running the exact CI pipeline inside the loop buys a single definition of correct with the only currency the loop cannot spend. Build the staircase instead: lint and a type check scoped to the changed package on the first rung, with dependency resolution, code generation and integration builds reserved for changes that already cleared it. This chapter also places the expensive build outside the loop and shows how shared configuration prevents drift without paying fourteen minutes an attempt.

   Read chapter