Parse the candidate in memory, then write by rename

The loop's own logs made the case without anyone having to argue it. Over one week, an agent running against a Django service performed 140 edit-then-verify iterations. The verify step was pytest tests/orders/, whose median wall time was 6 minutes 12 seconds. Nineteen of those iterations — a little under one in seven — came back like this:

$ pytest tests/orders/
ImportError while importing test module '/srv/app/tests/orders/test_refunds.py'.
E     File "/srv/app/orders/refunds.py", line 118
E       def _settle(refund: Refund) -> Ledger
E                                            ^
E   SyntaxError: expected ':'
!!!!!!! Interrupted: 1 error during collection !!!!!!!
=================== 1 error in 371.44s ====================

The 371 seconds is the part worth staring at. The syntax error was on the first line of a function the agent had just written, and the parser found it in the first few milliseconds of collection, the phase in which pytest imports every test module in order to discover the tests inside them, before it runs a single one; everything after that was pytest importing 94 other test modules, standing up a database, and loading fixtures, all in service of a run that could not happen. Across the week those nineteen iterations spent 1 hour 58 minutes of wall clock and roughly $6 of model tokens on retries whose outcome was determined before the process started.

The check that costs three milliseconds#

A syntax check is available in every language an agent is likely to edit, it is the cheapest check that exists, and it can run on a string. The candidate content produced by the applier is a string, so the check can happen before the filesystem is involved at all.

import ast

def parses(candidate: str, path: str) -> Result:
    try:
        ast.parse(candidate, filename=path)
        return Ok()
    except SyntaxError as exc:
        return Fail(
            check="parse",
            path=path,
            line=exc.lineno,
            column=exc.offset,
            message=exc.msg,
            excerpt=window(candidate, exc.lineno, radius=4),
        )

On the 41 kB orders/refunds.py, ast.parse takes about 3 ms. Against the 371-second alternative that is a factor of roughly 120,000, and the ratio understates the gain, because the failing test run also produced 40 kB of output that had to be summarised into the model's context before it could be acted on, while the Fail above is four fields and a nine-line excerpt.

Other ecosystems charge a little more but stay in the same order of magnitude. TypeScript's createSourceFile will return a syntax tree and a diagnostics list without a program, a type checker, or a tsconfig:

import ts from "typescript";

export function parses(candidate: string, path: string) {
  const sf = ts.createSourceFile(path, candidate, ts.ScriptTarget.Latest, true);
  // parseDiagnostics is internal but stable; it holds syntax errors only.
  const diags = (sf as unknown as { parseDiagnostics: ts.Diagnostic[] }).parseDiagnostics;
  return diags.length === 0
    ? { ok: true as const }
    : { ok: false as const, errors: diags.map(d => ({
        line: sf.getLineAndCharacterOfPosition(d.start ?? 0).line + 1,
        message: ts.flattenDiagnosticMessageText(d.messageText, " "),
      })) };
}

The tree-sitter row is the one that makes this a general policy rather than a Python trick. An agent that edits YAML, Terraform, SQL and four languages of source needs one gate, and a parser that reports error nodes for any grammar gives it one at a uniform cost of a few milliseconds per file. The gate is deliberately shallow — it answers whether the text is a well-formed program and nothing else — and shallow is what makes it universal.

The gate is worth extending past source files, because an agent's edits do not stop at source files and the non-code failures are often the expensive ones. json.loads over a 200 kB lockfile takes about 4 ms and catches the trailing comma that would otherwise fail an install step three minutes into a container build. yaml.safe_load over a deployment manifest takes about 1 ms and catches the indentation slip that turns a nested key into a sibling. Those parsers answer a narrower question than a language grammar does, since a YAML document can be well-formed and describe a service with no image, so the natural second rung for data files is a schema validation against whatever schema the repository already has — jsonschema against the package.json schema, kubeconform against the cluster's API versions — which stays in the tens of milliseconds and catches the class of error that structure alone cannot see.

There is a byte-level concern that sits alongside the parse and is easy to miss because it never produces an error. The candidate the applier produces is text, and the file on disk is bytes, and the translation between them has three settings that a naive write gets wrong. If the file used CRLF line endings and the write emits LF, a two-line edit produces a diff touching all 1,412 lines, which makes the change unreviewable and will be attributed to the agent forever afterwards in git blame. If the file ended without a trailing newline and the write adds one, or the reverse, the diff carries a spurious \ No newline at end of file marker and some linters fail. If the file begins with a UTF-8 byte order mark, dropping it can break a Windows toolchain that expects it, and adding one can break a shebang line. The fix is to record the dominant line ending, the trailing-newline state and the presence of a BOM when the file is read, and to reapply all three to the candidate before writing:

def restore_byte_conventions(candidate: str, original: bytes) -> bytes:
    text = candidate
    if original.count(b"\r\n") > original.count(b"\n") // 2:
        text = text.replace("\n", "\r\n")
    if original and not original.endswith((b"\n", b"\r\n")) and text.endswith("\n"):
        text = text.rstrip("\n")
    data = text.encode("utf-8")
    return b"\xef\xbb\xbf" + data if original.startswith(b"\xef\xbb\xbf") else data

Decoding has the same shape of problem at the other end. A repository with one latin-1 fixture file will hand the tool a UnicodeDecodeError on read, and the tempting recovery — decode with errors="replace" and carry on — silently converts every non-ASCII byte in the file into U+FFFD, so the edit lands and the file is quietly mangled from top to bottom. A read that cannot decode should fail the edit rather than approximate it.

Placing this check before the write has a consequence beyond speed. When validation runs on a string, a failure changes nothing: the file on disk is exactly what it was, no hook fired, no watcher woke up, and there is nothing to undo. When validation runs after the write, the same failure is a state to recover from, and recovery is a mechanism that can itself fail. The full ordering of checks around the write is a design question in its own right and gets its own chapter; the syntax gate is the uncontroversial floor, because a file that does not parse cannot produce a meaningful verdict from any check downstream of it.

A cancelled write can leave a file that parses#

The second failure lives on the other side of the same moment. An edit tool that opens the target path and streams content into it has, for the duration of that stream, replaced a valid module with a prefix of a new one. Most interruptions during that window leave obvious damage. One does not.

An agent run in this service was cancelled with Ctrl-C while it wrote a 918-line orders/refunds.py. The write had flushed 612 lines, and line 612 happened to be the blank line after a return statement at the end of _settle. The file that survived was a syntactically valid Python module: every block it contained was closed, every function it defined was complete. It was also missing the last 306 lines, which contained eleven functions including issue_refund, plus the module-level registration that wired refunds into the ledger.

What made this expensive is what happened next. The engineer re-ran the suite, and it passed:

$ pytest tests/orders/ -q
.........................................................    [100%]
189 passed in 344.12s

189 passed, where the previous run had reported 212. The 23 missing tests lived in modules that imported names defined below line 612, and the collection of those modules failed silently under a pytest.importorskip wrapper the repository used for optional integrations. A green suite with 23 fewer tests looks identical to a green suite unless somebody is comparing counts, and nobody compares counts.

The standard defence is older than any of this tooling. Mail transfer agents have written messages to a temporary name and renamed them into the spool for decades, and databases commit by making a rename or a link visible rather than by writing in place, for the same reason: rename(2) within a filesystem is atomic with respect to other processes, so an observer sees either the old file or the new one and never a prefix.

import os, pathlib, tempfile

def write_atomic(path: str, content: str) -> None:
    target = pathlib.Path(path)
    directory = target.parent
    mode = target.stat().st_mode if target.exists() else 0o644

    fd, tmp = tempfile.mkstemp(dir=directory, prefix=f".{target.name}.", suffix=".tmp")
    try:
        with os.fdopen(fd, "w", encoding="utf-8", newline="") as fh:
            fh.write(content)
            fh.flush()
            os.fsync(fh.fileno())
        os.chmod(tmp, mode)
        os.replace(tmp, target)          # atomic within the filesystem
        dir_fd = os.open(directory, os.O_DIRECTORY)
        try:
            os.fsync(dir_fd)             # make the rename itself durable
        finally:
            os.close(dir_fd)
    except BaseException:
        pathlib.Path(tmp).unlink(missing_ok=True)
        raise

Every line of that carries a specific failure it prevents. The temporary file goes in the target's own directory because rename is only atomic within a filesystem, and /tmp is frequently a different one — a temporary file there turns os.replace into a copy plus a delete, which is precisely the non-atomic write being avoided. The fsync on the file handle is what makes the contents durable before the rename publishes them, so a machine that loses power after the rename does not expose a file whose directory entry points at unflushed blocks. The fsync on the directory makes the rename itself survive the same event. The mode is copied because a fresh mkstemp file is 0600, and a script that silently loses its executable bit produces a failure that looks nothing like a permissions problem. The except BaseException catches KeyboardInterrupt and SystemExit, which the bare Exception most people write does not, and those are the exact signals a cancelled run delivers.

What each of these does not cover#

Parsing is a claim about grammar. The file that failed collection at line 118 would also have failed if the model had written a perfectly-formed call to a function that does not exist, and ast.parse would have accepted it without complaint, as it accepts every undefined name, every type error, every import of a module that was deleted last month. Treating a green parse as a green light is a specific and common mistake, and the next chapter is about drawing that boundary precisely.

Atomic rename gives you one file at a time. A patch that touches orders/refunds.py, orders/ledger.py and tests/orders/test_refunds.py performs three renames, and a process killed between the first and the second leaves a tree where one file is new and two are old — internally consistent nowhere, and every individual file valid. Nothing in the filesystem offers multi-file atomicity, so the honest construction is to validate every candidate in memory first, write them in one tight sequence with no model turn between them, and keep a way to put the tree back if the sequence does not finish. That second half is a real mechanism with real edge cases, and it arrives in chapter 6.

The rename also has consequences a naive in-place write does not. It replaces the inode — the filesystem's own record of a file's contents and metadata, which a directory entry merely points at — so hard links to the old file no longer see the new content, and editors watching the path see a delete-and-create rather than a modify, which some of them handle by reopening the file and some by warning the user. If the target is a symlink, os.replace replaces the link itself rather than writing through it, so a repository that symlinks a config file into a package directory will find the link quietly converted into a regular file. Extended attributes and ACLs are not carried across by the code above, which matters on macOS and on hardened Linux deployments and matters nowhere else. These are small costs against a half-written module that reports 189 tests passing, and they are worth knowing before something in the repository trips over one.

At this point an edit can be placed correctly, planned against the file that is actually on disk, and written without a window in which the file is neither one version nor the other. Every one of those is a claim about text. None of them is a claim that the program still works, and the gap between those two kinds of claim is where the next chapter lives.