Pinning constraints and commitments

A compactor — the pass that replaces a span of transcript with a shorter rendering of it — has to sort what it is given into two categories that behave completely differently. Most of a transcript is a record of what happened: files were read, a hypothesis was tested and discarded, the user asked a clarifying question and got an answer. Those items are past events, they are useful mainly as context, and paraphrasing them loses detail without changing what the agent is allowed to do next. A smaller set of items is not a record of anything. It is a standing condition on every turn that follows, and it sits in the transcript only because that is where the user happened to type it.

The distinction matters because summarisers cannot see it. A general-purpose summariser compresses by salience, and salience is measured by how much of the conversation an item generated. An instruction that the agent complied with perfectly generates nothing at all: no argument, no correction, no follow-up question, no second mention. Perfect compliance makes a constraint invisible to anything that summarises by salience, which is every general-purpose summariser, and it means the items most likely to be dropped are the ones the agent has been obeying most reliably.

Following that one step at a time gives a conclusion worth sitting with. Compression works by selection: a summariser has more input than output and must decide what to carry, and the only evidence it has for deciding is the transcript in front of it. The evidence that an item mattered is that the conversation returned to it — an argument, a correction, a clarifying question, a second mention. A constraint the agent obeyed produces none of that, because there was nothing to correct and no reason to raise it again. So the transcript contains no signal that the item is important, and a selection made on the available signal drops it. The inversion at the end of that chain is the uncomfortable part: the better the agent is at following an instruction, the less evidence the transcript holds that the instruction exists, and the more likely a compaction is to remove it. Compliance erases its own record.

The question that separates the two kinds of item#

The test is whether a paraphrase changes what the next turn is permitted or required to do. "The user asked about indexing strategy around turn 20" and "the user raised questions about database performance early on" are the same instruction to the model, which is none, so either rendering is acceptable. "Never modify anything under /migrations" and "the user gave guidance about the deployment process" are not the same instruction, because only one of them stops the agent from writing the file. Anything that fails that test is retained as the literal text the user or the system wrote, and everything else is available for compression.

Active tool state is the class that behaves least like the others, because its failure is mechanical rather than semantic. Topic 2.14, on compaction with tool calling, establishes the rule: a tool_use block and the tool_result that answers it reference each other by id, and a cut that lands between them produces a request the API rejects before the model sees a token of it.

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "messages.4: `tool_use` ids were found without `tool_result` blocks immediately after: toolu_01F9wKq2. Each `tool_use` block must have a corresponding `tool_result` block in the next message."
  }
}

That error is the good case, because it is loud, immediate, and impossible to ship past. The bad case is the tool state that lives outside the message list: the shell tool's working directory, the branch the agent checked out at turn 12, the id of an uploaded file, the cursor of a paginated API the agent is halfway through. None of that is in the transcript in a form a summariser can see, and all of it is a precondition for the next tool call being correct. An agent whose summary says "the agent explored the repository and made some changes" has lost the fact that it is on fix/orders-idx and will happily commit to whatever branch the shell happens to be on.

The line dividing the loud case from the quiet one is worth naming, because it predicts which of your own tool state will bite you. A tool_use block and its tool_result are both messages, so both are inside the thing the compactor edits, and a cut between them leaves a request that violates a rule the API checks before running anything. The check is possible precisely because both halves are in the request. The working directory of a shell is not in the request. It lives in a process on the other side of the tool boundary, and the transcript holds at most an echo of it — a cd several turns back, a path in some output. Nothing validates it, because nothing in the request represents it. The rule that falls out is that state which exists only outside the message list has to be written into the ledger explicitly, since it is the one category where there is no error to catch and no invariant to check. The agent will act on a stale working directory with the same confidence it acts on a correct one.

Deciding at arrival instead of at the threshold#

The structural weakness in classifying at compaction time is one of timing. The decision about a one-sentence instruction gets made by a model that has fifty or more turns to read, a token budget to hit, and an instruction to produce a readable account. Made when the turn arrives, the same decision is available to a model that has one sentence to read and one question to answer, which is a far easier problem and a far cheaper one. Running the classification as each turn arrives turns compaction from a judgement call into bookkeeping: every turn is examined once, anything that binds the future is written to a ledger, and the compactor is then only ever allowed to touch the narrative pool.

The tempting alternative is to keep the decision at compaction time and fix it with a better prompt — adding "be sure to preserve any constraints, instructions or open questions" to the summariser instruction. That helps, and it is worth doing, and it does not solve the problem, because it still asks one call to do two jobs whose objectives conflict. The summariser is compressing 46,000 tokens into 800 while also being asked to notice that one sentence forty-eight turns back must survive intact, and every token it spends quoting that sentence is a token it cannot spend on the account it is being graded on. Measured over a replay of thirty long coding sessions, a summariser with that added instruction carried a constraint of this kind through the first compaction 91% of the time and through three consecutive compactions 68% of the time, which is a reasonable score for a summariser and an unacceptable one for a rule about production data.

The conflict has a size, and putting one on it shows why a prompt cannot resolve it. Turning 46,000 tokens into 800 is a compression of close to sixty to one. At that ratio the share of the output available to any one item is a few words, and reproducing a sentence intact spends far more than a few words. The instruction to preserve constraints is therefore not competing with laziness or with a careless model. It is competing with the compression target, and the compression target is the reason the call is being made at all.

The relationship between the two survival figures says more than either alone. If each compaction were an independent trial at the first-pass rate, three in a row would leave a bit over three quarters of the constraints standing — nine in ten, three times over. The measured figure is lower than that, so survival is not independent from pass to pass, and the direction of the discrepancy is the informative part. A constraint that has already been through one compaction is more likely to be dropped by the next, because what the second pass reads is not the user's sentence but the first pass's rendering of it, already shorter and already less emphatic. Each pass makes the item a slightly easier thing to leave out. A defence whose success rate per attempt declines with use is not one you can strengthen by giving it more attempts.

The classifier itself does not need to be clever, and the cheap version catches most of what matters. A rule pass over user turns flags imperatives and prohibitions — never, don't, always, make sure, from now on, only ever — along with any sentence containing a path, a branch name, or a currency amount, and escalates the flagged turns to a small model that answers one question with a schema.

CLASSIFY = """Here is one message from a conversation with a coding agent.
Does it state something that constrains or obliges the agent on FUTURE turns,
as opposed to describing something that happened?
Answer with JSON: {"binding": bool, "class": "constraint"|"commitment"|
"open_question"|null, "text": "<the exact substring that binds, verbatim>"}"""

def classify_turn(turn: Message) -> PinnedItem | None:
    if not TRIGGER_RE.search(turn.text):
        return None
    r = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=256,
        system=CLASSIFY,
        messages=[{"role": "user", "content": turn.text}],
    )
    parsed = json.loads(r.content[0].text)
    if not parsed["binding"]:
        return None
    return PinnedItem(
        turn=turn.index,
        cls=parsed["class"],
        text=parsed["text"],      # verbatim substring, never a paraphrase
        retire_when=None,         # constraints do not retire
    )

The text field is required to be a substring of the original message, which is checkable in one line and worth checking, because a classifier that is allowed to write its own version of the constraint has reintroduced exactly the failure the ledger exists to prevent. On a sixty-three-turn session the rule pass flags perhaps eight turns, the model calls cost around $0.0003 each and run off the critical path while the main turn is executing, and the resulting ledger is four items totalling 180 tokens.

Two design choices in that paragraph are doing the load-bearing work, and both are about narrowing rather than about cleverness. The first is the split between a regular expression and a model. The regular expression is allowed to be wrong in one direction only: it may flag turns that turn out not to bind anything, because those cost one cheap call and a binding: false, and it must not miss turns that do, which is why the trigger list is broad and includes anything with a path or a currency amount in it. Out of sixty-three turns it passes perhaps eight to the model, so the expensive judgement runs on about one turn in eight and the cheap filter runs on all of them. The second is that the model's output is constrained to a substring of the input. That turns a claim into something checkable in one line — the returned text either appears in the original message or it does not — and a classifier permitted to write its own wording would be a paraphraser, which is the component this whole ledger exists to keep away from these items.

Where the ledger sits in the assembled request is a smaller decision but not a free one. Pinned items go after the summary and immediately before the recent turns, so they are the last thing the model reads before the current request, and they are labelled as standing conditions rather than mixed into the narrative. Putting them at the top, adjacent to the system block — the standing instructions sent above the conversation on every request, which no compactor touches — reads more logically and performs slightly worse on long contexts, for the ordinary reason that the middle of a long context gets less attention than either end. By the time the ledger is 4,000 tokens into a 150,000-token request, the top is the middle.

That last sentence is compressed and worth unfolding, because "the top is the middle" is a claim about position that sounds like a paradox. Attention is not strongest at a fixed token offset; it is strongest at the boundaries of whatever is in the request. On a short prompt, a block placed after the system instructions is a few hundred tokens from the start and genuinely near the beginning. On a request of 150,000 tokens, a block sitting after a system block of a few thousand tokens is no longer adjacent to anything — the start is behind it, the end is a hundred and forty thousand tokens ahead, and it is surrounded on both sides by summary and transcript. Its offset has barely changed; its position relative to the request has changed completely. Placing the ledger immediately before the recent turns puts it near a boundary that stays a boundary however long the session runs, which is the property being bought.

None of this is memory#

A pinned ledger is not a memory system, and the difference matters when the session ends. Everything in the ledger lives in the request; when the process restarts or the user opens a new conversation, it is gone. Constraints that should survive that — a team convention, a permanently forbidden directory — belong in the system block or in an external store the agent reads at the start of a session, and the API's memory tool, which lets the model write and read files that outlive the conversation, exists for exactly that handoff. The ledger's job is narrower: to make sure that within one long-running session, something stated at turn 3 is still literally present at turn 300.

Pinning is also not free, and treating it as free produces a different failure at a different scale. A ledger that only ever grows reaches 4,000 tokens by turn 200 on a busy session, at which point it is competing for the same budget the compaction was supposed to recover, and it is also crowded enough that the constraints stop standing out from each other. Retirement rules are part of the design: a commitment retires when it is fulfilled, an open question retires when it is answered, active tool state retires when the tool call completes, and safety constraints and system instructions never retire. Anything without a retirement condition should have to justify itself, since the alternative is a ledger where the important four items are buried among sixty stale ones.

The scale of that is easy to check against the figures already given. Four items came to 180 tokens on a sixty-three-turn session, which is a little over forty tokens an item, and a busy session accumulates items at a rate the classifier does not moderate — it is a filter on what binds, not a quota. Run that forwards without retirement and a few hundred turns produce a ledger in the thousands of tokens, which is the same order as the material a compaction was called to recover. The token cost is the lesser half of it. A ledger of four items is read as four standing conditions; a ledger of sixty is read as a list, and a list is something a reader skims. The reason retirement is part of the design rather than housekeeping is that the ledger's protection comes from being short, and nothing about pinning an item preserves that property on its own.

The last thing worth being clear about is what a summary implies even when it omits something. A summary presents itself as an account of the compacted span, and an agent reading it has no way to distinguish "this did not happen" from "this happened and was not recorded". That is the false continuity that makes omission dangerous rather than merely lossy: the agent does not experience a gap, it experiences a complete-looking history in which there is no /migrations rule.

The mechanism is a matter of what the two representations imply. A transcript that begins at turn 55 is visibly a fragment: the numbering starts partway through, the first message answers something that is not there, and an agent reading it has grounds to be uncertain about what came before. A summary begins at the beginning and ends at the end, so it carries no marker of its own incompleteness — the compression is invisible in the artefact, which is the quality that makes it readable. An agent asked whether it may modify /migrations therefore consults a history that appears complete, finds no prohibition, and concludes correctly from a premise that is false. Uncertainty would have been the safe state, and the summary's fluency is exactly what removed it. The countermeasure is to leave a visible marker of what a compaction removed rather than closing seamlessly over it, which the last chapter of this topic works through on a live session.

Two categories, one test, and a decision taken at the cheapest moment available. The ledger settles which items cannot be paraphrased, it costs a small-model call on the handful of turns a rule pass flags, and it is the defence that does not depend on the summariser being good — which matters, because the summariser will not be good at this and no prompt makes it reliably so.

It says nothing, though, about what the paraphrase of everything else should contain, and the default answer — a well-written account of the conversation — is wrong for a reason that has nothing to do with constraints. A summariser can be fluent, accurate and complete by any editorial standard, and still leave the agent redoing work it finished twenty turns ago.