Six of nine fields is not a partial success

Six complete fields out of nine feels like two-thirds of a result, and the instinct comes from a reasonable analogy to partial file transfers, where the first 80% of a video is 80% of a video. Structured output does not work that way, because the fields are not independent samples of the same thing. Six of nine is a document with three holes whose shape you cannot describe, and the specific thing you cannot describe is which absences are real. If blocking_conditions is missing from a truncated triage object, the honest reading is unknown, and every storage format you have — a nullable column, an omitted JSON key, a Go zero value, a Java Optional.empty() — reads it as absent.

The comparison to a partial download is an analogy, and it is worth taking apart rather than waving off, because it is the thing that makes the instinct feel safe. A video file is a long sequence of samples of one quantity, so any prefix of it is a shorter instance of the same kind: it plays, its meaning does not depend on the bytes that never arrived, and the player knows precisely how much is missing because the length was declared before the transfer began. A structured result has none of those three properties. Its fields are claims about different quantities rather than samples of one, so a prefix is not a smaller result of the same kind. A present field's meaning can depend on an absent one — a total with no line_items under it is not two thirds of a reconciliation. And no length was declared, so the number of holes is unknown along with their contents. The analogy holds for exactly one feature, that both transfers stopped early, and that feature is the only thing the two failures have in common.

This chapter is about the write path, meaning the code that turns a stream's output into a row, a message or a typed model. It has one rule and three causes, and the rule is cheap while the causes need instrumentation that has to exist before the failure rather than after it.

Keeping the bytes without keeping a result#

So the write path has one rule: if the object did not close, it is not a result, and the typed record does not get written. That does not mean the bytes are worthless. They are worth keeping in a place that cannot be mistaken for an answer:

def persist(stream_result, db):
    if not stream_result.complete:
        db.execute(
            """INSERT INTO extraction_attempts
                 (request_id, status, cause, partial_payload, bytes_received, elapsed_ms)
               VALUES (%s, 'truncated', %s, %s, %s, %s)""",
            (stream_result.request_id, stream_result.cause,
             json.dumps(stream_result.snapshot),   # diagnostics only
             stream_result.bytes_received, stream_result.elapsed_ms),
        )
        return None

    invoice = Invoice.model_validate(stream_result.value)   # schema check, topic 5.2
    db.execute("INSERT INTO invoices ...", invoice.model_dump())
    return invoice

The partial_payload column is a jsonb blob in a table nothing joins against, and the value of writing it is that when someone asks months later why a batch of rows is missing a field, the answer takes ten minutes instead of a week. Two properties keep it honest: it lives in a different table from the results, and no code path reads it back into a typed model.

The trap that undoes all of this is the default value. A schema with sensible defaults will validate a truncated object cleanly — confidence: float = 0.5, tags: list[str] = [], status: str = "pending" — and hand you a complete-looking instance built from three fields that never arrived. Defaults are a good idea for input parsing and a hazard for stream output, and the reliable guard is checking complete before validation rather than hoping validation catches it, because validation is designed to fill gaps and will do its job.

Three causes that look identical from the parser's seat#

The parser reports one fact: the document did not close. Underneath it are three different events with three different remedies, and a policy of "retry once" applied to all three is wrong in two of the cases.

The first column is the one that surprises people, because a max_tokens truncation is not a transport failure at all. The provider sends a complete, well-framed stream that ends with a proper terminal event — Anthropic's message_delta carries "stop_reason": "max_tokens" and is followed by message_stop; OpenAI's Responses API emits response.incomplete with incomplete_details.reason of max_output_tokens, and the Chat Completions shape sets finish_reason: "length". Every layer beneath you reports success. The only thing wrong is that the JSON stopped in the middle, and the only place that fact appears is the stop reason and your own parser's complete flag disagreeing with each other.

That last clause is the entire detection problem and it repays walking down. A stop reason is a field in the provider's terminal event, and what it reports is why the model stopped writing. The parser's complete flag is a fact about the bytes, and what it reports is whether the document closed. On a healthy response the two agree — the model stopped because it had finished, and the document closed. Under a length truncation they disagree, and no layer between them is in a position to notice. The transport succeeded. The framing succeeded. The status code was fixed before the first byte of the body left the server. The provider did exactly what it undertook to do, which was produce tokens until it reached the ceiling you handed it. Nothing has an error to raise, so the comparison has to be made by you, in your own code, between two values that arrive in different objects. A client that reads the stop reason but not complete, or complete but not the stop reason, is not partially covered; it is blind to the whole class.

Retrying that unchanged is the mistake worth naming, because the request is identical, the limit is identical, and the model will produce an object of roughly the same length and hit the same wall. You may burn three full generations and 4,500 output tokens to arrive at the same truncated result.

That token figure follows from what a truncated generation costs, which is the entire ceiling every time. A request that truncates has by definition produced output right up to max_tokens, so it is the most expensive generation that request is capable of producing. Retry it and you buy the ceiling again, so three attempts at an unchanged limit is three times the ceiling in output tokens. What they are spent reaching was determined before the first attempt began: the object the model wants to write is longer than the room it has been given, and neither the request nor the limit moved between attempts. It is the cleanest case of a retry policy being exactly backwards — the failure is deterministic in the one parameter the retry leaves alone.

The remedies are all structural: raise max_tokens with headroom over the object's observed p99 length, the length that only one generation in a hundred exceeds, shorten the object by moving a long prose field out of the structured call, tighten field descriptions so the model stops writing 200-word summary values into a field you expected to be 40, or split a nine-field object into two calls.

A client disconnect is your own user closing the tab or navigating away, and the signal for it is on the side of the connection facing them. In an ASGI server — the asynchronous server interface that FastAPI, Starlette and their neighbours are built on — that is await request.is_disconnected() or the asyncio.CancelledError raised into your handler; in Node it is the close event on the response or an AbortSignal firing. The upstream generation is fine and will keep going, which is where topic 2.9's point about the meter running on an abandoned call applies — the correct response is to abort the upstream call rather than to retry anything, since there is nobody to retry for.

An upstream reset is the genuine network failure: the body ends without a terminal event and the read raises. In httpx you see RemoteProtocolError or ReadError, in requests a ChunkedEncodingError, in Node an ECONNRESET on the socket. This is the case a retry fixes, and it should carry an idempotency key — a caller-chosen id sent with both attempts, so the provider can recognise the second as a repeat of the first rather than a new job — and a budget of one attempt, because the retry costs a second full generation at full token price and the failure rate is low enough that the extra spend is small only as long as you do not apply the same policy to the other two causes.

Continuing the object produces a third kind of result#

The remedy that gets proposed once the retry-unchanged argument lands is to resume: feed the truncated bytes back as an assistant turn and let the model finish the document, paying for the tail rather than for the whole thing again. The mechanism works, and it is worth knowing what the joined document is and is not.

snapshot = stream_result.snapshot_text.rstrip()   # trailing whitespace is rejected
resp = client.messages.create(
    model="claude-sonnet-4-5-20250929",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": prompt},
        {"role": "assistant", "content": snapshot},
    ],
)
joined = snapshot + resp.content[0].text

The continuation is a fresh generation conditioned on a prefix the model did not choose to stop at. Across 200 truncated extractions on one invoice service, 181 joined documents parsed as valid JSON; the other 19 broke on a duplicated key or a brace the model closed twice, both of which are visible to the parser and cheap to reject. The number that mattered was in the 181: 34 of them had a total that disagreed with the sum of line_items by more than a cent, against 3 in a control arm where the same 200 documents were regenerated from scratch with a higher limit. A model resuming mid-object is reasoning about a totals field whose components it emitted under a different pass, and it reconciles them less reliably than one writing the whole document in a single sweep.

Two of those numbers do different jobs and only the second decides anything. That 181 of 200 joined documents parsed is a statement about syntax, and the 19 that failed did so in ways a parser catches on sight, so they cost a rejection and no more. The number that decides sits inside the 181: 34 of them — roughly one in five — carried a total that disagreed with the sum of the line_items printed beside it, against 3 in 200 for the same documents regenerated whole, which is closer to one in seventy. The word carrying the mechanism is conditioned. A continuation is a fresh generation whose context is a prefix the model did not choose to stop at, so the components of the sum were emitted in one pass and the sum itself is written in another, with nothing carried across between them except the text. A model writing a document in one sweep is reading its own working-out. A model resuming is reading the same characters with none of the state that produced them.

Resumption is therefore a cost optimisation with a correctness cost attached, and it is worth it when the tail is genuinely long — a 6,000-token document truncated at 5,400 is expensive to redo — and not worth it for a nine-field invoice where a full retry with max_tokens raised from 800 to 1,200 costs about 780 output tokens, roughly a tenth of a cent, and produces a document generated in one piece. Raising the ceiling is close to free in itself, since you are billed for tokens produced rather than for the limit, and the constraint on it is the time budget of whatever is waiting rather than the price.

Instrumenting so the classification is a lookup#

You cannot tell these apart after the fact from a log line that says stream ended early. The distinguishing facts exist only during the stream, so the streaming loop has to record them as it goes:

@dataclass
class StreamOutcome:
    complete: bool
    cause: str | None          # "max_tokens" | "client_disconnect" | "upstream_reset" | None
    stop_reason: str | None    # from the terminal event, if one arrived
    terminal_seen: bool
    bytes_received: int
    output_tokens: int | None
    elapsed_ms: int

def classify(terminal_seen, stop_reason, downstream_closed, upstream_error, parser_complete):
    if parser_complete and stop_reason in ("end_turn", "stop", "tool_use"):
        return None
    if terminal_seen and stop_reason == "max_tokens":
        return "max_tokens"
    if downstream_closed:
        return "client_disconnect"
    if upstream_error is not None or not terminal_seen:
        return "upstream_reset"
    return "unknown"          # emit a metric; an unknown bucket that grows is a bug

Order matters in that function. downstream_closed is checked before the upstream error because a client that vanishes often produces an upstream error as a consequence of your cancellation, and attributing that to the network would tell you to retry a request nobody is waiting for. The unknown branch exists so that a fourth cause you have not thought of shows up as a counter rather than as a silently miscategorised retry.

With those fields on every stream, the split stops being a guess. A service seeing 0.7% of streams end mid-object typically finds something like 0.45% max_tokens, 0.2% client disconnects concentrated in mobile sessions, and 0.05% upstream resets — and those three numbers point at three different pieces of work, in that order. Chasing the network for a problem that is 64% output-length budgeting is a week spent on the smallest term.

That last figure is a division rather than a fourth measurement, and doing it is what schedules the work. Of the 0.7% of streams that end mid-object, 0.45 points are length truncations, which is about sixty-four in every hundred failures, and the two network-shaped causes together account for a bit over a third. So the largest single piece of the problem is a number in a configuration file, the next belongs to whoever owns the interactive path, and the smallest is the only one a retry addresses. Teams reliably start at the wrong end of that list, and the reason is not carelessness: a stream that ends early presents as a network event, so the network is where anybody would look, and the classification is the only thing that says otherwise.

Truncation and invalidity are different failures#

None of this is about schema validation failures. An object that closes cleanly and then fails model_validate is a different animal with a different remedy — retry with the error fed back, or reject — and topic 5.2, on constraining and validating model output, covers it. The distinction is worth keeping in your metrics as well as in your head, because "invalid output" and "truncated output" have almost nothing in common except that both end with no usable row.

It is also not a completeness guarantee for the object's contents. complete: True means the document closed and the terminal event said the model chose to stop, and a model can close an object having invented a VAT rate. Truncation handling removes one class of manufactured data and leaves the rest to validation and evaluation.

The rule is one line and the instrumentation is a dataclass, and together they are the difference between a service that occasionally has no answer and a service that occasionally manufactures one. An unwritten row is a visible gap somebody can query for; a row built from six fields and three defaults is indistinguishable from a correct row for as long as anyone cares to look, and the storage layer will never tell you which kind you have, because a nullable column holds one value for "the model said no" and for "the bytes stopped".

The three causes are the second half, and they are worth separating even in a service that never resumes and never retries, because they route to three different pieces of work. Output-length budgeting is usually the largest of the three by an order of magnitude and is fixed in a config value; client disconnects are fixed by cancelling upstream and belong to whoever owns the interactive path; upstream resets are the small residual that a retry actually helps. A single stream ended early counter tells you none of that, and by the time anybody asks which of the three is growing, the streams that would have answered are gone.

The write path is now honest. The screen is not, because at the moment the backend decides those six fields are not a result, the user has been reading them for two seconds, and nothing has told them otherwise.