Attachments and thinking blocks

Everything a message carries as text can be stored in a text column. The two things that cannot are a reference to bytes and a signed opaque string, and they fail in opposite directions. An attachment is large, obviously valuable, and stored as a short identifier that looks durable and is not; a thinking block is large, invisible to the user, looks like debug output, and is thrown away by whoever wrote the persistence layer. Both are cheap to keep and impossible to reconstruct, which is the combination that makes them worth a chapter.

A file id is a cache key#

Uploading a document to a provider is a performance optimisation. It lets you reference 40 pages by a short string instead of pushing several megabytes of base64 through every request in a long conversation, and it lets the provider keep the parsed representation warm.

The phrase "every request in a long conversation" is doing more work there than it looks. A request to a model API carries the whole conversation up to that point, so a document attached on the first turn sits inside the body of the second turn and of every turn after it. Encoding bytes as text for transport makes them larger still. Referencing the document by a short string moves that cost off every request but the first, and it moves something else at the same time: the bytes now live in the provider's storage, and your database holds a name for them rather than a copy.

A document block that references an uploaded file looks entirely healthy in storage:

{"type": "document",
 "source": {"type": "file", "file_id": "file_011CQ8kZr4TmxWvNq9dJ7pLh"},
 "_extracted_text_chars": 61_284}

Retention on the object behind that id is set by the provider's product decisions, not by your compliance commitments, and it varies enormously — Google's File API expires uploads after 48 hours, Anthropic's Files API keeps them until you delete them, and others count uploads against a workspace quota and evict when you cross it. Those are the published policies of the day, which is itself part of the problem: each one is a product decision, and product decisions are revised. Even the most generous of them is someone else's lifecycle, subject to workspace consolidation, key rotation, account migration, and a cleanup job written by a team you will never meet.

Four steps get from that to the word "cache". The id names a row in a store you do not own. Whether the name still resolves is decided by that store's retention rule rather than by anything in your system. Nothing in your system observes the moment the rule is applied, because expiry is not an event anyone sends you. So the identifier's failure mode is that it works on every read until the read that matters, which is the defining behaviour of a cache entry: valid until evicted, with eviction announced by a miss. What makes this one dangerous is that the usual property of a cache — the ability to recompute the value on a miss — is exactly what an uploaded document does not have.

The durable record has to be yours, and it has four fields that all earn their place:

CREATE TABLE attachments (
  id           uuid PRIMARY KEY,
  tenant_id    uuid NOT NULL,
  blob_key     text NOT NULL UNIQUE,   -- s3://att/9f/3a/9f3a1c…e70b
  sha256       bytea NOT NULL,
  media_type   text  NOT NULL,         -- 'application/pdf'
  byte_size    bigint NOT NULL,
  filename     text,
  created_at   timestamptz NOT NULL DEFAULT now()
);

CREATE TABLE attachment_provider_refs (
  attachment_id    uuid NOT NULL REFERENCES attachments(id) ON DELETE CASCADE,
  provider         text NOT NULL,
  provider_file_id text NOT NULL,
  uploaded_at      timestamptz NOT NULL,
  expires_at       timestamptz,
  PRIMARY KEY (attachment_id, provider)
);

blob_key is the address of the bytes in storage you control. sha256 is what makes the key content-addressed, which gives you deduplication for free — the same terms-of-service PDF attached by 4,000 users is one object — and, more importantly, gives you a way to prove that the bytes you fetch today are the bytes that were uploaded. media_type has to be the value that was sent on the content block, because the API rejects a document block whose declared media type does not match the bytes, and sniffing it back from a filename extension fails the day someone uploads contract with no extension and your uploader defaults to application/octet-stream. byte_size is what lets a deletion job or a cost query work without listing the bucket.

Both of the jobs sha256 does come from one property: the digest is computed from the bytes and from nothing else — not the filename, not the upload time, not who sent it. Identical bytes therefore produce an identical digest, which is the whole of the deduplication story, since the second upload of the same PDF computes a key that is already in the table and there is nothing new to write. Different bytes produce a different digest, so a digest recomputed today that still matches the stored one is evidence that the object has not moved underneath you. Truncation during a bucket migration, silent corruption, and a well-meaning re-upload under the same key all change the bytes; none of them change blob_key, and the digest is the only column that notices.

media_type is defended above by what the API rejects, and the reason the rejection exists is worth one step further. A media type is not documentation attached to the bytes; it is the instruction that tells the receiving side how to interpret them, and a provider told application/pdf reaches for a different reader than one told image/png. A mismatch is therefore refused rather than guessed at, by any provider that checks, and the declared value is part of the request rather than metadata about it. A request rebuilt with a media type sniffed from a filename is not the request that ran, even when the bytes are identical and the sniff happens to be right.

The provider reference moves to its own table with an expires_at, which reframes it correctly: it is a cache entry with a lease, and the code that needs a file id asks for one rather than reading one. The blob_key column on the message-parts table points at attachments.blob_key, so a part identifies the bytes and never identifies a provider.

The order of operations on the way in follows from that. Hash the bytes, write them to the object store, insert the attachments row, and only then upload to the provider and record the reference — so the durable copy exists before anything depends on it. Doing it the other way round, uploading first because that is what the request needs, leaves a window in which a crash produces a message part referencing bytes nobody kept. The window is small and it is the kind of small window that produces one unrecoverable conversation a month at volume, which is enough to matter when the unrecoverable one is the one under dispute.

Walking both orderings through a crash shows why the sequence is not arbitrary. Upload to the provider first and the response hands back a file id, which is what the outgoing request needs, so the natural next move is to write the message part against it. A process that dies between those two moments leaves a row pointing at bytes that exist only on the provider's clock, and nothing looks wrong until the lease runs out. Hash and store first and the same crash leaves an attachments row and an object with no provider reference, which the next request repairs by uploading again — a missing lease is precisely what the reference table is shaped to expect. The ordering does not make a crash less likely. It decides which half of the record survives one, and only one of the two halves can be rebuilt from the other.

Replaying then has a shape:

The re-upload step is what makes the record durable across a provider's retention policy: the file id in the replayed request is a new one, and nothing about the conversation's meaning depends on which id it was, because the identifier was never content. That is the difference between a file_id and a tool call's identifier — the tool call identifier is load-bearing within a request and must round-trip, whereas the file id is a handle you are free to re-mint, provided you still have the bytes.

Which of the two identifiers has to survive is settled by whether anything else in the same request body refers to it. A tool result names the call it answers by quoting tool_use.id, so the two blocks are joined by that string and the body is coherent only while both sides carry the same one. Mint a fresh value on replay and the pair stops referring to itself, which a provider can see without knowing anything about your tools. A document block's file id is quoted by nothing at all: its only job is to hand the provider a way to reach the bytes, and any identifier reaching the same bytes leaves the request meaning what it meant before. That is why message_parts.blob_key is the column a stored part depends on, and why the replay job above can throw the old file id away.

Extracted text is a derivative, not a copy#

The _extracted_text_chars annotation in the block above hints at the usual consolation prize: the text your parser pulled out of the document at upload time, kept in a column, on the reasonable-sounding assumption that it is a backup. It is not, for three separate reasons that compound.

Extraction is lossy in ways that matter for exactly the documents people dispute. Table structure collapses into a run of numbers, a signature block becomes a blank, a two-column layout interleaves, and a scanned page contributes nothing at all unless OCR ran. Extraction is also version-dependent: the parser that produced those characters in December is not the one installed today, and re-running it on the same PDF now would produce different text, which means the stored text cannot be validated against anything. And on a model that accepts documents natively, the extracted text may never have been what the model saw in the first place — the provider did its own conversion, possibly rendering pages as images, so your text is a parallel artefact rather than a record of the input.

The belief underneath the column is that a document is its words, and it is tempting because for most documents, most of the time, the words are the part anybody reads. The documents that reach a dispute are drawn from the other set. A contract turns on a table, an invoice on a figure in a box, a signed form on the fact that it was signed, and each of those is layout carrying meaning rather than decorating it. The three reasons compound because they arrive together on that same document: the layout is what extraction drops, the parser that dropped it is no longer installed, and the model may have been reading a rendered page rather than the characters you kept.

Keep the extracted text if your retrieval or search path needs it. Store it as a derived column with the parser version alongside, and treat the object-store bytes as the source of truth. The rule generalises past attachments: anything your pipeline computed from content is a cache of a function you may no longer be able to run.

Thinking blocks are dropped first and missed later#

The other half of the round-trip problem runs in the opposite direction. Attachments are large and obviously valuable, so people keep a pointer to them. Thinking blocks are large, invisible to the user, and look like debug output, so the write path filters them out — and nothing appears to break, because on the first turn of a conversation there is nothing to send back.

It breaks in the agent loop, which is the cycle a tool-using product runs on every turn: call the model, execute whatever tools it asked for, append the results to the history, call the model again, and stop when it answers without asking for anything further. The failure appears at one specific moment in that cycle, when the history is reloaded from the database between issuing a tool call and sending the follow-up:

while True:
    history = load_conversation(conv_id)          # reads Postgres
    resp = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=4096,
        thinking={"type": "enabled", "budget_tokens": 4000},
        tools=TOOLS,
        messages=history,
    )
    persist_assistant_message(conv_id, resp.content)
    if resp.stop_reason != "tool_use":
        break
    results = run_tools(resp.content)
    persist_user_message(conv_id, results)

That loop is correct in every respect except one, and the defect is not in the loop at all — it is in persist_assistant_message, which keeps parts whose type is text or tool_use and discards the rest. On the second iteration, load_conversation returns an assistant turn that starts with the text block instead of the thinking block. The provider verifies that the final assistant turn preceding a tool result carries its thinking block with an intact signature, does not find one, and returns a 400. The request never reaches a model.

What lets a provider notice at all is worth understanding, because it settles what "preserving" a thinking block has to mean. The block comes back with a signature: a value computed over its content by the side that produced it, checkable by that side and not producible by anyone who does not hold what it was computed with. A provider needs one because the block leaves its control the instant it is returned. Whatever arrives on the next request is a string in a body your service assembled, and there is otherwise nothing to distinguish working the model generated from working the caller wrote afterwards. Checking the signature against the block in front of it is how the provider establishes that the reasoning it is about to condition on is its own.

That makes the storage requirement stricter than "keep the thinking" suggests. The check is over exact content, so anything that alters the bytes fails it as completely as dropping the block does: re-serialising through a JSON library that orders keys differently, normalising unicode on the way into a column, trimming what looks like trailing whitespace, or keeping the text and discarding the signature field because the table had no column for it. A parts table that stores the raw payload verbatim survives all four. One that stores a tidied projection of the payload survives none of them, and it fails with the same status code as the serialiser that filtered thinking out altogether, which is why the first hour of debugging it is usually spent looking at the wrong function.

The provider documentation states the rule directly — Anthropic's Extended thinking guide has a section on preserving thinking blocks that is worth reading once in full, because it also covers the redacted_thinking case and the interaction with prompt caching, both of which are easy to discover the hard way. The short version is that thinking blocks are only required on the assistant turn that immediately precedes a tool result within the same assistant sequence, which is exactly the turn the agent loop above reloads from storage.

A provider that tolerates the omission produces the quieter failure, and it is the one worth planning for. The model receives a conversation in which it apparently called shipping_estimate for reasons it can no longer see, gets a result back, and has to reconstruct its intent from the tool arguments alone. In practice it re-derives a plan, and the re-derived plan is often not the original one: the commonest observed symptom is the model calling the same tool a second time with slightly different arguments, because the reason the first call was sufficient lived in the thinking block. A loop that would have finished in two tool calls finishes in four, at double the tool cost and roughly double the latency.

The doubling is worth taking slowly, because it is the half of this that reaches a bill. One pass through the loop is a model call, a tool execution, and a further model call to consume the result. Going from two tool calls to four therefore doubles the executions and doubles the model calls that carry them, and each of those is a network round trip against a service that answers in seconds rather than milliseconds. The tool cost doubles exactly, since it is a count of executions. The latency only roughly doubles, because the extra passes are not identical to the first two and the history being sent grows on each one.

Notice what makes this defect hard to catch. Held entirely in memory, the loop works perfectly, because resp.content still has the thinking block in it and the next request is built from the object rather than from the database. It only fails when the process is stateless across iterations, which is the arrangement a horizontally scaled service is built on. The test that catches it is cheap: restart between turn one and turn two, and assert the request body is byte-identical.

A service is stateless across iterations deliberately rather than by neglect. A conversation that has to survive a deploy, a crashed pod or a user closing the tab must be resumable from storage, and once any instance may pick up the next turn, the database is the loop's memory rather than a record of it. The single-process version is then the special case — one machine, one uninterrupted run, resp.content still in scope — and it is the version a fixture sets up, because it is the one that fits in a fixture. Anything the write path discards stays invisible for exactly as long as that in-memory object is available, and in production it stops being available at the first restart.

The cost of retention, and the limits of a thinking block#

The case against storing thinking blocks is a storage argument that does not survive its own numbers. A 1,200-token thinking block is roughly 4.8 KB of UTF-8, plus a signature of a few hundred bytes. At 3,000 assistant turns a day with thinking enabled on half of them, that is about 7 MB a day and 2.6 GB a year before compression, on a column that compresses well because it is prose. Against the cost of a support investigation that cannot be replayed, or a doubled tool-call bill from a loop that lost its plan, the storage is not the deciding factor.

Every step of that sum is worth having in the open, since the argument it answers is usually made with a wave rather than a multiplication. A token of ordinary prose is a few characters, which is what puts a block of twelve hundred of them near the 4.8 KB quoted, with the signature adding a few hundred bytes rather than a multiple. Thinking on half of three thousand turns is fifteen hundred blocks a day, and fifteen hundred blocks of not quite five kilobytes each comes to a bit over seven thousand kilobytes, which is the 7 MB. A year is three hundred and sixty-five of those days, so seven megabytes a day is a little over two and a half thousand megabytes, which is the 2.6 GB. Both multiplications are allowed only because the rates are assumed steady, and a team with a month of traffic can replace the assumption with a measurement in an afternoon.

The considerations that genuinely argue against retention are different ones. Thinking text contains user content, verbatim and often at length, which means it inherits every obligation the message content has — it is in scope for an erasure request, it belongs inside your tenant boundary, and it must not leak into a debug view with weaker access control than the conversation itself. Some teams reasonably decide to keep thinking blocks only for the duration of an active agent loop, deleting them when the loop terminates, which preserves the round-trip while shortening the exposure. That is a defensible policy precisely because it is a policy, arrived at deliberately, rather than a filter in a serialiser that nobody remembers writing.

The replay job is not the only consumer of the durable attachment record, and the second one shows up sooner. A user scrolling back through a six-month-old conversation expects to see the PDF they uploaded, which means the message renderer needs the same blob_key, and it needs media_type to set a Content-Type on the signed URL it hands the browser, and it needs filename for the download name. Building that path against the provider's file id instead gives you a viewer that works for two days and then shows broken attachments in old threads, which arrives as a product bug long before anyone tries a replay. Tenant scoping belongs in the key layout rather than only in the query, since a signed URL escapes your authorisation layer by design and a path that encodes the tenant lets a bucket policy backstop a mistake in application code.

It is worth being clear about what a thinking block is not, because storing it invites over-reading. It is not an audit trail of how the answer was computed, and it is not a faithful account of the model's internal state; it is generated text that the model conditions on, useful as evidence of what the model was working with and unreliable as an explanation of why. Treating it as a reason-for-decision record in a compliance artefact claims more than the artefact supports. It is also not a substitute for logging your own tool inputs and outputs, which are the ground truth about what actually happened outside the model.

Both halves of this chapter come down to the same rule applied to two very different objects: store the thing itself, not a handle to it that somebody else controls, and not a derivative you computed from it. For attachments the thing itself is the bytes, addressed by a checksum in a bucket you own, with the provider's identifier demoted to a cache entry that can expire without consequence. For thinking, the thing itself is the exact string and its signature, kept because the request you will build tomorrow has to contain them and nothing can regenerate them.

Between them, the attachment tables and the retained thinking parts mean a stored conversation can be rebuilt into a request that is byte-identical to the one that ran. That is the content side finished. What the record still cannot tell you is which system prompt was in front of all of it, and that omission is the reason regression investigations tend to end in a shrug.