Reasoning tokens

Reasoning modes go by several names — extended thinking, a thinking object in the request body, a reasoning effort level — and they all do the same thing. The model reaches its reply in two stages: it first generates intermediate text for itself, restating what the input contains, trying an approach, discarding it, settling on one, which is working in the sense a mathematician means it, and only then writes the message the caller receives. The tokens in that intermediate stretch are reasoning tokens, called thinking tokens on some APIs. They exist because writing the working down makes the answer better: a model that must commit to its reply immediately has to reconcile every constraint in the input in a single pass, whereas one that can spend a few hundred tokens setting those constraints out and testing a candidate against them gets multi-step judgements right considerably more often.

What makes them worth a chapter is that they are generated, counted and charged, and then not shown. Nothing in the response body renders them, so the most natural way to estimate the cost of a workload — take the answers it produces, divide the characters by four, multiply by the output rate — is wrong by a factor nobody has a feel for.

Where the extra tokens are#

The tokens are on the completion side. That text is produced by the same decoder, one token at a time, at the same cost per token as anything else the model writes; what differs is that the API does not hand it to you as part of the message content. It is withheld from the response body, or replaced by a short summary.

"The same decoder" is the load-bearing phrase, and it is worth taking apart. A language model writes text one piece at a time. At each step it takes everything in front of it — the prompt, plus whatever it has already written in this response — scores every candidate token in its vocabulary against that context, picks one, and appends it. The appended token is part of the context for the step after, and that is the whole loop: score, pick, append, repeat until a token comes out that means stop.

A reasoning token is produced by that loop, from those weights, in that run, in the same sequence as everything else. The boundary between the deliberation and the reply is a marker in the token stream — a token the model has been trained to emit when it has finished working — telling the serving layer where one stretch ends and the other begins. The API hands back what falls after the marker and withholds what falls before it. Structurally nothing else separates the two: same weights, same vocabulary, same step, different side of a boundary the model wrote itself.

That same fact is why the working is worth generating. Each token is appended and becomes part of what the next token is conditioned on, so a constraint the model writes out in its deliberation is one that later tokens can read straight off the context, while a constraint left implicit in the input has to be re-derived at every step. A model that must commit to its reply immediately has a single pass in which to reconcile everything the request contains; a model that may deliberate first gets as many passes as it writes tokens, each conditioned on what the previous ones concluded. That is why the technique works, and also why nobody can promise it will help on a given request: it buys further chances to reconcile constraints, and whether an input needed more than one is not knowable in advance.

The awkward properties of a reasoning token all fall out of the same place. They are billed at the output rate because they came off the completion path and are, position for position, the same work as a token of the answer. They arrive before the first visible character because each pick requires every pick before it to exist, which is why the token count and the wall-clock are the same measurement in different units. They cannot be served from a cache, because a cache operates on a prefix fixed before the request is sent and the deliberation does not exist until the request is being answered. And the count is not a quantity you set, because the end of the working is itself an emitted token, chosen by the same scoring step as every token before it, so its position is an outcome of the request in the way the length of any answer is.

Three places carry the consequences. Every response carries a usage object in which the reasoning count appears, on most providers as a nested detail under the completion-side total rather than as a field of its own. Every reasoning call carries the tokens in its latency, since they are generated sequentially before the first visible character reaches the client. And every invoice carries them at the output rate, folded into a line item that does not distinguish them from the text a user read.

Providers withhold the raw text for reasons that have nothing to do with billing, and the reasons are worth knowing because they explain why no flag will turn it back on. A model's unedited working is the most direct training signal a competitor could extract from a hosted API, so exposing it is a commercial problem. It is also the part of the generation that has been through the least polishing, which makes it the part most likely to contain a discarded hypothesis, a blunt characterisation of the user, or a restatement of something sensitive from the prompt. The provider's position is that the deliberation is an implementation detail of producing the answer, and the answer is the product. You are buying a computation whose intermediate state you cannot inspect and whose length you are charged for, which is an unusual thing to buy and worth being deliberate about.

Because it comes off the completion path, it is billed at the output rate, which on the published rate cards of most providers sits in the region of three to five times the input rate. That multiple is a fact about a price sheet on a particular day rather than a fact about the model, so it is worth reading off the page your own provider publishes. Take an endpoint that summarises support tickets, listing at $3.00 per million input tokens and $15.00 per million output tokens. Here is the same request, with the only difference being whether reasoning is enabled, as the API reports it:

{
  "id": "resp_8f2c14",
  "model": "reasoner-mid",
  "output": [
    {"type": "message", "content": [{"type": "output_text", "text": "Customer reports …"}]}
  ],
  "usage": {
    "input_tokens": 1850,
    "output_tokens": 210,
    "output_tokens_details": {"reasoning_tokens": 0}
  }
}

{
  "id": "resp_9a71bd",
  "model": "reasoner-mid",
  "output": [
    {"type": "reasoning", "id": "rs_44c1", "summary": []},
    {"type": "message", "content": [{"type": "output_text", "text": "Customer reports …"}]}
  ],
  "usage": {
    "input_tokens": 1850,
    "output_tokens": 1438,
    "output_tokens_details": {"reasoning_tokens": 1216}
  }
}

Neither payload states the length of the visible answer, which is the one figure a reader expects to find and the one the API never reports. output_tokens is the completion-side total and reasoning_tokens is the part of that total which was withheld, so what the user received is what is left when the second is taken out of the first: 1,438 minus 1,216, which is 222. Subtraction is the only route to it, and every provider that reports the split at all reports it this way round — a total and a hidden share, never a visible share.

The visible answer went from 210 tokens to 222, a difference of six percent that nobody would notice: twelve tokens on top of two hundred and ten, well inside the variation between two summaries of two similar tickets. The billed output went from 210 tokens to 1,438, because 1,216 tokens of deliberation sit inside that number. Dividing the hidden count by the visible one, 1,216 over 222, gives about five and a half. The ratio of hidden to visible on this workload is about 5.5 to 1, and it is that ratio, not the length of anything on screen, that determines the invoice. At 129,000 calls a month the arithmetic reconciles cleanly:

Two rows are worth working through; the rest follow the same shape. The input line: 129,000 calls at 1,850 input tokens each is a little over two hundred and thirty-eight million tokens for the month, and at $3.00 per million that is the $716. The same $716 appears in both columns because the prompt template did not change and reasoning is generated rather than sent, so it lands on the completion side by construction and enabling it adds nothing to the prompt. The output line with reasoning on: 129,000 calls at 1,438 billed output tokens each is a little over one hundred and eighty-five million tokens, and at $15.00 per million that is the $2,782. The same multiplication on the other column uses 210 tokens in place of 1,438 and comes to $406, so the whole gap between $1,122 and $3,498 is one cell of one row.

The belief that fails here is that deliberation is an overhead on the answer, a fraction of what the model writes back rather than a multiple of it. It is tempting because most hidden costs in a request do behave that way, and because a summary field, where a provider populates one, is short enough to keep the impression alive. Estimating the bill from stored responses — count the characters, divide by four, apply the output rate — is the most natural estimate available, and on a reasoning endpoint it is wrong by a multiple that has nothing on screen for its size to track, which is why re-reading the summaries never corrects it.

The cost is not the only thing that moves. Those 1,216 extra tokens are generated sequentially before the first visible character appears, so at the roughly 55 tokens per second this model sustains, median latency on such an endpoint goes from around 3.8 seconds to 26, and p95 goes past a minute on tickets carrying long pasted logs. Both figures are one division: 210 tokens at 55 a second is 3.8 seconds, and 1,438 tokens at the same rate is 26. Nothing on the reasoning path is slower per token; there are simply far more tokens between the request and the first word of the reply, and each is conditioned on the one before it, so they have to be produced in order. On an asynchronous summarisation job that is invisible. On anything a person is waiting for, the token count and the wall-clock are the same measurement in different units, and the second one usually surfaces first, as a support complaint rather than a finance alert.

Reading the split, and inferring it when nobody reports it#

The field names differ, but the shape is consistent: reasoning is reported as a detail nested under the completion-side count, not as a sibling of it. That nesting is the load-bearing part. reasoning_tokens is not added to output_tokens; it is already inside it, and code that sums the two double-counts the deliberation.

That last row is the one that catches people. Anthropic's usage.output_tokens includes thinking tokens, and there is no field that isolates them, so on that API the split is something you measure rather than something you read. The measurement is a differencing experiment: run a fixed sample of real production inputs twice, once with reasoning enabled and once without, and take the difference in mean output tokens as your estimate of the hidden load.

import statistics

def output_tokens(client, prompt, *, thinking):
    kwargs = {"thinking": {"type": "adaptive"}} if thinking else {}
    resp = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=4096,
        messages=[{"role": "user", "content": prompt}],
        **kwargs,
    )
    return resp.usage.output_tokens

sample = load_production_prompts(n=200)

off = [output_tokens(client, p, thinking=False) for p in sample]
on  = [output_tokens(client, p, thinking=True)  for p in sample]

print(f"visible only : mean {statistics.mean(off):7.1f}  p95 {sorted(off)[189]:6d}")
print(f"with thinking: mean {statistics.mean(on):7.1f}  p95 {sorted(on)[189]:6d}")
print(f"hidden share : {1 - statistics.mean(off) / statistics.mean(on):.1%}")

Two details of that experiment decide whether the result means anything. The sample has to be drawn from production traffic rather than written by hand, because reasoning length tracks how hard the input is and hand-written test cases are systematically easier than the tickets that actually arrive at three in the morning. And the two runs have to happen close together against the same model identifier, since anything that moves between them — a different snapshot behind the alias, a different effort setting, a different system prompt — lands in the difference and gets attributed to reasoning.

That second requirement is the assumption the whole measurement rests on, and it is worth stating as one. The estimate is a subtraction: the mean output with reasoning on, minus the mean output with it off. A subtraction returns one number and keeps no record of what produced it, so it cannot separate a difference caused by the reasoning mode from a difference caused by anything else that moved between the runs. Whatever else moved is inside the result already, wearing the name of the thing being measured. The experiment is sound exactly to the degree that the reasoning switch was the only change, which is a claim about how the runs were conducted rather than something the numbers can confirm.

Run it on two hundred distinct inputs rather than two hundred copies of one input, because the interesting number is not the mean. On the ticket workload above the mean hidden share was 85%, and the p95 was 3,900 output tokens against a mean of 1,438 — a long right tail, driven by the tickets with pasted stack traces and ambiguous severity. Reasoning spend is far more variable per request than visible output is, which means a monthly forecast built on the mean will be roughly right and a per-request timeout built on the mean will be wrong most days.

The two distributions have different shapes for a reason worth holding on to. How long the model deliberates tracks how hard the input is, and difficulty in a real queue is heavily skewed: most tickets are a sentence and a screenshot, a few carry a pasted stack trace and a contradictory reproduction, and the few are not slightly harder than the many but far harder. Visible output has no such spread, because its length is set by the instruction rather than by the input — an instruction to summarise in three or four sentences returns three or four sentences whether the ticket was trivial or awful. So a distribution of deliberation lengths inherits the skew of the workload while a distribution of summary lengths stays clamped by the prompt. That difference in shape is what makes one mean useful and the other misleading: summed over a hundred thousand calls the long requests average out against the short ones, whereas a per-request limit only ever meets the long ones.

Once you have the shape, the measurement belongs in the request path rather than in a notebook, because the ratio drifts and a one-off number ages badly. One structured field per call is enough:

log.info(
    "llm_call",
    extra={
        "model": resp.model,
        "input_tokens": resp.usage.input_tokens,
        "output_tokens": resp.usage.output_tokens,
        "reasoning_tokens": getattr(
            resp.usage.output_tokens_details, "reasoning_tokens", None
        ),
        "finish_reason": resp.stop_reason,
        "latency_ms": elapsed_ms,
    },
)

That record costs nothing to emit and answers, later and without an experiment, both the accounting question and the diagnostic questions raised in the next few chapters — which requests spent their whole allowance on deliberation, and which ones stopped because they ran out of it.

Caching, summaries and version drift#

The count is not an input-side cost, and that matters because input tokens are the cheap ones and are often cached. Hidden reasoning is charged at the output rate on every request, and it does not benefit from prompt caching — the discount providers give on a prompt prefix they have seen recently — because it is not part of the prompt. It is freshly generated each time, even for two identical requests thirty seconds apart.

The reason no cache can reach it lies in how caching works rather than in a policy anyone chose. A prompt cache matches on a prefix of the input that is fully determined before the request is sent: the provider hashes the leading stretch of tokens, finds the intermediate state it computed for that same stretch recently, and skips recomputing it. All of that requires the tokens to exist when the request arrives, and reasoning tokens do not — they are produced during the request, by the loop above, and nothing guarantees they are the same tokens on the next call with a byte-identical prompt. Expecting a caching discount to cover a reasoning bill is tempting because caching is the lever that has historically absorbed surprises in an LLM invoice, and it is the one lever with no purchase here.

It is not a summary you can read to justify the spend either. When a provider returns a summarised version of the reasoning, you are billed for the full reasoning that was generated, not for the summary you receive; the summary is a rendering of a longer artefact and typically a small fraction of its length. Paying for 1,216 tokens and being shown 80 is the normal case, not a billing error. The summary is produced from working already generated and already counted, so its length is evidence about the rendering rather than about the computation. Reading it as a receipt is the natural move, since it is the only trace of the deliberation anyone hands you, and a receipt is the one thing it cannot be.

It is also not persistent context in the way a longer answer would be. In an ordinary multi-turn conversation the reasoning from turn one is usually not resent on turn two, so it does not inflate the input side of subsequent turns the way visible output does. That is a genuine relief on cost and a genuine trap elsewhere, because there is one situation — a tool-call turn — where the reasoning must go back, and dropping it breaks the request. That case has its own chapter.

And it is not a quantity you set. The hidden count is an outcome of the request, decided by the model as it works, which is why it moves with input difficulty and why two runs of the same prompt can differ by hundreds of tokens. The knobs an API gives you bound it or bias it; none of them make it a fixed line item. A budgeting process that treats reasoning tokens as a configured constant will therefore be wrong in the same direction every month, because the requests that overshoot are the same hard requests that arrive in bursts. Nor is it stable across model versions: it is a behaviour of a specific set of weights at a specific setting, so the same alias pointing at a newer snapshot can move the hidden share by tens of percent with no change on your side. Any measurement of it is a snapshot with a date on it, worth re-running whenever the model identifier you send changes, including when it changes underneath you.

"Decided by the model as it works" has a precise meaning, and it goes back to the marker. The deliberation ends when the model emits the token that ends it, and that token is chosen by the same scoring step as every token before it, conditioned on everything written so far. There is no counter running beside the generation for a request field to set, so a length cannot be asked for the way a temperature can be. Believing the hidden count is a setting is tempting because almost everything adjacent to it in a request body is one; the count belongs with the length of the answer, among the things a request produces rather than the things it specifies.

Reasoning tokens, then, are ordinary completion tokens with two unusual properties: you cannot read them, and you did not choose how many there would be. Everything else about them is familiar — same decoder, same output rate, same sequential generation and therefore the same contribution to latency. The practical consequence is that any cost or latency model for a reasoning workload has to be built from the usage payload rather than from the artefact the user sees, and that the split is worth logging per call rather than establishing once, because the ratio moves with input difficulty, with effort settings, and with whatever the provider is serving behind your model name this week.

Having the split in hand answers the accounting question and immediately raises the control question. If the hidden portion is five times the visible portion and varies with the input, the obvious next move is to cap it — and most APIs offer a knob that looks exactly like a cap, usually called a thinking budget or a reasoning budget. What that knob actually promises, and how it interacts with the output limit you already set, is where the interesting failures live.