Reasoning models and token accounting

Reasoning models are adopted the way most model changes are adopted: as a configuration edit to an endpoint that already works, one extra parameter in a request body, with nothing else moving that would prompt anyone to revisit the cost model. This topic is about the measurements that make that decision defensible — what the setting costs, which parts of your traffic it helps, how to tell an exhausted token budget from a slow request, and when a cheap model with a check in front of it beats the expensive one outright.

None of that is answerable without knowing which number is being billed. This chapter defines what a reasoning model does, establishes how the tokens it generates are accounted across the three major APIs, and builds the per-call record that every later chapter reads from. The instrumentation is about forty lines and the task set behind it is a frozen file of production requests, and together they are the difference between an argument settled by measurement and one settled by whoever spoke last.

What the extra tokens are#

Given a request, a reasoning model first produces a stretch of intermediate text — restating the problem, listing the cases, doing the arithmetic, checking a conclusion against the constraints it was handed — and only then produces the reply that reaches the caller. Providers label that stretch extended thinking, and the tokens it consumes reasoning tokens or thinking tokens. On a support-triage workload, a model might spend 2,750 of them deciding whether a ticket mentioning a reversed charge belongs with billing disputes or with fraud, and then emit a 410-token answer.

Such models exist because an ordinary instruct model — one post-trained to follow a request with a helpful reply, the sibling of the base and multimodal checkpoints from the previous topic, and the variant most production traffic runs on — has nowhere to put its working. It produces the answer one token at a time, and the computation available before each token is committed is fixed by the architecture rather than by the difficulty of the question. On a request needing four dependent steps, that means beginning the answer before the fourth step has been worked out, so correctness depends on the first token being a good guess about where the answer lands. Prompting a model to work through a problem step by step was the first workaround, and it helps because the intermediate text becomes part of the context the model reads while writing everything after it, which is to say the scratch paper is genuinely being used. Reasoning models make that behaviour trained-in rather than prompted, and attach a control to it, so the amount of work spent is decided per request instead of by whoever wrote the prompt. The general name for the trade is test-time compute: buying accuracy by spending more at inference time rather than by training or serving a larger model.

Naming it that way makes the alternative visible, which is what turns it from a feature into a trade. The other way to spend more computation on a hard question is to run a bigger model, and a bigger model spends it on every token of every request, including all the requests that did not need it. Test-time compute spends it per request, in a quantity that can vary, through a control you hold. That is the better shape when difficulty is unevenly spread across your traffic and the worse one when it is not, which is the next chapter's argument compressed into a sentence.

Underneath the observation that an instruct model has nowhere to put its working there is a harder constraint, and every number in this topic follows from it. A model performs a fixed amount of computation to produce one token. The same stack of layers runs once, in the same order, whether the question is which of two categories a ticket belongs to or how to unwind a deadlock across four services. Nothing about a hard question makes the stack deeper or the pass longer. So there is exactly one way to spend more computation on a request, and that is to produce more tokens.

The second half of the constraint is what those extra tokens can be used for. Each token is appended to the sequence as it is produced, and every token generated after it is scored from a sequence containing it. There is no register the model can write a partial result into and read back three tokens later, and nothing carries forward between tokens except the text itself. So the intermediate work is not a description of thinking that happens to get written down. Writing it down is the only way the work can exist and still be available to the tokens that use it.

Two conclusions follow, and they are worth having before the accounting rather than after the invoice. Thinking is not a mode the model switches into — it is text, produced by the same loop and the same weights that produce the answer. And text of any kind costs one token per token, which is why the bill takes the shape it does: the provider ran one forward pass for each of those tokens before anybody decided whether you would be shown them.

The description this displaces is that a reasoning model is an instruct model told to think step by step, and it is tempting for two honest reasons — the two produce text that reads alike, and prompting genuinely does help, as the paragraph above says. What separates them is not what the intermediate text is made of. It is who decides how much of it there is, and where it comes out. A prompted model writes as much working as the phrasing happens to elicit, in the same field as the answer, on every request alike. A reasoning model has been trained to produce that working as a distinct stretch of output, carries a control on the request that bounds how much of it is permitted, and emits it on a channel the provider can summarise or withhold. That is why the setting in the next chapter exists at all, and why the counts below have somewhere separate to be reported.

Where you meet it is an ordinary API call carrying one extra parameter. Anthropic takes a thinking budget, a ceiling in tokens the model may spend before answering. OpenAI takes a reasoning effort, a named level from minimal through high with no token figure exposed. Google takes a budget again, with a value that hands the decision back to the model per request. The workloads where the setting earns its keep are the ones with dependent steps — multi-step planning, hard debugging, mathematics, agents that must decide what to do next — and the workloads where it earns nothing are most of ordinary production traffic: classification, extraction, routing, summarisation.

Across the major APIs, tokens a model produces while working through a problem are accounted as output tokens and charged at the output rate, whether or not the text reaches you. The pricing follows the compute: those tokens were generated one at a time through the same decode loop as the answer, so they cost the provider the same and are priced the same. What differs between providers is where the count appears in the response and whether it is already included in the headline output number.

That is a statement about these three APIs at the time they were measured rather than a law, and it is exactly the kind of claim that ages badly. Pricing pages, field names and whether a trace comes back at all have each moved between model generations. What is durable is the reason underneath the rule — those tokens went through the same decode loop, so they cost the provider the same — and the practice that survives a change is reading the usage record of a real response from the model you are calling, rather than trusting a table somebody wrote down last year, this one included.

Which field holds which count#

Three response shapes, all from the same conceptual call:

// Anthropic Messages API
{
  "content": [
    {"type": "thinking", "thinking": "The customer mentions a chargeback…", "signature": "Er4B…"},
    {"type": "text", "text": "Routing to billing-disputes, priority 2."}
  ],
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 1902,
    "output_tokens": 3161,
    "cache_read_input_tokens": 1740,
    "cache_creation_input_tokens": 0
  }
}

// OpenAI Responses API
{
  "usage": {
    "input_tokens": 1902,
    "output_tokens": 3161,
    "output_tokens_details": {"reasoning_tokens": 2751},
    "total_tokens": 5063
  }
}

// Google Gemini
{
  "usageMetadata": {
    "promptTokenCount": 1902,
    "candidatesTokenCount": 410,
    "thoughtsTokenCount": 2751,
    "totalTokenCount": 5063
  }
}

Read those carefully and the trap is visible. On the first two, the reasoning tokens are already inside output_tokens — 3,161 is 2,751 of thinking plus 410 of answer, and the OpenAI reasoning_tokens field is a breakdown line rather than an additional charge. On the third, candidatesTokenCount is the visible answer alone and thoughtsTokenCount sits beside it; a cost function that multiplies candidatesTokenCount by the output rate under-reports by a factor of nearly eight on this workload. A team that ports its cost function from one provider to another by renaming fields will make this mistake in one direction or the other, and only one direction produces an invoice surprise.

The column that costs people most is the third. Where a provider returns a summary of the model's reasoning rather than the reasoning itself, you are billed for the tokens the model generated, not for the summary you received. A 2,751-token thinking pass condensed into a 180-token summary is a 2,751-token charge, and accounting built by counting the characters that arrived over the wire will under-count by fifteen times. The Extended thinking documentation is explicit about this, and the billing section is worth reading once rather than inferring.

The belief that does the damage here is that a token you cannot read is a token you were not charged for, and it is not a careless belief. It is the ordinary rule almost everywhere else: a response body you never receive is bandwidth nobody billed you for, and a log line dropped inside the process costs nothing. What breaks the comparison is that the charge is for generating the token, and generation had already finished by the time anything decided to withhold it. A summary is produced from the thinking pass, after the thinking pass, so it sits downstream of the entire cost rather than being a cheaper substitute for it. The quantity to price is the one the provider reports, never the one you can count on the screen.

The same accounting affects a limit you have probably already set. Where thinking is folded into the output total, max_tokens bounds the sum of thinking and answer, so a request configured with max_tokens: 1024 and a thinking budget of 8,000 is malformed, and a request with max_tokens: 4096 and a thinking budget of 3,500 leaves the model 596 tokens for its actual reply. Teams migrating an endpoint frequently carry over a max_tokens value that was generous for an instruct model and is now a ceiling the thinking pass alone can reach, producing truncated answers that look like model failures. Record stop_reason for that reason: a run where four percent of calls come back on a length limit is telling you about your configuration rather than about the model's ability.

Two further fields belong in the same record even though they have nothing to do with reasoning. Cached input is priced differently from fresh input, where prompt caching means the provider has stored the leading portion of a request it has served before and charges less to reuse it — typically around a tenth of the base rate on a cache read, and somewhat above the base rate on a cache write — so a workload with a large stable system prompt has a real input cost far below its input_tokens figure. Because reasoning workloads tend to carry long instructions and tool schemas, the cache discount is often large enough to change which option wins, and a comparison that prices every input token at the list rate will systematically flatter whichever model is used without caching.

Six numbers per call, and one fixed task set#

Every call records the reasoning tokens as the provider reports them, the visible output tokens, the input tokens split by cache status, wall-clock latency, time to first visible token, and a quality score from a grader. Anything less and you will be arguing from anecdote within a week.

from dataclasses import dataclass, asdict
import time, json

@dataclass
class CallRecord:
    task_id: str          # stable id from the fixed task set
    model: str
    effort: str | None    # provider's effort or budget setting, verbatim
    input_tokens: int
    cached_input_tokens: int
    reasoning_tokens: int  # provider-reported, 0 if the model has none
    visible_output_tokens: int
    ttft_ms: float
    total_ms: float
    stop_reason: str
    score: float | None = None   # filled in by the grader, later

PRICES = {  # USD per million tokens
    "claude-sonnet-4-5": {"in": 3.00, "cached_in": 0.30, "out": 15.00},
    "claude-haiku-4-5":  {"in": 1.00, "cached_in": 0.10, "out": 5.00},
}

def cost_usd(r: CallRecord) -> float:
    p = PRICES[r.model]
    fresh = r.input_tokens - r.cached_input_tokens
    return (
        fresh * p["in"]
        + r.cached_input_tokens * p["cached_in"]
        + (r.reasoning_tokens + r.visible_output_tokens) * p["out"]
    ) / 1_000_000

The line that matters is the last term in cost_usd: reasoning and visible output are summed and multiplied by the same rate, because that is what the bill does. Write the adapter that fills CallRecord once per provider and put the field-mapping decisions there, where they can be tested against a real response, rather than scattering usage["output_tokens"] through your analysis code.

def from_anthropic(resp, t0, t_first, task_id, effort) -> CallRecord:
    u = resp.usage
    thinking = sum(len(b.thinking) for b in resp.content if b.type == "thinking")
    return CallRecord(
        task_id=task_id, model=resp.model, effort=effort,
        input_tokens=u.input_tokens + u.cache_read_input_tokens,
        cached_input_tokens=u.cache_read_input_tokens,
        # output_tokens already includes thinking on this provider
        reasoning_tokens=u.output_tokens - _visible(resp),
        visible_output_tokens=_visible(resp),
        ttft_ms=(t_first - t0) * 1000,
        total_ms=(time.perf_counter() - t0) * 1000,
        stop_reason=resp.stop_reason,
    )

That reasoning_tokens line is a derivation rather than a reported figure, and it is worth a comment in your own code saying so. Where a provider gives you the number directly, use the number. Where it does not, subtracting a visible-token count you computed yourself from the reported output total is the documented proxy, and it is close enough for cost work as long as you record that it is a proxy — otherwise someone will later compare it against a directly reported figure and be confused by a small systematic gap.

The other half of the instrumentation is the task set, and one property matters more than its size: it is fixed. Fifty to two hundred real requests, drawn from production traffic, frozen in a file, with an expected output or a rubric attached to each. Every configuration runs the same items in the same order. This sounds obvious and is routinely violated, usually by sampling fresh traffic for each run, at which point a three-point accuracy difference between two models is indistinguishable from the difference between last Tuesday's tickets and this Tuesday's. A fixed set also lets you look at per-item deltas rather than aggregate means, which is where nearly all the useful signal in this topic lives.

Per-item deltas earn that place for a reason specific to what is being measured here. A mean taken over a fresh sample can move for two reasons at once — the configuration changed, and the items changed — and nothing inside the mean separates them. Hold the items fixed and one of those two causes is gone by construction, so a difference has one explanation left rather than two. It is also what makes the question the rest of this topic keeps asking answerable at all: not whether the score went up, but which items moved, in which direction, and whether the ones that got worse are the ones you can afford to lose.

Two latency numbers are recorded rather than one because they answer different questions and, on this class of model, they diverge sharply. Total wall-clock time is what a batch job and a billing forecast care about. Time to first visible token is what a user staring at a cursor experiences, and because the thinking pass is generated before any answer text exists, a call can be silent for eleven seconds and then stream 410 tokens in under two. Record both from the same clock, started before the request is dispatched so that connection setup and queueing are inside the measurement, and store raw milliseconds per item rather than a pre-aggregated mean, since you cannot recover a p95 from an average.

Grading has to produce a number per item, and how you get it depends on the task. Exact match against a stored answer works for classification and extraction. A unit test or a schema validation works for code and structured output. A model-graded rubric works for open-ended text, with the caveat that the grader has its own error rate and a sample of its judgements should be hand-checked before a two-point difference is trusted. Record the score on the same CallRecord as the tokens, because the question this topic answers is whether score moved enough to justify what the tokens and the milliseconds cost, and that is unanswerable if the two live in different systems.

What the numbers will not explain#

A harness like this reports what happened; it does not explain it and it does not choose for you. It cannot tell you why a model spent 2,751 tokens on one item and 340 on another, because the intermediate text is either withheld or summarised and, where it is shown, it is a rendering of the model's process rather than a faithful log of it. Treating a returned thinking block as a debuggable trace of the computation leads to hours spent reading plausible prose that does not causally determine the answer you got.

Nor is this an evaluation framework in the sense your machine-learning colleagues mean. There is no held-out split, no statistical testing, and no guard against tuning against the same two hundred items until they stop representing production. It is cost and latency accounting with a quality column bolted on, which is exactly what is needed to decide whether a configuration is worth its bill and is not enough to claim a model is better in general. If the decision is which model family to standardise on for a year, this harness is an input rather than the answer.

The accounting rule underneath all of it is one sentence: tokens generated while the model works are output tokens, at the output rate, however little of that text you are shown. Everything else here follows from taking that seriously — summing reasoning and visible output in the cost function, checking whether your provider folds the two into one figure or reports them separately, sizing max_tokens above the thinking budget rather than beside it, and pricing cached input at its own rate so the comparison is not decided by an artefact of how each arm was configured.

The measurement so far has been taken at one setting. The interesting question is what happens when the effort or budget moves — whether those 2,751 tokens were necessary, whether a quarter of them would have bought the same score, and whether the answer is the same for every item in the set. That needs precisely the six numbers above, recorded per item, at more than one setting.