Skip to content
Model APIsReasoning-token mechanics

Senior · 7 chapters

Reasoning-token mechanics

Billed as output, preserving thinking blocks across tool-call turns, budgets interacting with max_tokens, never surfacing raw reasoning

Hidden reasoning affects both cost and control flow, and mishandling it produces expensive, opaque failures in otherwise ordinary tool-call requests.

Read these first

What you will know

  • Measure billed output tokens for a request with and without reasoning enabled, and reconcile the difference with the provider's usage fields

    Hidden tokens can appear only in completion-side accounting, so cost estimates based on visible text understate spend.

  • Preserve thinking blocks across a tool-call turn, and say what breaks if you don't

    Some APIs reject mismatched message sequences, while others silently degrade the model's ability to continue its prior plan.

  • Predict the interaction between a thinking budget and `max_tokens`

    Provider-specific caps may combine hidden deliberation and visible output, making an apparently sufficient answer limit fail after internal work consumes it.

  • Run a response through a user-facing serializer and verify raw reasoning fields are excluded while tool arguments and results remain available

    Reasoning payloads may contain sensitive or unpolished material, whereas removing tool data can make the application impossible to audit or continue.

  • Debug a truncation caused by reasoning consuming the output budget

    Compare configured limits with usage metadata and finish reasons to distinguish hidden-token exhaustion from prompt length, stop sequences, or transport truncation.

Chapters

  1. 1

    Reasoning tokens

    A model in a reasoning mode generates a stretch of intermediate text for itself before it writes the reply, and those tokens are billed at the output rate even though nothing renders them. This chapter covers where the count appears in each provider's usage payload, how to measure it by differencing when nobody reports it, what the hidden-to-visible ratio does to a cost forecast, and why the number is not one you set.

    Read chapter
  2. 2

    The thinking budget

    Where max_tokens covers thinking plus visible output, a thinking budget is a ceiling on deliberation drawn from the same pool as the answer, so the answer is only guaranteed the difference. Reasoning length and answer length rise together on hard inputs, so an under-sized limit truncates exactly the requests that mattered. This chapter covers the pool arithmetic, the validation error that teaches the semantics, and how to size both limits from a measured p99.

    Read chapter
  3. 3

    Reasoning allowance regimes

    Whether thinking is drawn from the same allowance as visible output is a property of the API, not a universal: some providers reject a budget larger than max_tokens, some accept it and bill a budget max_tokens never bounded, and newer families drop the integer entirely for an effort level. This chapter gives three probes that settle which regime you are on, the sizing rule for each, and a configuration shape that cannot be misread across adapters.

    Read chapter
  4. 4

    Diagnosing an empty completion

    A response with no visible text has four plausible causes that look identical from the application and quite different in the payload: reasoning that consumed the whole allowance, a prompt too long for the window, a stop sequence, and transport truncation. This chapter reads a captured response, separates the four in one comparison, explains why a repointed model alias shifts the reasoning distribution, and gives two alerts that fire on the mechanism rather than on latency.

    Read chapter
  5. 5

    Thinking blocks in the tool loop

    With reasoning enabled, an assistant turn that calls a tool must be replayed verbatim on the next request — thinking block with its provider-issued signature first, then the tool-call block, in the order they arrived. Some providers reject a mismatched sequence with a 400; others accept it and let the model re-plan from the tool result alone, which costs 50% more tool calls and seventeen points of task success with nothing to grep for.

    Read chapter
  6. 6

    History compaction and task boundaries

    A history compactor that filters conversation blocks by type is harmless on chat turns and fatal on tool turns, so staging passes and production returns 400s from long conversations. This chapter shows why the window has to be bounded by the task boundary rather than by a turn count, replaces the whitelist with a denylist, and separates the security rule about persisting raw reasoning from the question of holding it in memory for the duration of one task.

    Read chapter
  7. 7

    Reasoning on human-facing surfaces

    Three block types come back from a reasoning model that uses tools, and they have three different relationships to the people downstream: text is the answer, tool calls and results are the record of what the assistant did, and deliberation belongs on no human surface. This chapter fixes a serializer that keeps the wrong one, a stream handler that renders the wrong one, and the proposal to store reasoning text as an explanation.

    Read chapter

External resources

Start reading