Reasoning models

Read these first

• Base vs instruct vs reasoning vs multimodal variants

What you will know

• Instrument calls to record provider-reported reasoning tokens or a documented proxy, visible output tokens, latency, cost, and quality on a fixed task set.

  Usage fields and billing semantics differ across providers, so totals that omit hidden or uncached tokens can produce a false cost comparison.

• Set a reasoning-effort or thinking budget and observe the trade

  The same nominal setting can produce nonlinear changes in answer quality, tail latency, and billed tokens across task types.

• Break a reasoning-model workflow with an intentionally tight token or latency budget, then trace whether truncation, timeout, or degraded verification caused the failure.

  A short response may indicate an incomplete internal budget rather than a successful solution, and deadline handling is often separate from generation limits.

• Route a mixed workload between reasoning and non-reasoning models using measured quality, cost, and latency thresholds, and test the fallback when a budget is exceeded.

  Routing rules need an explicit failure path because a request can exhaust its thinking allowance without producing an answer that downstream code can safely use.

• Decide when a cheaper non-reasoning model plus a verifier beats a reasoning model

  A verifier must cover the actual error modes and its own execution cost; otherwise the apparent savings or correctness gain is illusory.

Chapters

1. 1

   Reasoning models and token accounting

   A reasoning model writes out its working before it answers, and those tokens are billed at the output rate whether or not the text ever reaches you. This chapter defines extended thinking and reasoning tokens, shows where each major provider reports the count and whether it is already inside the headline output figure, and builds the six-number-per-call instrumentation the rest of the topic depends on.

   Read chapter
2. 2

   Reasoning effort and thinking budgets

   Sweeping the reasoning-effort or thinking-budget dial across a task set banded by difficulty shows three different functions, not one: hard items gain enormously, routine items gain nothing while spending six times the money and four times the tail latency. This chapter covers how to band a task set, why the aggregate hides both the gain and the cost, why nominal effort levels are not comparable between providers, and where more thinking makes accuracy worse.

   Read chapter
3. 3

   Deliberation and format precision

   Errors are not a homogeneous quantity that a better model reduces proportionally. This chapter classifies the field errors of a twelve-field extraction service, shows that 87% of them are rendering rather than deciding, computes the ceiling that puts on any deliberation-based fix, and covers what does move format precision — constrained decoding, worked examples and normalisation — along with the audit to run when a reasoning migration reports an accuracy jump anyway.

   Read chapter
4. 4

   Exhausted budgets and client deadlines

   One bar labelled TimeoutError hides at least three different failures: a deadline that fired while generation was healthy, a thinking allowance consumed before any answer existed, and a budget that ran out during the model's own checking pass so an answer was produced but never verified. This chapter separates them with stream-level instrumentation, works out the decode-rate arithmetic that makes a budget reachable inside a deadline, and defines what a fallback is allowed to hand downstream.

   Read chapter
5. 5

   Verifiers and retry loops

   Where checking an answer is much cheaper than producing one, a cheap model wrapped in a verifier and a retry loop can match a reasoning model at an eighth of the cost. This chapter works the arithmetic of that loop, compares a deterministic verifier against a model-graded one on the same task, and shows why the design's value is decided entirely by the overlap between what the verifier checks and the errors the generator actually makes.

   Read chapter
6. 6

   Routing requests across model tiers

   Prompt length correlates with difficulty at 0.09, so a length-based router is a random assignment with extra steps. This chapter covers the three signals that are actually available before the expensive call — the task type the calling surface already knows, a cheap classifier, and a verifier-driven cascade — sets quality, cost and latency thresholds per class from measured runs, and lists the four findings that would justify collapsing the whole arrangement to one model.

   Read chapter