What a language model is

What you will know

• State in one sentence what a language model computes when you send it text

  If the sentence does not mention scoring what comes next, it is a description of the product rather than of the thing.

• Explain why the same request sent twice can come back different

  The answer is in the picking step, not in the weights and not in the provider being unreliable.

• Say where a model's behaviour came from and why your request cannot change it

  Separating training from inference removes a whole class of misconception, including the belief that correcting the model teaches it.

• Send one request to a hosted model and account for every field of the response

  Content, stop reason and usage. Not recognising the stop reason is how a truncated answer gets diagnosed as a model that trailed off.

• Name three things the model cannot know unless the request contains them

  Anything after the training data was collected, anything private to you, and anything said in an earlier call.

Chapters

1. 1

   Scoring the next piece of text

   A language model is a large mathematical function that takes the text so far and returns a score for every piece of text that could come next; a separate step picks one, appends it, and the function runs again on the longer text. This chapter defines that function, walks one step of the loop with a real distribution over candidates, shows the loop in eight lines of Python, and settles the first thing that surprises people — the same request can come back different twice, because the picking step is allowed to be random.

   Read chapter
2. 2

   Weights, training and inference

   The scores from chapter 1 come from billions of numbers called weights, fixed during a training run that finished before the model was released. Training and inference are separate events: training is where the behaviour came from, inference is the request you send, and your request reads the weights without writing to them. This chapter sizes the fixed artefact, contrasts the two events field by field, runs a four-minute experiment that settles the question, and works through what the separation means for corrections, knowledge cutoffs and 'giving the model our data'.

   Read chapter
3. 3

   Stateless calls and conversation history

   A model call is stateless: the provider keeps nothing between calls, and the model begins each one with only what the request contains. A chat that appears to remember is a client re-sending the whole transcript every turn, tagged with who said what. This chapter builds that client in ten lines, works out the arithmetic that makes turn 30 cost sixteen times turn 1, explains what prompt caching does and does not change, and lists the three categories of fact a model cannot know unless the request says so.

   Read chapter
4. 4

   API request and response fields

   One call to a hosted model, accounted for field by field: the model name, the messages array, the ceiling on how much may be generated, and on the way back the content blocks, the reason generation stopped and the usage counts you are billed on. This chapter reads a real request and a real response line by line, prices a single call, shows the four lines of client code worth writing on your first call rather than your hundredth, and separates a truncated answer from an error that never reached the model.

   Read chapter
5. 5

   Plausibility, fabrication and reliability

   A model that continues text plausibly continues it plausibly when it has no basis for the answer, and the wrong answer arrives in the same confident register as the right one. This chapter derives that behaviour from the machinery of the first two chapters, sorts tasks by where the answer actually lives, works through a fabricated regulatory citation and the cost of checking it, and shows the design move that keeps a number that must not be invented out of the weights. It closes by handing off to tokens, the unit everything in this topic was measured in.

   Read chapter