Truncation settings under greedy decoding

Setting temperature to 0 makes the sampler take the highest-scoring token, and the highest-scoring token is inside every nucleus and every top-k set, so the truncation parameters sitting beside it cannot change a single output. The misreading that keeps the pairing alive is that p is a share of the vocabulary rather than cumulative probability mass. This chapter proves the inertness, catalogues the other settings that are quietly doing nothing, and shows the two-request test that confirms it.

Sampling — read the current version at https://daviddominguez.dev/learn/en/sampling/temperature-zero

Back to top