Frequency and presence penalties

Frequency penalty subtracts from a token's logit in proportion to how often it has already appeared; presence penalty is a flat one-off charge for having appeared at all. Both suppress repetition, and both do it by pushing the model away from words it has good reason to reuse — entity names, order numbers, figures — which is how a repetition metric improves several points while factual accuracy falls. This chapter covers the arithmetic, how to find the cause of a loop before suppressing it, and the workload shape where a penalty is the right tool.

Sampling — read the current version at https://daviddominguez.dev/learn/en/sampling/frequency-and-presence-penalties

Back to top