Generating distinct options

Raising temperature makes a single completion less predictable; it does not make ten items in one list differ from each other, and past a point it buys incoherence rather than variety. Tighten the nucleus to cut the low-probability tail that produces nonsense, keep enough temperature for movement, and get distinctness structurally through separate calls, an enumerated prompt or oversampling with a similarity filter. This chapter also covers which of these settings can safely be exposed outside engineering, and which cannot.

Sampling — read the current version at https://daviddominguez.dev/learn/en/sampling/generating-distinct-options

Back to top