History trimming policies

Dropping the oldest messages when the window fills is a cache eviction policy applied to something with no backing store: the model cannot fetch on a miss, and the opening turns are where users state their constraints. The room for history is also the most volatile line in the budget, so a tool whose schema embeds 3,000 SKUs makes conversations start forgetting ten turns sooner with no code change. Pinning, rolling summaries, extracted state and retrieval over history each keep something different, at a different price.

Context window — read the current version at https://daviddominguez.dev/learn/en/context-window/history-trimming-policies

Back to top