Effective context length

A prompt can sit comfortably under the limit and still produce a worse answer than a short prompt containing the same relevant passage, because accuracy declines as material is buried among near-miss distractors. The published benchmarks — NoLiMa, Context Rot, and the effective-length work — agree on the shape and not on the numbers, so the length at which your task stops being answered acceptably has to be measured on your own corpus. Fix the task, fix the scoring, pad with your own retriever's second-best passages, and read the working limit off the curve.

Context window — read the current version at https://daviddominguez.dev/learn/en/context-window/effective-context-length

Back to top