Context overflow and silent truncation

A request that exceeds the window is either rejected with an error naming the count, or quietly shortened by whichever layer of the stack was written to be convenient. The first is a discrete event you can catch, count and act on; the second returns a fluent, confident answer computed over content you believe you sent, and nothing in the response distinguishes it. Establish which behaviour each backend has, normalise to the loud one by checking sizes yourself, and compare the provider's reported input tokens against your own count.

Context window — read the current version at https://daviddominguez.dev/learn/en/context-window/context-overflow-and-silent-truncation

Back to top