Chain-of-thought on character tasks

Asking the model to list a word's characters one per line before answering raises the hit rate on counting and reversal, because the characters become tokens in the context that later tokens can attend to. That is a real mechanism with three limits: the enumeration is recalled rather than read, so its accuracy caps the whole procedure; it must come before the answer, not after; and it buys a probabilistic result for questions that often have an exact one. This chapter measures the gain, the decay with string length, the token cost, and the four failure classes the technique is applied to.

What tokenization actually breaks — read the current version at https://daviddominguez.dev/learn/en/what-tokenization-actually-breaks/chain-of-thought-on-character-tasks

Back to top