Mechanical character work in code

Redacting card numbers, truncating to a byte limit and validating an identifier against a pattern are exact operations over characters, and a prompt performs them approximately at latency and cost. This chapter sets out the code versions, compares them with the prompt versions on recall, determinism, latency, cost and testability, and gives the propose-and-check loop for the harder case where the model has to generate text under a mechanical constraint. It ends on the division of labour that holds up: the model returns spans and judgements, the code performs the transform.

What tokenization actually breaks — read the current version at https://daviddominguez.dev/learn/en/what-tokenization-actually-breaks/mechanical-character-work-in-code

Back to top