Normalisation in the write path

Unit norm is a promise a particular provider makes at the end of a particular model, and plenty of write paths break it: self-hosted encoders, mean-pooled token vectors, averaged or concatenated vectors, truncated Matryoshka prefixes, mean-centring. This chapter measures the L2 norms of what is about to be stored, catalogues the sources that do not deliver unit vectors, compares the four places normalisation can live, and prices an in-place repair against a re-embed.

Vector arithmetic — read the current version at https://daviddominguez.dev/learn/en/vector-arithmetic/embedding-models

Back to top