A defensible embedding quote is a range built from a stratified sample

The invoice was $47,113.28. The quote had been $18,000, written on a Friday afternoon from a sample of 5,000 documents drawn at random out of a 12 million document archive of technical manuals, contracts and support exports. The sample averaged 15,000 tokens per document, which gave 180 billion tokens across the corpus, which at $0.10 per million gave $18,000, and the finance partner had approved it on the strength of a single line of arithmetic that was correct in every step. Nine weeks later the engineer who wrote it was in a meeting explaining a 2.6x overrun.

The first instinct in that meeting is almost always retries. The job had logged tens of thousands of 429s — the HTTP status a provider returns when requests arrive faster than the account's rate limit allows — and the story writes itself: every rejected request was billed, the client retried each one three times, and the token spend multiplied. It is a satisfying explanation and it is wrong on the mechanics. A request rejected for rate limiting never reaches the model, so there is nothing to bill, and a retry after a 429 sends the same tokens once rather than twice. Genuine partial failures do cost real money — a 500 returned after the provider has already embedded half a batch, a client timeout on a request that completed server-side and gets resubmitted — and those are worth an allowance of two or three percent. They do not produce a 2.6x. Reaching for retries first means spending the meeting on a line worth $900 while the $28,000 sits unexamined.

Reconstructing the miss from the ledger#

The job logged token counts per request, so the attribution is arithmetic rather than speculation, and it goes in a fixed order because each step isolates one factor.

Start by reconciling the total billed tokens in your own logs against the provider's usage report for the same period. If they agree, the miss is in the estimate rather than in the execution — no duplicate submissions, no second job sharing the key, no price tier you did not expect. In this case they agreed at 470 billion tokens against a predicted 180 billion.

Then divide the realised totals two ways. Billed tokens over document count gives tokens per document, which came out at 37,900 against the sample's 15,000. Billed tokens over chunk count, compared against the raw extracted text length, gives the pipeline's own overhead. Those two ratios separate "the documents were bigger than we thought" from "the pipeline sent more tokens than the documents contain", and they have completely different remedies.

A stratified sample would have caught the third line and most of the first. Stratifying means splitting the corpus into groups you expect to behave differently, then drawing a fixed quota of documents from each group and reweighting the results by how large each group is, in place of drawing uniformly at random across the whole archive. Stratifying by content type puts the scanned OCR output, meaning text recovered from page images by optical character recognition, and the non-English contracts into their own strata with their own token-per-character ratios, and stratifying by length guarantees the 400-page appendices are represented in proportion to the tokens they contribute rather than in proportion to how many of them there are. Those two together are 2.06x of the 2.61x.

It would not have caught the chunk overhead, and this is the part worth sitting with. Overlap and instruction prefixes are properties of the pipeline, not of the documents, exactly as chapter 2 established for chunk size. No sample of documents, however carefully drawn, contains any information about them, because they are added after the documents are sampled. The only way to see them is to run the sample through the production chunker and count the tokens in the chunks it emits, rather than counting the tokens in the documents you fed it. Corpus growth is similarly invisible to sampling and needs a growth rate and a run duration instead.

What the quote for 40 million documents looks like#

Now the same question arrives for a new corpus: 40 million documents, mixed formats, and someone wants a number this week. The estimate is built from a sample stratified by document length and content type, sized so that the strata contributing the most token variance get the most sample, and every sampled document is pushed through the production chunker with the production tokeniser.

import math
from dataclasses import dataclass

@dataclass
class Stratum:
    name: str
    n_docs: int          # documents in this stratum, corpus-wide
    n_sampled: int
    mean_tokens: float   # billed chunk tokens per document, from the real chunker
    sd_tokens: float


def stratified_total(strata: list[Stratum]) -> tuple[float, float]:
    total = sum(s.n_docs * s.mean_tokens for s in strata)
    variance = sum(
        (s.n_docs ** 2) * (s.sd_tokens ** 2) / s.n_sampled
        for s in strata
    )
    return total, math.sqrt(variance)


STRATA = [
    Stratum("html help pages", 18_000_000, 900, 1_240, 640),
    Stratum("support tickets", 12_000_000, 600, 480, 310),
    Stratum("pdf manuals", 6_000_000, 1_400, 9_800, 7_400),
    Stratum("contracts", 2_400_000, 1_200, 26_500, 19_000),
    Stratum("spreadsheet exports", 1_200_000, 500, 3_100, 5_600),
    Stratum("scanned / ocr", 400_000, 400, 14_200, 11_900),
]

total, se = stratified_total(STRATA)
print(f"{total/1e9:.1f} B tokens, se {se/1e9:.2f} B")   # 159.9 B tokens, se 1.86 B

Note the sample allocation. The contracts stratum is 6% of the documents and 40% of the tokens, and it has the widest spread, so it gets 1,200 of the 5,000 sampled documents rather than the 300 that proportional allocation would give it. Sampling is spent where the variance is, which for token counts means on the long, irregular formats. The same 5,000 documents drawn uniformly at random would put about 300 documents in the contracts stratum and 50 in the OCR stratum, and the standard error — the typical distance between an estimate built this way and the true total, which shrinks as the sample grows — contributed by contracts alone would be 2.63 billion tokens — larger than the entire stratified estimate's error. Uniform sampling also carries a real chance of drawing zero documents from a small, expensive stratum, in which case the estimate is not merely imprecise, it is missing a category.

Five thousand is a reasonable total, and going much beyond it has poor returns. Sampling error falls with the square root of the sample, so quadrupling to 20,000 documents halves the standard error, taking it from 1.86 to 0.93 billion tokens — an improvement of half a percent on a number whose real uncertainty is several percent for reasons that have nothing to do with sampling. The effort is better spent on making sure every stratum exists and that the chunker used on the sample is the same code path production will run, byte for byte, rather than a notebook approximation of it.

The confidence interval is not the uncertainty range#

The stratified estimate is 159.9 billion tokens with a standard error of 1.86 billion, so the 80% sampling interval is 157.5 to 162.3 billion — plus or minus 1.5%. Quoting that as the range would be a mistake, because sampling error is the smallest source of uncertainty in the number and pretending otherwise is how a quote acquires false authority.

The range comes from named scenarios instead. At $0.10 per million tokens the central estimate is $15,990. A 3% partial-failure allowance takes it to $16,470. A six-week run against a corpus growing 0.8% per week adds 4.9%, giving $17,280 as the figure you actually expect. The upper bound is then built by asking what specifically could be wrong: if the OCR stratum needs a second pass because the text layer is unusable on a third of it, that is $2,400; if the contracts stratum's tail is 25% heavier than the sample suggests, that is $6,400. Quote $16,000 to $24,000, say which scenarios produce the top of the range, and say what you would do if you saw them.

Alongside the token figure the quote carries a storage figure, because 159.9 billion tokens becomes roughly 390 million chunks at a mean of 410 billed tokens each, and 390 million vectors is a standing monthly cost rather than a one-off. It also carries its assumptions in writing: the corpus count and the date it was taken, the tokeniser and its version, the chunker configuration, the list price and its date, and the assumed growth rate. Every one of those is a thing that can move between the quote and the run, and a quote that does not name them cannot be defended when they do.

Pricing at list rate is a deliberate choice rather than laziness. Committed-use discounts, negotiated enterprise rates and the provider's asynchronous batch endpoint at roughly half price are all real and all conditional — the discount may not be signed by the time the job runs, and the batch endpoint's turnaround may not fit the schedule. Quoting at list and noting the available discounts as upside means the number can only move in the direction nobody minds. The reverse, quoting at the batch rate and then discovering the job has to run synchronously, doubles the bill on the first day.

There are also costs in this project that are not tokens. Fetching 40 million documents out of a bucket in another region is egress, the per-gigabyte charge a cloud provider levies on data leaving its network; parsing 6 million PDFs is CPU measured in thousands of core-hours; the OCR stratum needs a vision model or an OCR service priced per page rather than per token, and at 400,000 scanned documents averaging 24 pages, a service at $0.0015 per page is $14,400 — comparable to the entire embedding bill. A quote that answers "what will embedding cost" when the question was "what will it cost to make this corpus searchable" is accurate and useless.

Finally the quote carries a checkpoint. A job that will spend $17,000 should stop at $10,000 and compare realised tokens per document against the estimate, because at that point the miss is visible and the remaining spend is still a decision. What made the $47,000 invoice a bad afternoon was not the overrun; it was that the overrun was discovered by the invoice.

What a quote does not do#

It does not cap anything. A number in a spreadsheet has no relationship to what the job will actually spend unless something enforces it, and the enforcement lives in the job — a token budget it decrements, a hard stop, an alert wired to the provider's usage endpoint.

It also says nothing about how long the work takes or whether it can be done inside the window you have. Tokens divided by price gives a bill; tokens divided by throughput gives a schedule, and those two quantities are constrained by completely different things. The next chapter finishes the cost side by working out the storage line the quote has been carrying as a single number, and after that the question stops being what the corpus costs and becomes how fast a provider will let you push it.