Per-page cost is not the provider's per-image price

Sending a page to a vision model has a price, and providers publish it in the most convenient possible form: one figure for one page-sized image. That figure is real and it is quoted honestly. What it is not is the cost of running a page through an extraction path, and the gap between the two is not a small margin to pad. On a corpus of archived filings it is a factor of about three, and the factor does not shrink as pages get simpler.

Per-page cost, for the rest of this chapter, means every unit of spend a single page causes from the moment it leaves a document store to the moment it lands in an index: the compute that turns it into an image, the tokens that image becomes, the tokens the model emits, whatever a failed attempt costs, and whatever checking the result costs. Wall clock means how long a fixed number of pages takes to get through that path end to end. The two are independent quantities. They are worth deriving separately, because a fix aimed at one of them routinely does nothing at all to the other, and a post-mortem that treats them as one problem produces a plan that fixes one of them.

Every line the per-image price does not include#

Cost per page is a sum, and every term in it is a product. That structure is worth stating before any figures, because it is what makes the arithmetic checkable. Providers price tokens linearly: a call that consumes twice the tokens costs twice as much, with no fixed fee per call and no volume step inside a single request. Linearity is the property that licenses multiplication — when unit price does not depend on quantity, total cost for one line is simply count times unit price, and nothing more elaborate is needed. And because a page's terms are incurred independently of each other, the page's total is their sum rather than their maximum or their product.

That gives a recipe with one line per thing that gets billed. Here it is, filled in from measurement on a four-hundred-thousand-page backfill of archived filings onto the vision path.

Take the first row apart, since every token row is built the same way. The unit MTok is a million tokens, so a price of $3/MTok is three dollars for a million tokens, or three millionths of a dollar for one. A page that arrives as 1,850 image tokens therefore costs 1,850 times that, and multiplying a token count by a per-token price is the right operation precisely because of the linearity above. Doing it gives the $0.00555 in the cell. The second row is the same multiplication over a different count, at the same input price, because instructions and images are both things the model reads. The third row is the same multiplication again at the output price, which is five times the input price on this provider — $15/MTok against $3/MTok — because generating a token costs a provider more than ingesting one.

The fourth row is a different shape and it is the one people get wrong. A retry is not a component sitting alongside the others; it is a second occurrence of all of them. When 4.1% of pages are attempted twice, the extra spend is 4.1% of everything already counted above that line, which is why the retry cell is computed against the three token rows rather than against a price list of its own. Four point one per cent of those three rows lands on the figure in the cell, and the reason the retry line is small relative to the rows it multiplies is simply that a few per cent of a large number is a small number.

Those six rows close. Multiply each per-page figure by four hundred thousand and five of the six reproduce their backfill total to the cent; add the backfill column downwards and it lands exactly on the $10,613 in the last row, which is the invoice. The one residual is worth naming rather than smoothing over. The per-page total, multiplied out, gives a dollar less than the column sum, because the retry cell is the only one whose true value does not terminate at five decimal places — the exact 4.1% carries further, and the table shows it rounded. That single truncation, multiplied by four hundred thousand pages, is the whole of the discrepancy. A column that closes to within one rounded cell licenses something specific: the table is not six independent estimates that happen to sit near a known total, it is one arithmetic with six measured inputs, so a proposed change to any input can be priced by rerunning the sum rather than by rerunning the backfill.

Output tokens are the largest line by a distance, and they are the line the per-image price cannot possibly include, because the provider does not know what you are going to ask for. A full-page transcription into the schema from chapter 3 — every text block with its role and bounding box, every table cell with a resolved column label, figures with legend entries and series — averages 1,150 output tokens on this corpus, and output is priced at five times input. Two thirds of the bill is the model writing, and the estimate accounted only for the model reading.

That asymmetry is the reason a per-image price can never be extended into a per-page one by multiplication alone. The price is a function of the input, which the provider can measure the moment the image arrives; the bill is dominated by the output, which depends on a schema chosen in your repository. Two teams sending byte-identical images to the same model on the same day can receive bills differing several-fold, and neither pricing page is wrong.

Image tokens came in above the quoted per-image figure for a reason that is worth owning rather than treating as a surprise: chapter 2's fix costs money. Sending a full page plus a region crop on the 22% of pages with small print raises the average from about 1,540 image tokens to 1,850, which is a 20% increase on that line bought deliberately in exchange for legible digits. Quality decisions made upstream land in this table, and a budget produced before those decisions were made was never going to survive them.

The retry rate breaks down as 2.4% schema validation failures where the emitted JSON was well-formed but violated a cross-field rule, 1.2% provider overload responses, and 0.5% image decode errors on pages whose render produced a malformed PNG. Each retry costs a full call, so the 4.1% adds 4.1% to everything above it. The validation pass is a small model scoring the emitted page against the gates from chapters 3 and 5, plus one OCR pass over a figure bounding box; it is cheap per page and it is not zero, and it is mandatory rather than optional, which makes it part of the cost of the path rather than an add-on.

Render and encode is the line people assume is free because it runs on machines they already own. Rasterising a page at 300 DPI, cropping, JPEG-encoding and base64-wrapping it takes about 180 ms of CPU, and running that for nine days on the pool sized for the job came to $180. Small, but it is the line that grows if you raise resolution, and it is the only line that shows up on an infrastructure bill rather than a model bill, which is why it is usually missing from the model-cost conversation entirely.

The steady-state figure that came out of this — $0.0265 per page against the parser's $0.00066, which is dominated by OCR fallback on the 12% of pages with no usable text layer — is the 40× multiple the previous chapter spent.

The belief this table is aimed at is a specific one, and it is tempting for a good reason: the per-image price looks like a measurement of the page, so multiplying it by a page count looks like arithmetic rather than a guess. It is worth seeing exactly why that guess cannot be caught by sanity-checking it. An estimate built from the image price is wrong by roughly the same factor whatever the pages contain, because the term it omits — output — moves with the schema and not with the input. Simplify the documents and the estimate stays wrong by the same factor. Sample more pages and it stays wrong by the same factor. A check whose verdict does not move when the thing it is checking moves is not a check, and this one returns "about right" for a corpus it is understating threefold and for one it would understate tenfold.

Nine days was fixed before the job started#

The wall clock has nothing to do with any of the above. Every page consumes 1,850 image + 640 instruction + 1,150 output = 3,640 tokens, and the organisation's account carried a 120,000 tokens-per-minute ceiling.

A ceiling of that kind is a bucket that refills at a fixed rate, and a page is a withdrawal of fixed size. When a rate of supply meets a fixed cost per unit, the number of units per minute is the one divided by the other — dividing tokens per minute by tokens per page cancels the tokens and leaves pages per minute, which is a check worth doing on any rate calculation, because a unit that fails to cancel means the division was the wrong way round. Dividing the page count by that rate then gives minutes, and dividing minutes by the minutes in an hour and again by the hours in a day gives days.

TOK_PER_PAGE = 1_850 + 640 + 1_150      # 3,640
TPM_CEILING  = 120_000
PAGES        = 400_000

pages_per_min = TPM_CEILING / TOK_PER_PAGE          # 32.97
minutes       = PAGES / pages_per_min               # 12,133
print(minutes / 60 / 24)                            # 8.4 days

# What the plan assumed: bounded by workers, not by the bucket
WORKERS, SEC_PER_PAGE = 48, 4.2
print(PAGES * SEC_PER_PAGE / WORKERS / 3600 / 24)   # 0.41 days

The second calculation in that block is a different model of the same system, and the contrast is the lesson. A worker pool's capacity is the number of workers divided by the seconds each spends on a page, which is a statement about how much work can be in flight. A token bucket's capacity is a statement about how much work is allowed to start. When a system has both, throughput is the smaller of the two, since the constraint that binds first is the one that binds — and no arrangement of the larger one can lift it.

The plan's model of the system was 48 workers each taking 4.2 seconds, which predicts under half a day and got padded to two. The system's actual model was a token bucket refilling at 120,000 tokens a minute, which predicts 8.4 days, and retry backoff after rate-limit responses accounts for the rest of the ninth. The worker pool was never the constraint; for most of those nine days, 45 of the 48 workers were sleeping in exponential backoff.

That count of sleeping workers is not a separate observation; it falls out of the two capacities the diagram already carries. The pool can serve 686 pages a minute and the bucket admits 33, so the share of the pool doing anything at a given moment is the second divided by the first, a little over one part in twenty — which puts between two and three workers busy and the remaining forty-five idle. Recovering the same figure from the diagram and from the narrative, by routes that share no step, is what makes it safe to reason about the pool as effectively stopped rather than merely slow.

The important property of that diagram is that the constriction sits after the worker pool, so every lever the team had — more workers, bigger machines, faster rendering — pushed against a wall. Concurrency limits behave unlike compute limits in exactly this way, and pipelines that have only ever been scaled by adding workers tend to discover it in production.

Whether better token accounting would have helped is worth working out precisely, because it is the plausible-sounding fix. Pages tile into more tokens than a flat per-image price suggests, so re-estimating at the true token count is a real correction — to the money. On the clock, it cuts both ways: fewer tokens per page means more pages per minute through the same bucket. Suppose the schema were trimmed to drop coordinates and the output fell to 600 tokens, taking the page to 3,090 tokens. That yields 38.8 pages a minute and 7.2 days. Even a perfectly lean 1,800-token page, which would mean no schema, no crops and a bare markdown transcription, gives 66.7 pages a minute and 4.2 days. The two-day projection was unreachable under a 120,000 TPM ceiling for any output format whatsoever, which means it was not an estimation error at all. It was a plan built on the wrong model of the bottleneck.

Two of these change the money substantially and one changes the clock substantially, and only the last row changes both, which is the argument the next chapter builds on.

The table is more useful read across than read down, because the interesting thing about it is where its two columns come apart. They would be locked together if cost and time were the same quantity wearing two units, and the first row proves they are not: re-estimating moves neither, since describing a number correctly changes nothing about the system that produced it. Raising the ceiling moves the clock alone, since the account limit is not a price. Batch pricing and caching move the money alone. The two columns agree only on the last row, and that agreement is the finding — the one lever that acts on both is the one that removes pages from the path entirely, because a page not sent costs nothing and occupies no share of the bucket.

Two of the savings can be recovered from the cost table without any new measurement, which is what a closing arithmetic buys. Trimming the schema drops output from 1,150 tokens to 600, a difference of five hundred and fifty tokens; priced at $15/MTok across four hundred thousand pages that is exactly the −$3,300 in the row. Prompt caching takes the instruction line from $768 to about $78, and the difference is exactly the −$690 in its row. Neither number needed a trial run.

One row looks arithmetically impossible and is not. Lowering render resolution is credited with −$400 while the whole render-and-encode line is $180, and a saving cannot exceed the line it comes from. It does not: a lower render resolution produces a smaller image, a smaller image tiles into fewer image tokens, and the balance of that $400 comes off the top row rather than the bottom one. The row is naming a change that touches two lines, and the cost column records only its net.

Prompt caching deserves a note because it is the cheapest item on the list to implement. The instruction text and the JSON schema are byte-identical on every call, so marking them as a cache breakpoint drops their cost to roughly a tenth on cache hits and takes the 640-token line from $768 to about $78. It does not help wall clock, since cached input tokens still count against most rate limits, but it is a few lines of change for 7% of the bill.

Measuring cost per page rather than estimating it#

None of the table above required forecasting. Every provider returns usage counters on the response, and the whole exercise is joining them to a page id at write time rather than reading an aggregated bill at the end of the month.

def record_cost(page_id: str, usage, render_ms: float, attempt: int) -> dict:
    PRICE_IN, PRICE_OUT = 3.0 / 1e6, 15.0 / 1e6
    CPU_PER_HOUR = 0.034
    return {
        "page_id": page_id,
        "attempt": attempt,
        "input_tokens": usage.input_tokens,
        "cache_read_tokens": getattr(usage, "cache_read_input_tokens", 0),
        "output_tokens": usage.output_tokens,
        "model_cost": usage.input_tokens * PRICE_IN
                      + usage.output_tokens * PRICE_OUT,
        "render_cost": render_ms / 3.6e6 * CPU_PER_HOUR,
    }

The two prices in that function are the same $3/MTok and $15/MTok as the table, written as dollars per token by dividing by a million, which is why they appear as 3.0 / 1e6 rather than as decimals nobody can read. The render line converts milliseconds to hours before applying an hourly rate, since 3.6e6 is the number of milliseconds in an hour — again a unit cancellation, milliseconds against milliseconds, leaving hours to meet a per-hour price. The hourly figure this function carries is a deployment constant rather than a published one, and it is set per pool; the render-and-encode line in the cost table above was measured on the pool that actually ran the job, so treat the table as the authority on what rendering cost and this function as the shape of the calculation.

Writing one of these rows per attempt, rather than per page, is what makes the retry line visible at all; a schema that stores one cost per page silently absorbs the 4.1%. Keeping input_tokens separate from cache_read_tokens is what lets you see whether prompt caching is working, since a cache miss looks identical to a cache hit in the aggregate bill.

With per-page rows, the cost distribution is available rather than just its mean, and it is skewed. Median page cost on this corpus is $0.0231, the p95 is $0.0492, and the p99 is $0.117 — a dense landscape table with fourteen columns produces four times the output tokens of a prose page. A budget built on the mean is fine for a backfill and misleading for capacity planning on a customer whose documents are all financial statements, which is a per-source question the same rows answer directly.

The gap between the median and the mean is itself a reading. A symmetric distribution would put them on top of each other; here the mean sits above the median, which says the average is being pulled up by a minority of pages rather than describing a typical one. That is the signature of a corpus in which document type, not page count, is what predicts spend.

Backfill economics also differ from steady state in one direction worth flagging. A backfill runs against a fixed corpus at maximum concurrency, so it is rate-limit bound and can use batch pricing; steady-state ingestion arrives at whatever rate customers upload, is usually far below the ceiling, and cannot wait 24 hours. Numbers measured during a backfill transfer to steady-state cost and do not transfer to steady-state latency.

Tail latency, and where a per-page median stops being the right number#

The 4.2 seconds used throughout this topic is a median. The distribution matters as soon as ingestion has an SLA rather than a completion date.

The p99 of 34 seconds is a page that hit a rate limit, backed off, retried, failed schema validation, and went round again. The p99.9 of 96 seconds is a page that did that twice. A user-facing upload flow that promises a document is searchable within a minute is fine on the parser path and violated on roughly one page in a thousand on the vision path, which for a 200-page document is a coin flip on whether the document as a whole misses the promise.

The step from a rare page to a likely document is where this gets missed, and it is short enough to walk. A document meets its promise only if every page in it does, so the chance the document is clean is the chance one page is clean, multiplied by itself once per page. A probability slightly below one, raised to a couple of hundred, falls a long way — which is how a per-page failure rate small enough to ignore becomes a per-document failure rate you would never agree to in a contract. The parser column of that table never gets near the promise at any percentile, so the same compounding does nothing there, and that difference between the two columns is the whole of the routing case on latency grounds. Reporting only the median hides this entirely, and the median is what a benchmark script prints by default.

None of this is an argument against the vision path. $0.0265 a page is unremarkable for the pages where it wins 19 points, and the previous chapter showed the routed configuration buying those wins for $885 a month. The overrun was in the estimate, not in the technology. The 19 points and that chapter's 14-point corpus average are two different measurements and both stand: one is the margin on the pages where the vision path wins, the other is the average across all pages including the great majority where it wins nothing, and a routed configuration exists precisely to buy the first without paying for the second.

Nor is it an argument against per-image pricing, which is an honest simplification of a genuinely complicated tokenisation — the image-inputs topic covered how a page tiles — and which is accurate for what it covers. It covers input. The invitation to multiply it by a page count comes from the reader rather than from the pricing page.

And the diagnosis is not "we needed a better spreadsheet". A spreadsheet with every line of the cost table in it would still have projected two days, because the concurrency ceiling is not a cost input. The two misses are independent, they have independent fixes, and a post-mortem that lands only on token arithmetic will produce a corrected budget attached to an uncorrected schedule.

What a page costs is a sum you can write down before you spend anything, and the per-image price is one row of it. Six rows, each a count times a unit price, with a retry term that multiplies the rows above it rather than adding beside them — that is the whole model, it closes against the invoice, and it prices a proposed change without running one. What a backfill takes is a different sum entirely, computed from a refill rate and a withdrawal size, and it is indifferent to every one of those prices.

Both fixes point the same way. Sending 6% of pages instead of 100% cuts the bill by 94% and the wall clock by 94%, without trimming a schema or negotiating a rate limit. What that requires is a rule that decides, per page, which path runs — and a rule that can be checked afterwards, because a router that quietly sends the wrong pages somewhere is worse than no router at all.