Extraction is a run rate, not an invoice, once the corpus churns

The proof of concept is done and the numbers in the write-up are good: a frozen 20,000-document snapshot, $900 of extraction, about six hours end to end, and multi-hop accuracy on the eval set up from 41% to 68%. The production corpus is 1.2 million documents. The line in the funding request reads: "60× the documents is approximately $54,000 of extraction, a one-time cost, against a 27-point accuracy gain."

Two things are wrong with that sentence, and they compound. The corpus changes at roughly 3% of documents per day, which means the $54,000 is not a one-time cost. And the projection is linear in document count, which several of the largest line items are not. Notice what is not wrong with it: the arithmetic. The POC spent $900 on 20,000 documents, the production corpus is sixty times larger, and sixty times $900 is exactly $54,000. Every multiplication in that sentence closes, which is why nobody caught it in review. What fails is the model the arithmetic was performed on, and a correct sum over a wrong model is the hardest kind of estimate to argue with.

Churn turns the invoice into a subscription#

Churn is the share of the corpus that changes in a given period, and at 3% of documents changing per day the expected time for this corpus to turn over completely is about 33 days. Re-extraction is per changed document — a document whose text has changed produces different entities and different relations, and there is no way to know which without reading it again.

That 33 days is one divided by the daily churn fraction, and it is exact for the quantity the cost model needs: the period over which the number of re-extractions performed equals the number of documents in the corpus. It is looser as a statement about coverage. If the changing documents are drawn afresh each day, some of them come up repeatedly and a share of the corpus is still untouched after a month; if the same documents change on a fixed cycle, as the policy example below has them do, most of the corpus is never re-read at all. Either way one corpus-equivalent of extraction work is bought every thirty-three days, and that work rate — not the coverage — is what the bill is made of.

DOCS         = 1_200_000
CHURN_DAILY  = 0.03
COST_PER_DOC = 900 / 20_000          # $0.045, measured on the POC

initial   = DOCS * COST_PER_DOC                       # $54,000
monthly   = DOCS * CHURN_DAILY * 30 * COST_PER_DOC    # $48,600
print(f"initial {initial:,.0f}  monthly {monthly:,.0f}  year one {initial + 12*monthly:,.0f}")
# initial 54,000  monthly 48,600  year one 637,200

Each line there is one multiplication and worth reading as one. The unit cost is the POC's spend divided by the POC's document count. The build is that unit cost times the corpus. The monthly figure is the same unit cost times the number of documents that change in a month, which is the corpus times the daily churn times the days in the month. The year is the build plus twelve of those months.

The first-year extraction bill is around $637,000, of which 92% is refresh. Extraction is therefore a run rate — a recurring monthly cost that continues for as long as the system is in service — rather than a capital line that appears once. That is the shape to carry into any GraphRAG costing conversation: the build is the small number, and it is the number everyone quotes.

Setting that year out as algebra shows why the shape is stable rather than a property of these particular figures. The year-one bill is the corpus size times the unit cost, times one plus three hundred and sixty times the daily churn fraction — three hundred and sixty being the twelve thirty-day months the code loops over. The leading product is the build. Everything in the bracket past the one is refresh. So the ratio of refresh to build is three hundred and sixty times the churn fraction and nothing else: the corpus size cancels out of it, and so does the unit cost. At three per cent a day that ratio is a shade under eleven, which puts refresh a little over nine-tenths of the year and reproduces the 92% above.

That cancellation is the useful part, because of what it says about every lever later in this chapter. Anything that changes the price of one extraction call — a cheaper model, a batch discount, a cached prompt — multiplies the build and the refresh by the same factor and leaves the ratio between them exactly where it was. The bill gets smaller and stays the same shape. Only two quantities change the shape: the churn fraction, and the number of documents you choose to extract at all.

Some of that churn is avoidable and some is not. Documents that change in ways that do not affect their extracted content — a footer date, a re-export with different whitespace — can be skipped by hashing the normalised text and comparing, which on most corpora removes a meaningful share of nominal churn, sometimes a third of it. What remains is real: a revised contract, an edited runbook, a supplier agreement with a new termination date, all of which change the graph.

The churn figure also hides a distribution. If 40% of the corpus is policy documentation on a monthly revision cycle, then 480,000 documents are re-extracted in a burst every cycle, which is a different operational problem from a smooth 36,000 documents a day: it needs 480,000 extraction calls and the graph work that follows them inside whatever window the business tolerates, and it means the graph is materially out of date for the duration. Both of those figures are the corpus times a fraction — two-fifths of it for the burst, three per cent of it for the daily rate — and the two describe the same annual quantity of work arriving in very different shapes.

The line items the POC never ran#

The POC unit cost of $0.045 per document is extraction only, on a clean snapshot, with no refresh and no verification. Several other lines exist, and three of them are the ones routinely missing from an estimate: retries on malformed output, the community summarisation pass, and re-extraction itself.

Add the monthly column and it comes to a little under fifty-six thousand dollars. That sum is worth doing by hand once, because it is the source of the monthly range quoted later in this chapter rather than an independent guess sitting beside it; the range is the table, added up. Two rows are outside the sum because they are not priced in money: entity resolution's CPU-hours and the Leiden re-run's wall clock. Whatever machine time costs, it sits on top of the figure, so the sum is a floor rather than a midpoint. One further row is outside it deliberately — the flat pipeline is still running either way, so it is not an incremental cost of the graph.

The build column adds up to more than its largest entry, and the gap matters when this table is quoted against the funding request. Extraction is $54,000 of the build. Retries, adjudication and community summarisation are three further build lines, and together they add about a quarter as much again on top of it.

The summarisation line is the one that surprises people, because it is invisible at POC scale. Twenty thousand documents produce a graph with maybe 900 leaf communities; 1.2 million documents produce something in the region of 45,000 leaf communities plus 6,000 at the next level up, and each of those is a model call reading the nodes, edges and source snippets it contains. That is not 60× the POC's summarisation cost, it is more, and it recurs.

Reading those two community counts as a gap is what makes that claim make sense, because on the face of it the counts move the wrong way. The document count grew by sixty and the leaf community count grew by about fifty, so the graph is producing fewer communities per document than the POC did — and yet the cost grows by more than sixty rather than less. The reconciliation is that a summarisation call is not a fixed price. Its cost is the number of calls multiplied by the tokens each call reads, and both moved. Fewer communities per document means each community holds more nodes, more edges and more source snippets, so every call reads more than a POC call did. Then the second level of the hierarchy adds roughly an eighth again in calls on top of the leaves. A count that grew by about fifty, times a per-call size that grew by more than a fifth, clears sixty without difficulty.

How one revision propagates through the graph#

The tempting mental model is that a changed document is a changed row: re-extract it, replace its edges, done. Graph edges are not owned by one document in that sense, and the effects propagate.

Three distinct consequences, none of them confined to the revised document. Removing an edge can orphan a path that other answers depended on, and nothing in the system knows which answers those were. Adding an edge can contradict an edge asserted elsewhere — the revised policy says approvals now route through Legal, an unrevised runbook says they route through Finance, and the graph now holds both with equal authority until something adjudicates. And a node whose degree changes can move between communities, which invalidates the summaries of the community it left and the one it joined, at every level of the hierarchy those communities participate in.

The third of those is the one that breaks locality, and the causal chain under it is short enough to follow all the way. A community is not a label stored on a node; it is the output of an algorithm that partitions the whole graph at once, by asking which grouping of nodes leaves the most edges inside groups and the fewest between them. That question cannot be answered about one node in isolation, because the answer depends on where its neighbours ended up, and where they ended up depends on theirs. So changing the degree of one node changes the input to a global optimisation, and a global optimisation is entitled to return a different answer anywhere. The summary of a community is written from that community's membership. Change the membership and the summary describes a set that no longer exists — which is why a local edit produces non-local staleness, and why no amount of care about which edges you touched bounds which summaries went wrong.

The last of these is why a corpus on a monthly revision cycle makes each cycle a partial rebuild rather than an update. Community detection is a global optimisation over the graph; run Leiden again on a graph where 40% of documents' edges have changed and the partition does not shift slightly, it shifts enough that a large fraction of summaries no longer describe their cluster. Nothing in a summary marks it as stale, which is the same silence chapters 3 and 4 kept running into: the system's failures are all of the form "an answer that reads exactly as complete".

Bounding this is the main engineering work in a production graph pipeline. A workable approach diffs the extraction output against the previous version of the document and touches only what changed, tracks a dirty set of communities by membership delta, and re-summarises only those whose membership changed beyond a threshold, accepting some staleness in exchange for not re-running everything. Incremental graph indexing is an active area rather than a settled one; the Microsoft GraphRAG implementation added incremental update support after the original paper, and the general problem of keeping hierarchical community summaries consistent under a stream of edits does not have a clean answer yet.

Six hours becomes a scheduling problem#

The POC's other headline figure was wall-clock: six hours end to end for 20,000 documents. That number does not scale linearly either, and the way it fails is operational rather than financial. Extraction parallelises well and is bounded by provider rate limits rather than by your own compute, so 1.2 million documents at a sustained 200 requests per minute is around 100 hours of continuous extraction for the initial build. Community detection does not parallelise the same way, because Leiden is a global optimisation over the whole graph, and the summarisation pass that follows it cannot start until the partition is final.

Those two wall-clock figures cover different stretches of the same pipeline and should not be divided by one another. The hundred hours is extraction alone, at a throughput the provider sets rather than one your hardware sets — the corpus divided by the rate, which is what that arithmetic is. The six hours is end to end, extraction plus everything after it. How the POC's six hours split between the two is not recorded, so nothing below leans on it.

The consequence is a pipeline with a serial tail, and the tail sets how fresh the graph can be. If detection plus summarisation takes three hours on the production graph, then no matter how fast extraction runs, the community summaries are at best three hours behind the corpus, and the monthly policy-revision burst pushes that to a day or more. Teams discover this when someone asks why an answer cites a runbook step that was corrected yesterday morning, and the answer is that the graph is on a nightly cycle nobody wrote down.

Two consequences for design. Local search — traversal from seed nodes — can be made near-real-time, because writing new edges is incremental and a traversal reads current edges. Global search over community summaries cannot, and it is worth being explicit in the product about which answers are as-of-now and which are as-of-last-build. And the backlog behaviour under a burst matters: if the pipeline can process 36,000 documents a day and a revision cycle drops 480,000 at once, it is thirteen days behind before it starts catching up, which is a capacity decision rather than a cost one.

Thirteen is a queue depth expressed in days of capacity — the backlog divided by the throughput — and it is the right measure of how far behind the graph is at the instant the burst lands. It is not how long the backlog takes to clear. Clearing means draining a queue that is still filling, so catch-up time is the backlog divided by the spare throughput, which is whatever capacity is left once the ordinary daily churn has been served. The spare is the smaller denominator, so the catch-up time is the larger number, and it is the one that says when the graph can be trusted again. Reporting the depth and calling it the recovery time is how a capacity problem gets signed off as a small one.

The two ways the linear projection is wrong#

The 60× projection assumes every cost scales with documents. Two of the largest do not.

Entity resolution scales with the candidate pairs generated over distinct entity surface forms, and the number of distinct surface forms grows sublinearly with documents while the pairs grow roughly quadratically in the surface forms within each block. Blocking keeps this tractable, as chapter 3 showed, but the blocks themselves get denser as the corpus grows: a corpus with 41,000 company names has bigger prefix blocks than one with 8,000, so the cost per name rises. Going from 20,000 documents to 1.2 million might multiply distinct names by 12 and candidate pairs by 40.

Those two multipliers are consistent with each other, and checking that they are is the clearest way to see what blocking buys. Compare every name against every other and twelve times the names gives roughly a hundred and forty times the pairs, because pairs go as the square. Blocking only compares names that land in the same block, so the pair count is the number of blocks times the square of the names in each. Twelve times the names spread over something like three and a half times as many blocks lands at about forty times the pairs, which is the quoted figure. Divide the pair growth by the name growth and the cost per name has risen by a little over three — each additional entity is more expensive to resolve than the last, which is what a denser block means once it is written as arithmetic. The two company-name counts above illustrate that mechanism; they are not this corpus's own before-and-after.

Summarisation scales with graph size. Communities are a function of nodes and edges, and while nodes grow with documents, they do not grow proportionally — the thousandth document about the same supply chain adds few new entities and many new edges, so the graph gets denser rather than wider. Denser graphs produce larger communities, larger communities produce longer summaries with more input tokens per call, and the hierarchy gains a level or two.

The direction of the two errors is the same, which is why "60× the POC" is optimistic rather than merely imprecise. A realistic production estimate on this corpus lands near $54,000 to build and $55,000 to $60,000 a month to keep, or somewhere around $700,000 in year one, against a flat pipeline whose ingestion is embeddings at roughly $0.0002 per document and whose refresh cost is proportionally trivial. The build figure in that sentence is the extraction line, carried across unchanged so it can be set beside the funding request's own number; the table's three other build entries sit on top of it. The monthly range is the table's monthly column, added up. And the per-document comparison at the end is the sharpest of the three: extracting a document costs more than two hundred times what embedding it costs, which is why the flat pipeline's refresh never appears in anybody's funding request in the first place.

Bringing the number down#

The levers, roughly in order of how much they return:

Extract over the document classes that carry relations, not the whole corpus. Chapter 5's argument applies inside a corpus as well as between corpora — if 300,000 of the 1.2 million documents are meeting notes and chat exports that produce almost no durable edges, excluding them cuts extraction by a quarter with a measurable, and usually small, cost in coverage.

Use a smaller model for extraction with a sampled quality check against the larger one. Extraction is a constrained task with a declared schema, which is where smaller models do best; a 3–5× unit cost reduction is common, and the audit sample from chapter 4 is already the instrument for deciding whether the drop in edge precision is acceptable.

Batch the refresh. Asynchronous batch processing, where a large set of requests is submitted for completion within a window of hours instead of being answered one at a time, is typically half the price of synchronous calls, and re-extraction has no latency requirement, so the entire monthly run rate can go through it.

Cache the schema prompt. The extraction system prompt carrying the schema and type descriptions runs 400 to 1,200 tokens and is identical on every call, and prompt caching — where the provider keeps the processed prefix and charges a fraction of the usual input rate for it on subsequent calls — cuts its cost substantially across a million-call run.

Skip unchanged content by hashing normalised text, and diff extraction output rather than replacing wholesale, so that the graph work following a re-extraction is proportional to what actually changed.

Together these move the monthly figure by a factor of three to five, which changes the decision from "obviously not" to "depends on the gain".

Four levers that each sound large combining into a factor of four is not a disappointment, and seeing why decides which one to reach for first. Re-extraction is a little under seven-eighths of the monthly bill. The smaller model and the batch discount apply to that seven-eighths and to nothing else, and multiplied together they are worth something between six and ten times. The remaining eighth — summarisation, verification, adjudication, storage — is untouched by either, so it stays exactly where it is. Divide the seven-eighths by, say, eight, add back the untouched eighth, and what is left is a little under a quarter of the original bill: a factor of about four, sitting in the middle of the quoted range. The eighth is the cap. Even an extraction call that became free would leave a bill an eighth of the size, so nothing done to that call alone can beat a factor of eight — and the first lever in the list is the only one that shrinks both parts, because dropping 300,000 documents removes their entities from the graph as well as their extraction calls.

What the hashes and the diffs quietly decide#

Two of the levers above are defences, and both are the kind that hold a number still while the thing they are defending against moves. The first is the content hash. Skipping a document whose normalised text is unchanged is safe when the normaliser only folds whitespace, because whitespace does not change what an extractor emits. Every further rule added to that normaliser — stripping punctuation, lowercasing, dropping headers — buys a larger share of skipped churn and converts some class of genuine edit into a document whose hash does not move. The skip is silent, the bill goes down, the graph is quietly wrong, and the metric that would have caught it is the one the hash was introduced to reduce.

The second is the extraction diff. Diffing extraction output against the previous version does bound the edge work: only the edges that changed are written. It does not bound the summary work, for the reason set out above — membership is decided by a global partition, so the set of summaries invalidated by an edit is not a function of that edit. A dirty set tracked by membership delta is a heuristic with a threshold in it, and a threshold is a choice about how much staleness to accept, made once, in code, on behalf of every future answer. It is worth writing down as a policy rather than leaving as a constant.

None of this says the graph is not worth building. A $15,000-a-month pipeline that lifts multi-hop accuracy 27 points on questions that drive procurement decisions may be cheap. That monthly figure is the levered one — the table's monthly total divided by a factor inside the three-to-five range lands about there — and quoting it beside an un-levered comparison is its own way of getting the decision wrong. What the chapter says is that the comparison has to be between the graph's total monthly cost of ownership and the flat pipeline's, held against a quality gain measured on the questions users actually ask, and that a projection built by multiplying a frozen snapshot by 60 answers neither half.

That leaves the other half of the decision, which is the accuracy number. Twenty-seven points on an eval set is a claim, and it is a claim of a particular kind: a gap between two measured levels, 41% and 68%, which means it inherits whatever uncertainty each of those levels carries and cannot be more precise than either. A claim of that size deserves the same scrutiny as the cost projection — where the questions came from, what was scored, whether the flat baseline was given the same budget, and where in the question distribution the gain actually sits.