Copying an ACL onto every chunk makes revocation a rewrite

An access control list, usually shortened to ACL, is the enumerated set of identities permitted to read something. A retrieval index stores chunks, and a chunk needs some way for the query path to tell whether the caller may see it, so one obvious design copies the source document's ACL into each chunk's metadata. That copying is denormalization — holding the same fact in many places so that the place which reads it does not have to go and ask — and it is a well-understood trade with a well-understood direction: reads get cheaper and writes get more expensive.

What makes this instance of the trade worth a chapter is that the write in question is a revocation, and a revocation is the one write a company signs contracts about. The requirement is that the query path can decide, cheaply, whether a caller may see a chunk. Materialising the answer at ingest time is one design that satisfies it, and the cost it moves onto writes is not a general inefficiency but a delay on exactly the operation with a deadline attached.

What a revocation actually costs here#

At ingest, each chunk's metadata carries a copy of its source document's ACL, so the query path needs nothing but the caller's own id:

{
  "id": "d/9f2a41/c12",
  "text": "Termination for convenience requires 60 days written notice...",
  "metadata": {
    "doc_id": "9f2a41",
    "folder": "legal/msa-templates",
    "acl": ["u_0f31", "u_2b7c", "u_44a9", "u_51e0", "u_6d12", "u_7f8b", "..."]
  }
}

The query path is a single predicate — acl contains u_0f31 — which is fast, selective and requires no round trip to a permission service. That is the whole appeal, and it is a real one, because it removes exactly the synchronous dependency chapter 3 spent a chapter pricing.

The cost lands on writes, and the arithmetic is worth doing in full because it is the argument. Removing one user from a group means rewriting the acl array on every chunk derived from every document that group's grant covers. Chunks are derived from documents at some average rate, so the number of chunk updates is the document count multiplied by that rate. On an index of 2.3 million chunks at an average of 6.2 chunks per document, a folder of 40,000 documents is 248,000 chunk updates. Divide that by the write throughput the store sustains — roughly 1,500 metadata updates per second from a single writer against a live index — and the elapsed time is 248,000 ÷ 1,500 ≈ 165 seconds of pure update time. That is inside five minutes, which is why the design survived its review.

Three things pushed the real number to 47 minutes. The job read the current ACL for each chunk before writing it, because the update was a list mutation rather than a set operation, which halved effective throughput. The engine did not support in-place metadata mutation for one of the two collections involved, so those chunks were deleted and re-upserted — written back whole, since an upsert replaces the record rather than patching a field — carrying their 1,536-dimension vectors along for the ride at about 800 vectors per second. And the job was rate-limited to 40% of write capacity by a guard added months earlier, after an unthrottled backfill had pushed query p99 from 90 ms to 2.4 seconds.

Those three do not close the gap, and saying so is more useful than implying they do. Two of them are quantified — a halving, and a throttle to a little under half of capacity — and composing those two against the design figure lands somewhere in the region of a quarter of an hour rather than three-quarters of one. The third is not recoverable at all, because the chapter gives a rate for the re-upserted collection without saying what share of the 248,000 chunks sat in it. So the correct reading is that the named causes account for a substantial minority of the overrun and the remainder is unexplained by the figures on the page — queueing, retries, or contention with live traffic are the usual candidates, and none of them is evidenced here.

Each of the three is fixable, and fixing all three is what makes this the tempting wrong answer. Batched set operations, an engine that supports partial updates, and a higher throttle would plausibly get 248,000 updates down to 90 seconds. The reason that is not a fix is that it does not change the shape of the cost. Work per revocation stays proportional to the number of chunks the grant touches, so the question is not whether five minutes holds for this folder but which folder breaks it.

Running the same arithmetic backwards gives the answer, and it is the same identity read in the other direction. At 1,500 updates per second, five minutes buys 450,000 chunk updates, which at 6.2 chunks per document is about 72,500 documents. Every folder smaller than that meets the SLA and every folder larger than it does not. The first calculation asked how long a folder takes; this one asks how large a folder may be, and both are the same three quantities — documents, chunks per document, updates per second — rearranged.

That threshold is where the stumble lives, and it is a belief that a defence works. The belief is that the commitment is met, and the evidence for it is that every revocation so far has finished in time. It is tempting because that evidence is real and accumulates daily. It is false because the quantity being held constant is the threshold — a number derived from write throughput that nobody computed and nobody monitors — while the dangerous property, the size of the folder a revocation happens to touch, varies with what customers put in their shared drives. Nobody is checking folder sizes against a threshold derived from write throughput. The company's largest shared drive held 310,000 documents, which is several times the threshold above, and the only reason it had not broken the commitment is that nobody had yet been removed from it in a hurry.

The same cost applies to grants, which teams often forget because the incident is always about revocation. Adding a user to that group is also 248,000 writes, so onboarding a new team member into a broad group means a 47-minute window during which they see a partially-populated corpus and file tickets about missing documents.

There is a second cost that shows up as an infrastructure bill rather than a latency, and it comes from the same duplication. An ACL of 300 user IDs at 36 bytes each is about 10.8 KB of metadata per chunk. Across 2.3 million chunks that is roughly 24.8 GB of ACL data duplicated into the vector store, most of it identical between chunks of the same document and between documents in the same folder. In engines that keep filterable metadata resident in memory for predicate evaluation, that is 24.8 GB of RAM bought to store the same list a few thousand times.

Storing a reference instead of an answer#

The change is to stop materializing the authorization decision at ingest and store only what the chunk genuinely knows about itself, which is which resource it came from:

{
  "id": "d/9f2a41/c12",
  "text": "Termination for convenience requires 60 days written notice...",
  "metadata": {
    "doc_id": "9f2a41",
    "resource_id": "folder:legal/msa-templates",
    "acl_class": 41207
  }
}

The distinction between the two records is the one the section title names. The first stores an answer — who may read this — which is a fact about a document and a set of users jointly, and therefore changes whenever either side changes. The second stores a reference — which resource this came from — which is a fact about the chunk alone and changes only when the chunk is re-derived. Moving from the first to the second means the chunk no longer holds anything that a membership change can invalidate.

The caller's effective grants are resolved at query time from the grant store — the same resolution chapter 2 built from verified claims — and turned into a predicate over resource_id. A revocation becomes a single row change:

update grants
set revoked_at = now()
where subject_id = 'u_0f31'
  and group_id = 'g_legal_reviewers';
-- 1 row affected

Being explicit about what does and does not get rewritten is the part that makes this proposal land in a design review. Nothing about the embeddings changes: no re-embedding, no re-chunking, no vector writes, no index rebuild, and no impact on any other document in the collection. The chunk metadata schema changes once, in a one-time backfill — a job that walks every chunk already in the index and rewrites its metadata to the new shape — that replaces the acl array with a resource_id, and after that backfill the metadata is smaller — a resource_id and an integer are under 60 bytes against 10.8 KB, which returned about 24 GB of memory in this case. The grant store becomes the single source of truth for who may read what, which it should have been anyway. What is added is a resolution step on the read path and a predicate that is now a set membership over resource identifiers rather than a single scalar comparison.

The memory figure is the earlier one arrived at from the other end, and the small gap between them is worth naming rather than smoothing over. The duplicated ACL data was a little under twenty-five gigabytes and the replacement is not zero, so what comes back is that figure minus the new metadata's own footprint — a difference of roughly a tenth of a gigabyte across the whole index, which is why the returned amount is quoted a shade lower than the amount that was there.

The acl_class field is there for the case where the resource predicate gets too wide. A caller with 4,000 group memberships produces an IN list of 4,000 resource identifiers, and several engines cap filter clause sizes in the low thousands or degrade badly past a few hundred terms. The compression that works is that distinct ACLs are far rarer than documents: in this corpus, 2.3 million chunks resolved to 3,180 distinct permission sets, because folders share policies and most documents inherit. Assigning each distinct permission set an integer id and storing that on the chunk turns the predicate into a membership test over a much smaller set, at the cost of a job that maintains the class table when a permission set splits or merges — which is a small, bounded amount of work compared to rewriting chunks, and which happens on policy change rather than on membership change.

That last clause is the whole reason the compression is safe, and it is worth separating the two kinds of event. A membership change is one person joining or leaving, which happens constantly and touches no chunk under this design. A policy change is a folder's permissions being restructured, which is rare and touches only the chunks whose permission set actually split. The class table converts a frequent event into no work and an infrequent event into bounded work, which is the opposite of the arrangement the first design produced.

Read the first row as the one that changes kind rather than degree. Every other row moves along a scale — more bytes or fewer, one term or thousands, a dependency or none — and the first row moves between an expression that grows with the corpus and one that does not. That is why no amount of tuning in the first column reaches the second, and it is also why the last row follows: a design whose write cost is constant can undo a mistake with another constant-cost write.

The third column is the design that keeps the ANN query cheap by filtering only on something coarse and stable, usually the tenant, and then performing an exact authorization check on the small candidate set that comes back — 50 point lookups against the grant store, or one batched call. Oso's The Right Approach to Authorization in RAG is the clearest comparison of the three enforcement points if you want the trade-offs laid out formally. The inline check is genuinely attractive when the permission model is too dynamic to compress into index metadata, and it carries the truncation problem from chapter 1 in a reduced form, since the candidates that fail the check still shrink the result set and still entered your process. It is the right choice when the coarse predicate already provides tenant isolation and the fine-grained check is about which documents inside one tenant a colleague may read, because the blast radius of a mistake is then contained to a single customer.

The copies that still hold the old grant#

Query-time resolution does not make permissions consistent, and it is worth being precise about that before the next chapter, because teams routinely present it as if it did. What it guarantees is that the authoritative answer changes in one write. Every reader between that write and the ANN query — the grant store's read replicas, the retrieval service's local grant cache, the connection that already holds a snapshot, the request that resolved grants 800 ms ago and is still assembling its prompt — may still observe the previous state. The window shrinks from tens of minutes to hundreds of milliseconds, which is a difference in kind for a compliance commitment, and it does not go to zero.

The list in that sentence is not a set of implementation flaws, and reading it as one is the mistake to avoid. Each item is a component that holds a copy of an answer for a reason: a replica exists to spread read load, a cache exists to remove a network hop, a snapshot exists to give a transaction a consistent view, and an in-flight request exists because generating an answer takes time. Copies are what make the system fast, and every copy is a place the old grant can still be observed.

The design also does nothing about tokens that carry grants as claims, which chapter 2 flagged: if a session token embeds the group list, revoking the grant row does not revoke the token, and the effective revocation latency is the token lifetime no matter how fast the write was. And it does not help with material that has already been extracted from the documents — a summary written into a conversation history, a cached answer, a vector store of previous responses — because those are new resources whose ACLs were derived from the old grant and are now stale in a way no predicate catches.

That last category differs from the others in a way worth stating, because it is the one no consistency work reaches. A replica or a cache holds a stale copy of the grant, which a faster propagation eventually corrects. A summary holds content that was legitimately read at the time and now lives under a different resource's permissions, so there is no grant to correct — the derived object is its own resource, and whether it should inherit its source's ACL is a product decision nobody usually makes explicitly.

Five minutes was a number in a contract that nobody had ever measured. The reference design makes it plausible; what makes it defensible is knowing the actual interval between a grant change committing and the last request that can still observe the old grant, which is a quantity you have to go and measure.