A delete hides the row while copies of it stay reachable elsewhere

Erasing a document from a retrieval system is a different operation from deleting a row, and the gap between the two is where the trouble lives. A delete statement removes the record a person can point at. What a retrieval system actually answers questions from is a set of derived copies of that record, made at ingest time, held in different engines, owned by different parts of the stack and ended by different mechanisms. A delete that succeeds against the record can therefore leave the content answerable, and the situations that force the question — a legal instruction, a licence withdrawn, a file uploaded to a place it should never have reached — are exactly the ones where "the row is gone" and "the content is unreachable" are separate claims and only the second is being asked for.

Being precise about what gets searched is the first step, because it is neither the workspace a user sees nor the document store behind it. A retrieval system of this kind searches a vector index: at ingest each document is cut into passages of a few hundred tokens, called chunks, every chunk is run through an embedding model to produce a vector, and those vectors are held in a structure that can return the nearest ones to a query vector in a few milliseconds. That index is built once and thereafter updated only by whatever writes reach it, so what it holds is a record of everything an ingest pipeline has ever done rather than a view of a corpus as it stands this morning. The corpus, meanwhile — the set of documents you have ingested and embedded — does not hold still. Documents arrive, get edited, get withdrawn on legal or licensing grounds, and every year or two the embedding model is replaced by a better one, which makes every vector in the store obsolete on a single day.

Index maintenance is the work of keeping that store in agreement with the source while it continues to answer queries: applying inserts, updates and deletions, reclaiming the space and search budget that deletions leave behind, rebuilding the structure as it degrades, and re-embedding the whole corpus when the model changes. It needs a name of its own because none of those operations behave the way their equivalents in a relational database do. A delete usually marks a vector rather than removing it, an update overwrites only the chunks the writer happens to know about, and a query never fails when the index and the corpus disagree, because there is no constraint anywhere that would notice. Teams meet the subject in one of four situations — the first document somebody has to erase on legal instruction, the first published page edited after it was ingested, the first embedding-model upgrade, and the first quarter in which retrieval quality falls with no deployment to blame — and erasure is the one that arrives first.

Where one document actually lands#

A vector index is not the system of record; it is one of several derived stores that a single ingest event populates. Ingesting a document is a fan-out, and every branch of it is a copy with its own lifetime, its own owner and its own purge mechanism.

Count the boxes. There are twelve: a documents row, its chunk rows, the vector index, a lexical index, an embedding cache keyed by content hash, a summary written by a model, that summary's own vectors, read replicas, a nightly snapshot, a retrieval result cache, prompt and completion logs, and an eval set built out of those logs. An erasure tool that sets a flag on the documents row and deletes the chunk vectors by id has written to two of those twelve, which leaves ten it has not touched. The diagram is the dozen, and it is worth holding that count in mind, because it is the only place anyone has written the ten down.

The asymmetry between the two directions of that picture has a mechanism, and it is worth walking, because the slogan it usually gets compressed into is easy to repeat and gives nobody anything to do. An ingest event is a single trigger with many consequences. Each consequence is written by whoever added that derived store: the engineer who added keyword search wrote the branch into the lexical index, the engineer who added summaries wrote the branch into the summary table. The fan-out is therefore documented by construction — somebody had to write each branch, in the ingest code, where the next person to read that code will find it. A delete has the opposite shape. It has to reach every branch, and no branch registers itself with the delete path. What the delete path ends up being is the union of the destinations one person happened to remember on the day they wrote it. The write path was designed. The delete path accumulated.

The link that makes that inevitable is a design decision rather than a requirement, and it is the one worth changing. Nothing forces a derived store to declare itself to the erasure code, and no property of any stack prevents one from doing so. Putting the enumeration in code, as a constant the erasure function iterates, is what converts a forgotten store from an incident into a CI failure — a test can assert that every store the ingest path writes appears in that constant, and a runbook cannot assert anything.

The diagram is not exhaustive for any particular stack, and that incompleteness is the reason to draw it: the version for your system is a document somebody has to write, because the ingest code alone does not tell you where things end up. Three of the branches make the point on their own. A summary written by a model at ingest time contains a salary figure in paraphrase, sits in its own row, has its own vector, and is not reachable from the chunk ids an erasure tool deleted. A prompt log contains the retrieved chunk text verbatim in the context window that was sent to the provider, and it is retained for whatever your observability vendor's retention is — commonly 30 days, sometimes longer if a trace was pinned to an incident. An embedding cache keyed by sha256(chunk_text) still holds the vector, so a re-upload of the same file skips the provider call and reproduces the exact vector you thought you had removed.

Each of those copies has a lifetime, and each has one specific mechanism that ends it. The pair is worth reading side by side, because the mechanism is what a delete call has to invoke and the lifetime is what happens instead when it does not.

The row that matters most in that table is the last one, because it is the only one where the copy has already been separated from the identifier you would use to find it. Once a chunk has been folded into a JSONL file used to build an eval set, there is no doc_id you can issue a delete against without reprocessing the file. Every other row names a mechanism a program can call; that row names a person.

Backups deserve their own attention because they invert the direction of the problem. A snapshot taken at 03:00 contains the document, the erasure ran at 09:12, and a restore performed at any point in the following month reintroduces content that has been formally deleted, along with the vectors, into a system whose audit log says the deletion succeeded. Every other copy in the table is a retention problem, where content persists past the delete; this one is a reintroduction problem, where content arrives after it. Teams that have to answer for this keep a durable deletion ledger outside the backup domain — an append-only list of erased document ids with timestamps — and replay it as a mandatory step after any restore. Three properties make that work, and all three are load-bearing: the ledger is small, so keeping it forever costs nothing; it lives outside the backup domain, so a restore does not roll it back along with everything else; and it survives exactly the event that destroys the evidence. Given those, it turns "we restored last Tuesday and hoped" into a deterministic post-restore job. Without it, the honest answer to whether a document is gone is "until the next restore".

One flag, two query paths#

A hybrid retriever runs two searches and merges their results: a vector search over the chunk embeddings, and a lexical search — BM25, which scores documents by weighted keyword overlap rather than by vector distance — over the same text in a separate engine. A soft delete has to be honoured by both halves, and a system in which only one half honours it still answers. In one such system the vector path filtered on a joined documents.deleted_at IS NULL predicate, because the team had been careful about exactly this, and after an erasure it returned nothing. The lexical engine held its own copy of each document with a deleted boolean in the metadata, and the query builder for that path had been written six weeks earlier by someone adding keyword search, with a filter for workspace_id and language and nothing else. Both paths were correct in isolation. The union of them was not.

That is the general shape a defence fails in, and the soft-delete flag is its clearest example in a retrieval system. A defence works when it varies with the thing it has to discriminate. This one does not vary at all: the flag is the same boolean whatever question is asked and whichever engine is asked it, and it separates deleted content from live content only where some reader has been written to consult it. What does vary is the set of paths that reach the content, because paths get added by different people at different times, and the flag has no way to notice a new one.

The mistake is tempting for a nameable reason. The team in that example had been careful about deletion — more careful than most — and they had hardened the path they thought of as the retrieval path. In a system that grew a lexical half later, that phrase keeps naming the vector search in everybody's head long after it has stopped being the whole of retrieval, and a defence installed on the path you think of is a defence installed on a subset you never enumerated.

A soft-delete flag is a convention rather than a mechanism, and a convention holds only where every reader knows about it. That is the practical difference between soft delete and hard delete for retrieval systems, and it is a stronger argument than the usual one about storage: a hard delete fails loudly on the paths that were not updated, whereas a soft delete fails silently on exactly the paths that did not get the memo. A query that joins to a row which is no longer there errors or comes back empty, and somebody investigates within the hour; a query that ignores a flag returns a plausible, well-cited answer that nobody questions at all. When a team chooses soft delete — and there are good reasons to, including undo, audit and the cost of cascading through derived stores — the flag needs to be enforced somewhere every path must pass through, such as a view, a shared query builder or a post-retrieval filter applied to the merged candidate set before ranking. The last of the three is the strongest, because the merged candidate set is the one place where the two halves have already become a single list and a filter applied there cannot miss a half.

Replica lag is the same failure in a different coordinate, and it is worth walking rather than asserting. Suppose the vector store runs one writer and three read replicas. A delete acknowledged by the writer is durable at the writer and nowhere else yet, and it becomes visible on a replica only once the replication stream carrying it has been applied there. Under normal load that catch-up is around 200 milliseconds. During a large backfill it can be several minutes, because the stream is ordered and the delete is queued behind the backfill's own writes — the replicas are not slow, they are busy applying everything that was written before it. A user who asks a question 400 milliseconds after the erasure tool returns success can therefore be routed to a replica that has not applied the delete, and gets an answer built from data that no longer exists upstream. Nothing in the erasure tool's return value tells you this, because the tool asked the writer and the writer told the truth. Closing that gap needs the tool to wait rather than to ask, which is what the replica_watermark step in the erasure listing below does.

Deleting as a fan-in with receipts#

The version of the erasure path that survives an audit treats deletion the way the ingest path treats writing — as an enumerated set of destinations, each of which returns evidence.

DESTINATIONS = [
    "primary_row", "chunk_rows", "vector_index", "lexical_index",
    "embedding_cache", "result_cache", "derived_summaries",
    "trace_logs", "replica_watermark",
]

def erase(doc_id: str) -> dict[str, Receipt]:
    receipts: dict[str, Receipt] = {}
    chunks = db.fetch_chunk_ids(doc_id)          # capture ids BEFORE the row goes
    hashes = db.fetch_chunk_hashes(doc_id)

    receipts["primary_row"] = db.hard_delete(doc_id)
    receipts["chunk_rows"] = db.delete_chunks(doc_id)
    receipts["vector_index"] = vectors.delete(ids=chunks)
    receipts["lexical_index"] = lexical.delete_by_query({"doc_id": doc_id})
    receipts["embedding_cache"] = cache.evict_many(hashes)
    receipts["result_cache"] = cache.invalidate_tag(f"doc:{doc_id}")
    receipts["derived_summaries"] = db.delete_summaries(doc_id)
    receipts["trace_logs"] = tracing.delete_spans(attribute={"doc_id": doc_id})
    receipts["replica_watermark"] = vectors.wait_for_replicas(
        lsn=receipts["vector_index"].lsn, timeout_s=30
    )

    audit.record(doc_id, receipts)               # fails the erasure if any is missing
    return receipts

Three details in that listing carry the weight. The chunk ids and content hashes are read before the row is removed, because a hard delete with a cascade destroys the only index you have into the derived stores — a mistake that turns a recoverable partial erasure into an unrecoverable one. Every destination returns a receipt rather than None, so a destination that silently no-ops (a lexical delete-by-query that matched zero documents because the field is named document_id there, not doc_id) shows up as a zero count in the audit record instead of as success. And the replica watermark is a wait, not a fire-and-forget, so the tool does not report completion while three replicas are still answering with the old data.

What a receipt is evidence about is worth stating plainly, because it bounds what the audit record can ever mean. A receipt is evidence about one destination: it says that this call, against this store, matched and removed this many things. It says nothing whatsoever about any store the call was not made against. An audit record assembled out of receipts is therefore only as complete as the list of destinations it iterates, and a clean audit trail is a statement about the list, not about the content.

Count the destinations a third time and compare the counts. The diagram has twelve nodes, the table has ten rows, and DESTINATIONS has nine entries. Line the last two up and the omissions turn out not to be arbitrary. Every entry in the list is a store with an identifier-addressable purge: a row deletable by primary key, vectors deletable by id, a lexical document deletable by query, a cache entry evictable by hash or by tag, a summary deletable by doc_id, a span deletable by attribute, a replica you can name a log position for and wait on. The two rows of the table the list does not contain are the nightly snapshot and the fine-tune/eval sets, and those are exactly the two the table itself describes as having no callable purge at all — one ends by "backup expiry; restores reintroduce it", the other by "a human remembering".

That identity is what licenses the design, and it also states its limit. An enumerated erasure function is a complete answer for precisely the destinations that can be addressed by document id, and for the other two it is not a partial answer but no answer, because there is no call to make. The eval set is the case where the copy has already been separated from the identifier you would use to find it. The snapshot is the case that runs the problem backwards, reintroducing content after the erasure rather than merely retaining it through one. Both need the mechanism described above rather than a longer DESTINATIONS list: the durable deletion ledger, replayed as a job after any restore, which covers the snapshot outright, and read by a person before any eval file is rebuilt, which is the only thing that covers a file no API can address.

The list is not portable. Yours will have a graph database in it, or a caching layer in front of the retriever, or a Slack bot that posted the answer into a channel where it now lives as message text. Enumerating the destinations is a design activity performed once and revisited whenever someone adds a derived store, and the enumeration belongs in code — a constant that the erasure function iterates — rather than in a runbook, because a runbook does not fail CI when somebody adds a store and forgets it.

None of this reaches the weights#

None of this makes the model forget. If chunks from that offer letter were included in a fine-tuning run or in the few-shot examples baked into a prompt template, the parameters and the template carry the content in a form no delete API reaches, and the only remedies are retraining or editing the template. Deletion in a retrieval system is about reachability of stored copies; it says nothing about what has been absorbed into weights. The distinction is worth holding because the two are often reported as one status: an erasure audit can be green while the content is still recoverable from a model that was trained on it, and no field in that audit is the field that would say so.

The vector your delete call did successfully accept is a second gap, and a subtler one. Most vector engines do not remove a vector from the graph when you delete it; they mark it, and the marked node stays in memory and keeps being traversed until a compaction or merge reclaims it. That is why the row for chunk vectors in the table above says "until compaction" rather than "immediate", and it is a large enough subject — with its own cost curve and its own monitoring — that chapter 3, on what tombstoned vectors cost before compaction runs, is about nothing else. For the erasure question specifically the consequence is narrow and worth stating exactly: a tombstoned vector will not be returned as a result by a compliant engine, but it is still resident, still walked during search, and still present in any snapshot taken before the compaction ran. Deleted and unreachable are not the same state, and the snapshot row of the table is where the difference becomes visible.

Everything in this chapter has been a count. The diagram counts the destinations one ingest event creates, the table counts what ends each of them, and the erasure listing counts the ones a program can address by id. Where those counts disagree is where a document survives its own deletion — twelve nodes against two writes leaves ten, and nine enumerated destinations against ten rows of the table leaves the snapshot and the eval file. That last pair is not an oversight in the listing; it is the boundary of what any enumerated erasure function can do, and it is why the deletion ledger sits outside the erasure path rather than inside it.

Every mechanism above also assumes somebody issued a delete for a document, and that the ids to delete are the ids the index holds. When a document is edited rather than removed, no delete is issued at all, the writer believes it has replaced the document's chunks, and the index quietly keeps the ones the writer no longer knows about. That case leaves no tombstone, trips no gauge and passes every check in the erasure audit, which is why it is next.