A filter can run before the search or after it, and the order changes the answer

A vector search ranks documents by similarity to the query and by nothing else, so left alone it returns the best match anywhere in the collection at a moment when the user is entitled to see, or has asked to see, one small corner of it. Metadata filtering is the mechanism that constrains which documents are eligible to come back. Alongside each vector the store keeps a record of structured attributes — document type, jurisdiction, owning tenant, creation date, the folder a file lives in — and a filter is a boolean predicate over those attributes that a document must satisfy before it can appear in the results at all. Eligibility is a hard constraint rather than a preference, so a document failing the predicate stays out however similar it is to the query.

You meet it wherever retrieval is scoped, which in a product is nearly everywhere: the facet checkboxes down the side of a search page, a "search within this project" box, a date range, a language selector, and, carrying the highest stakes of the lot, the tenant predicate that keeps one customer's contracts out of another customer's results. Two boxes ticked in a sidebar and the tenant_id = ? clause that a repository layer adds to every query are the same mechanism with very different consequences when it goes wrong.

Saying which documents are eligible does not settle when the engine is told. Approximate nearest neighbour search — ANN, the family of index structures underneath the vector databases of the previous topic — answers a query by examining a small, carefully chosen fraction of the stored vectors rather than all of them. Whether the predicate is visible during that examination, or applied afterwards to its output, is a decision someone makes, deliberately or by default. Getting it by default is the normal case, because the default is whatever the application code happens to do.

Two places the predicate can live#

Post-filtering means the index answers first and the predicate runs on its answer. It is the version you get whenever the filter lives in application code:

# post-filter: the index is blind to the predicate
hits = index.query(vector=q_vec, top_k=10)
eligible = [
    h for h in hits
    if h.metadata["doc_type"] == "msa"
    and h.metadata["jurisdiction"] == "DE"
]
return eligible  # len(eligible) is whatever survived

Pre-filtering means the predicate is part of the query, and the engine restricts the candidate set before or during the search, so every one of the ten results it returns is already eligible:

# pre-filter: the predicate is pushed into the engine
hits = index.query(
    vector=q_vec,
    top_k=10,
    filter={
        "doc_type": {"$eq": "msa"},
        "jurisdiction": {"$eq": "DE"},
    },
)
return hits  # len(hits) is 10 if 10 eligible docs exist

The two snippets are four lines apart and the second one is not obviously more expensive, which is why the mistake is easy to ship. What changed is who owns the constraint. In the first version the constraint is a fact about the response; in the second it is a fact about the search, and the engine's traversal is allowed to use it.

Why a blind top-k comes back short

The gap between the two plans is arithmetic, and it is worth deriving rather than asserting, because the derivation is what tells you where the boundary between the plans sits.

Start by naming the quantity that governs everything. The selectivity of a predicate, written s, is the fraction of the stored documents it matches: the number of eligible documents divided by the total number of documents. A filter matching two documents in every five has a selectivity of two fifths. One matching two in every hundred thousand has a selectivity a great deal smaller, and it is that end of the range where the plans separate.

Now take one candidate out of the index's blind top-k and ask whether it survives the list comprehension. It survives exactly when it satisfies the predicate. If eligibility were unrelated to similarity — if being a German master service agreement told you nothing about being near this query vector — then the chance that any one candidate is eligible is just the fraction of the collection that is eligible, which is s by definition. The index handed back k candidates. Each survives with probability s, independently of the others under the same assumption, so the expected number of survivors is k added to itself s times over: expected survivors = k × s.

Multiplication is the right operation here for a reason worth stating, because this is the step people skip. Expectation adds across candidates whether or not the candidates are independent of each other. Each of the k candidates contributes its own probability s of being counted, and adding s to itself k times is what multiplying by k means. So the product survives even when the candidates are correlated, which they usually are — the independence assumption is doing work in a different place, which is the next paragraph.

Where independence earns its keep is in the probability of getting at least one survivor rather than the average count. A candidate fails the predicate with probability one minus s. If the k verdicts are independent, all k fail together with probability one minus s, multiplied by itself k times, and the chance of at least one survivor is the complement of that. For a selectivity of a fraction of a percent and a k of ten, that expression sits just below the raw product k × s, because the small chance of drawing two survivors at once is being removed from the count. In this regime the two numbers agree to about a tenth of their own size, so the expected-survivor figure doubles as the odds of the page being non-empty at all.

The reason the failure feels intermittent rather than total lives in the assumption that was just made and is false. Eligibility is not independent of similarity. German master service agreements cluster together in embedding space, because a model that maps meaning to direction maps documents about the same thing in the same language to nearby directions. A query phrased in German pulls that cluster closer. So the real survival rate is higher than k × s and wildly variable across queries: some return four results, most return zero, and a support ticket saying "search is broken for this filter" gets closed as unreproducible because the person testing it happened to pick a query where the correlation was strong.

Reading the same formula backwards gives the standard patch. If k × s survivors is too few, ask the index for more candidates. Fetch k divided by s of them instead of k, and the expected survivor count becomes k divided by s, multiplied by s — the s cancels, and what is left is exactly k. That cancellation is not an approximation. The over-fetch multiplier is the algebraic inverse of the dilution the predicate causes, so on average it puts back precisely what the predicate took out, which is why over-fetching works at all and why it works so cleanly in the loose-filter regime.

The cancellation also shows where the technique dies, and it is not in the algebra. At s = 0.5 and k = 10 that is 20 candidates, which is nothing. At s = 0.05 it is 200 candidates, which is a heavier but still ordinary query. At s = 0.00088 it is 11,400 candidates, and at that point you are asking a graph index to return more than eleven thousand neighbours in order to keep ten, which costs more than any pre-filtered plan and still gives you no guarantee. The formula still says the page fills. The index is the thing that stops cooperating: over-fetching is a real technique with a real ceiling, and the ceiling arrives faster than people expect.

What pushing the predicate down actually costs

How an engine uses a predicate it can see depends on the index. On a graph index, the usual implementation keeps a payload index — a lookup from each attribute value to the set of document ids carrying it, held as an inverted list or a roaring bitmap, which is a compressed representation of a set of integer ids — and consults it during the walk, either by skipping ineligible neighbours as it expands or by intersecting the neighbour list with the eligible bitmap. On an inverted-file index, the eligible set restricts which vectors inside each probed cell get scored, and because eligible vectors are scattered across cells rather than concentrated, the engine often has to probe more cells to fill its result list.

Neither of these is free, and the three costs land in different budgets. The bitmap costs memory proportional to collection size per indexed attribute. The intersection costs time on every hop. The extra cell probes cost latency in direct proportion to how many extra cells are probed.

A number makes the trade concrete. On a 10M-vector HNSW index — the hierarchical navigable small world graph, the default index in most vector stores — queried with ef_search = 128, meaning the walk keeps at most 128 candidate nodes in play at any moment, an unfiltered query on a mid-range instance runs in roughly 6-9 ms. The same query with a pre-filter matching 40% of the collection runs in about 8-12 ms, because most neighbours the walk considers are still eligible and the walk barely notices. The same query with a pre-filter matching 0.09% runs in 40-300 ms and returns fewer than ten results, because the walk spends its entire budget stepping through neighbours it must reject.

Read those three latencies as gaps rather than as levels and the mechanism falls out. The step from no filter to a filter matching two documents in five is almost free — a couple of milliseconds — because the predicate is rejecting roughly three neighbours in five and the walk simply takes a few more hops. The step from there down to a predicate matching under a thousandth of the collection is an order of magnitude and more. The cost of pre-filtering is not a constant. It is a function of how rare the eligible documents are, and it gets worse in exactly the regime where post-filtering also fails.

The vocabulary here is not standardised across vendors, which makes documentation harder to read than it should be. What one engine calls pre-filtering another calls filtered search or single-stage filtering, and a third describes the same mechanism as an in-graph or in-place filter to distinguish it from the older approach of materialising the eligible id set up front and searching a temporary sub-index. The materialising version is genuinely pre- in the literal sense — the eligible set exists before the search starts — while the in-graph version interleaves the predicate check with the traversal and is closer to a filtered walk than to a two-phase plan. Qdrant's A Complete Guide to Filtering in Vector Search is the clearest vendor-neutral treatment of these three shapes if you want the mechanics in more detail than this chapter needs. For choosing between plans, the distinction that matters is not what the engine calls the strategy but whether the predicate is visible to the traversal or only to the response.

Which failure you would rather have#

Both plans are correct implementations of something. Post-filtering computes "the eligible members of the global top-k", pre-filtering computes "the top-k of the eligible members", and those are different questions that happen to have the same answer when the filter is loose.

The result-count column checks out against the derivation above and is worth verifying rather than taking on trust. Ten candidates at a selectivity of two in five gives an expected survivor count of four, which is the "~4 of 10" in the table. Ten candidates at the tight selectivity gives an expected count of about nine thousandths of a document, which rounds to the "0 or 1" one row below. The table is not reporting two separately measured facts; it is reporting one formula evaluated twice.

The row that matters for design is the last-but-one, and it names the stumble in this chapter. A reader who has just seen the latency column will conclude that a healthy latency graph means a healthy filter path, and that conclusion is comfortable, defensible and wrong. Look at what post-filtering's latency does as selectivity falls through three orders of magnitude: it stays at about seven milliseconds. The index is doing exactly the same work in the catastrophic case as in the healthy one, because it never learned that anything was being asked of it. A quantity that does not move when the dangerous property moves cannot be used to rank queries by that property. Latency is constant across precisely the axis it would need to vary along to be a defence, so it will not show up on a latency dashboard and will not trip a timeout alert. It shows up as users saying search is bad.

The band where post-filtering is the right plan#

The temptation after seeing a starvation incident is to declare pre-filtering the correct approach and push every predicate into the engine. That is the answer most candidates give in an interview, and it is wrong in a specific, measurable way: it converts a correctness problem into a latency problem across all the queries that never had a correctness problem.

Consider a support-ticket search where the standard filter is status != "spam", matching 97% of the stored tickets. Post-filtering with a top-k of 12 and a page size of 10 returns ten eligible results essentially always. Work out why: the page fails only if four or more of the twelve candidates are spam, and spam is three tickets in a hundred, so the expected number of spam candidates in a draw of twelve is around a third of one document. Getting four when you expect a third requires four independent low-probability events to coincide, and the probability of that is under 0.2%. The cost of the guarantee is the two extra candidates.

Pre-filtering the same query forces the engine to maintain a payload index on status, consult a bitmap on every hop of every walk, and add a few milliseconds to a query that was already returning the right answer. On a service doing 400 queries per second, that is real hardware for no gain.

There is a second reason not to treat pre-filtering as a global default, which is that it changes what the index can be. Some engines require an attribute to be declared and indexed before it can appear in a filter, and each declared attribute adds build time and memory. A team that pre-filters on eleven attributes because eleven attributes appear somewhere in the UI ends up with eleven payload indexes, a slower ingest path, and a rebuild that takes long enough that schema changes stop being routine. The attributes worth indexing are the ones whose predicates are selective enough to matter, which is a question you can only answer by looking at traffic.

Eligibility, preference and score#

The hard-constraint framing from the opening is what separates a filter from two neighbouring things it gets confused with.

A filter is not a ranking signal. "Prefer documents from the last 30 days" and "only documents from the last 30 days" are different requirements, and implementing the first as the second is the most common product bug in this area — a user asks for recent material, gets a created_at > now() - 30d filter, and the perfect answer from six weeks ago becomes unreachable. Recency preference belongs in the score, as a decay term applied after retrieval, or in a fusion step. Eligibility belongs in the filter.

A filter is also not hybrid search. Combining a BM25 keyword score with a vector score, and fusing the two ranked lists, is a relevance technique that operates on scores; a metadata filter operates on a predicate over structured attributes and admits no partial credit. The two are often configured through the same API surface, which is where the confusion starts, and an engine that supports both will happily let you express "must contain the word indemnity" as a filter when what you wanted was a keyword contribution to the score.

Finally, filtering does not improve relevance within the eligible set. Restricting to German master service agreements does not make the ranking of German master service agreements better; it just removes the competition. If the top result inside the filter is unhelpful, the filter is not the thing to tune.

Rarity survives the pushdown#

Pushing the predicate into the engine fixes the failure above, and if the story ended there this topic would be one chapter long. It does not end there, because the pre-filtered query in the table carries a footnote: at 0.09% selectivity it returns up to ten results.

The reason is visible in the latency figures already given. A pre-filtered walk at that selectivity takes 40-300 ms, which is five to forty times the unfiltered cost, and that time is being spent somewhere. It is being spent examining neighbours and rejecting them. A graph walk explores a bounded number of candidates — the ef_search = 128 above is exactly that bound — and when eligible documents are rarer than that bound, the walk can exhaust its allowance among ineligible neighbours and stop early. It returns six results, or two, with a normal status code and unremarkable latency, from an index that is behaving exactly as configured.

That is a different failure from the one in the API handler, and the difference is worth holding onto, because the two produce the same user-visible symptom. In the post-filter case the index found the right answer to the wrong question. In the pre-filtered case the index was asked the right question and ran out of budget before it could answer. The first is a plan bug and pushing the predicate down fixes it at every selectivity. The second is a property of the index structure, it survives the fix everyone reaches for first, and it is the subject of the next chapter.

Choosing between the two plans is therefore a choice about which question you want answered, made per query rather than once per codebase. Post-filtering answers "which of the globally best documents happen to be eligible", which is the right question when nearly everything is eligible and a wrong one when almost nothing is. Pre-filtering answers "which eligible documents are best", which is the right question far more often and costs an amount that scales with how rare eligibility is.

The quantity that decides between them is selectivity, and the decisive fact about selectivity is that neither plan announces it. A filter matching most of the collection and a filter matching a thousandth of it produce responses that are identical in status code, near-identical in latency, and different only in how many rows came back. Every measurement in the rest of this topic exists because that one number has to be recovered from somewhere other than the search path.