Repo-scale code retrieval

Read these first

• Why code is the highest-leverage agent domain
• Chunking strategies

What you will know

• Chunk code by symbol rather than by character count

  AST boundaries preserve functions, classes, and signatures while metadata can retain file paths, namespaces, and enclosing scope.

• Traverse an import or call graph to find related code

  Graph expansion exposes dependencies that lexical search misses, but stale indexes, dynamic dispatch, and generated code can produce misleading neighbors.

• Show text chunking destroying a function

  A boundary-crossing test should demonstrate how incomplete syntax removes the implementation or context needed to modify and verify the function.

• Rank files by relevance to a stated task

  Evaluate rankings against a labeled task set using recall at a fixed cutoff, because the most relevant file is often not the one with the closest words.

• Compare code retrieval with document retrieval on a fixed task set

  Identifiers, call relationships, and symbol scope can matter more than prose similarity, so identical retrieval metrics may hide very different coding usefulness.

Chapters

1. 1

   A fixed window cuts the body off the signature

   Index a Python monorepo in 1200-character windows and a task about prorating refunds retrieves a chunk that scores beautifully: the docstring and signature of calculate_proration, and none of the arithmetic. The boundary fell between the two halves, and the half full of prose is the half that wins on similarity against a task description. Run the boundary-crossing case yourself and watch the patch get rewritten from documentation.

   Read chapter
2. 2

   A symbol chunk needs its enclosing scope attached

   One chunk per method fixes the severed body and creates a new gap: retrieve _evict alone and the agent writes a reasonable patch that breaks an invariant stated in the class docstring. Prepending the class signature and docstring to every method chunk is the minimal repair, with package clause and imports playing the same role at file level. Covers what AST chunking costs — a parser per language, files that will not parse, wildly uneven chunk sizes.

   Read chapter
3. 3

   Import and call edges reach code that words never will

   The file a change must touch frequently shares no vocabulary with the task, but it sits one or two edges away from something that does. Seed on lexical matches, then expand along resolved imports and call sites, and the retrieved set starts matching the set a patch actually edits. Covers how far to expand before precision collapses, and how stale indexes and generated code inject neighbours that look structural and are not.

   Read chapter
4. 4

   Runtime edges are unreachable at any traversal depth

   Handlers registered with @register('payment.succeeded') and dispatched from a dict are not one hop from the dispatcher, or two, or ten — the edge exists only at runtime through a string key. Django URL confs of dotted strings and Spring @Qualifier injection have the identical shape. More depth spends the context budget on unrelated modules without crossing the gap; indexing registration sites and treating the decorator argument as a searchable symbol does.

   Read chapter
5. 5

   Relevance is the file that must change, not the one that discusses it

   For "the /export endpoint is missing rate limiting", the top results are the limiter middleware, its tests, two documents about rate limits and a config schema, while the file that must change is routes/export.py, which never uses the phrase. Relevance for a code task is what has to be read or edited for the change to compile and pass, and that set is routinely disjoint from the files that talk about the topic.

   Read chapter
6. 6

   One ordered list out of four disagreeing signals

   Knowing which file ought to be first is not the same as building the thing that puts it there. A code ranker scores each candidate on several signals at once — lexical or embedding similarity to the task, hops from the seed along resolved import and call edges, whether the chunk is the definition of a symbol or merely a mention of it, and how recently the file changed alongside the seeds in version history — and combines them into a single order. Each signal is weak alone and they routinely disagree, which is why the definition-site and graph-distance terms have to outrank term overlap for a file that never uses the task's words to surface at all. Then decide the cutoff: how many files enter the prompt, and what you drop to make room.

   Read chapter
7. 7

   Labels drawn from merged commits, not from judgement

   A retriever scoring 0.91 recall against labels two engineers assembled by writing down what looked related can coexist with a 47% first-attempt compile rate, because a caller in a module they never opened was never in the labels. Take ground truth from real merged changes instead: the files a correct change touched, plus the files that had to be read to make it. Includes the measurement that would justify keeping the current retriever.

   Read chapter
8. 8

   Equal recall hides unequal usefulness on code

   A document retriever and a code retriever both at recall@10 of 0.84 are not equally good, because a prose answer survives a missing source and a patch does not: one unretrieved call site makes the whole change wrong. This is what breaks when a document RAG stack is ported to a repository unchanged, and why swapping in a code-trained embedding model addresses neither the chunking upstream of it nor the graph relationships downstream. Give the encoder its due, though: it splits camelCase and snake_case so calculateProration matches a task that says prorate, it knows code vocabulary from prose vocabulary, and it puts less weight on the docstring text that won the severed chunk its top score. That is a real gain over a general text model and a small share of the gap.

   Read chapter