Repo Context: Why Tools Succeed or Fail on Big Codebases
Indexing, retrieval, and memory files: how each tool finds what matters in 100k files.
AI coding tools work on big codebases because they build a structured map of your repository. Indexing pipelines turn source code into graphs of symbols and relationships. Retrieval systems then feed exactly the right slice of that map into each model request. Memory files add the rules and conventions that no amount of scanning can infer. The combination makes sense of a hundred thousand files.
But here is the surprise: tools with smaller context windows often outperform those that dump entire repos into prompts. Structured context beats raw volume every time. Last time we saw how verification workflows let you trust AI written code. But trust depends on the AI having the right context when generating that code. On a big codebase, that is the hardest part.
- 200k+ tokens
- Noisy context
- Lower accuracy
- Slower inference
- ~10k tokens
- Relevant files only
- Higher accuracy
- Faster inference
Why Does a Model That Nails Small Projects Fall Apart on a Monorepo?
A monorepo has cross-file dependencies, multiple languages, and decades of decisions that no model can guess from a single file. Without a map, the model reimplements existing functions and ignores architectural guardrails. It simply lacks the information needed to act like a developer who knows the codebase.
Imagine you are dropped into a library with a million books and asked to find the one paragraph that explains why a checkout API (application programming interface) must hash user IDs before logging. You could read every book cover to cover, but you would run out of time. If instead you had a card catalog organized by topic, plus a shelf map showing related books, you would find it in minutes. Repository context is that catalog.
Our developer starts the week with a feature request: add multi-factor authentication logs to a payments service. The AI agent, only seeing the local file, writes a new hashing function. A well indexed tool finds the shared utility in another service, pulls its signature and docs, and calls it instead. The difference is a map.
How Do Tools Build a Map of a 100k-File Repository?
They run an indexing pipeline that parses every source file, builds a symbol graph, and stores embeddings for fuzzy search. This pipeline typically connects to the build system or language servers to get precise semantic information. The result is a queryable representation that spans the entire codebase.
The pipeline starts with code host integration. Sourcegraph, for example, connects to GitHub, GitLab, and Bitbucket instances, clones repositories, and indexes them on a schedule Sourcegraph indexing docs. Kythe integrates directly into the Bazel build system via aspects that hook into compilation, extracting definitions, references, and types without reimplementing language parsers Kythe documentation. ABCoder’s TypeScript indexer uses the TypeScript compiler API directly for fast, accurate symbol extraction on repos up to 1.2 million lines.
The parsed data is stored in multiple forms. A graph of nodes and edges captures symbols, calls, extends relationships, and cross-file references. A full text search index supports fast keyword lookups. A set of vector embeddings enables semantic nearest neighbor queries. Together they answer queries like “find callers of this function” or “show files most similar to this one.”
But graphs and vectors only capture what the code is. They miss how the code should be used and why decisions were made. That is where memory files come in.
What Exactly Goes Into a Memory File and Why Does It Matter?
A memory file is a machine readable document that encodes project conventions, architecture decisions, and validation commands. AI agents read these files to behave like engineers who have absorbed the team’s tribal knowledge. They bridge the gap between scanning code and understanding the rules that govern it.
Packmind’s context engineering framework breaks this into four layers: the what (architecture map, folder layout, technology stack), the how (coding standards, naming conventions, approved libraries), the why (architectural decisions, trade offs, and constraints), and validate (commands to run tests, lint, and build) Packmind context engineering. Tools like Repo-contextr package directory structures and metadata into a single file designed for LLM (large language model) consumption.
Our developer’s team creates a file called AI_CONTEXT.md that states: all API responses must be wrapped in a Result<T> type. The agent reads this file and generates code that follows the rule. Without it, the agent might return raw data and break client contracts. Memory files encode the intent that scanning alone cannot recover.
How Does Retrieval Pick the Right 50 Files Out of 100,000?
Retrieval combines graph traversal, code search, and LLM aware ranking to identify a minimal sufficient set of context. It avoids drowning the model in noise while ensuring it sees the dependencies it needs. The goal is not to cram in as much code as possible but to select the exact slices that change model behavior.
RepoCoder showed that iterative retrieval improves repository level code completion by more than 10 percentage points over single pass retrieval RepoCoder paper. CodeRAG extends this by constructing retrieval queries guided by the code LLM’s own log probabilities and then ranking candidates with a preference aligned BestFit reranker that learns which snippets the model will actually use CodeRAG paper. This drastically reduces the scenario where a model receives a dependency implementation but reimplements it anyway.
RepoExec introduced the Dependency Invocation Rate (DIR) metric to measure exactly that. When given full dependency implementations as context, models achieve the highest functional correctness and DIR. With only signatures or docstrings, DIR drops and models invent their own versions of functions that already exist RepoExec paper. The developer’s agent started the week with low DIR. After the team connected a graph based index and applied a preference aligned reranker, the agent called the shared hashing utility correctly.
Hybrid stores make this fast. Graph edges answer structural queries. Text search finds keywords. Embeddings provide a fuzzy fallback. The reranker aligns everything to the model, producing a prompt that is dense with useful information.
Can a Tool Use Context From Other Repositories and Services?
Yes. Enterprise setups like Sourcegraph Cody maintain a central index over all repositories and can resolve cross repo references in real time. Kythe’s graph can model multi language dependencies. This is what makes AI assistance work when your feature spans Go and Java services owned by different teams.
Cody’s remote repository context works by having a Sourcegraph instance index all code hosts. When you ask a question in your IDE (integrated development environment), the instance searches its index and returns relevant files from any repository, without you needing to clone them locally Cody remote context docs. Kythe’s crossref service provides RPC methods that traverse a single graph for definitions, references, and callers across languages Kythe crossref service.
The developer’s feature requires a JWT (JSON Web Token) validation function that lives in an auth service written in Java. With remote context, the tool retrieves that function’s signature and usage pattern. Without it, the agent invents a client that breaks during integration. Cross repo context turns a multistep research task into a simple lookup.
Quick Reference
| Property | Value |
|---|---|
| Common indexing pipelines | Sourcegraph (code host sync), Kythe (compiler integration), ABCoder (TS compiler API) |
| Context representations | Symbol graphs, full-text search indexes, vector embeddings |
| Memory file layers (Packmind) | What (architecture), How (conventions), Why (decisions), Validate (commands) |
| Recommended retrieval granularity | Function level snippets with call edges, not whole files |
| Dependency Invocation Rate (DIR) | Percentage of tasks where model calls an existing dependency instead of reimplementing |
| Staleness impact | Leads to hallucinated or missing APIs, broken imports, and reimplemented utilities |
Frequently Asked Questions
Q: Doesn’t a 200k token context window make all this indexing unnecessary?
Even with large windows, models get lost in long, noisy contexts. Research on repository level completion shows structured retrieval helps small models outperform larger models with raw dumps RepoExec. The winning strategy is a dense, high-signal slice, not a larger window.
Q: How often should I re-index?
Sourcegraph re-indexes on a configurable schedule. Build integrated indexers like Kythe can update incrementally when new commits land. For AI tools, an index that is hours old can cause hallucinations of functions that were renamed or deleted. Target a freshness of minutes after merges for critical repositories.
Q: Can I autogenerate memory files from existing code?
Some tools can extract coding patterns and conventions automatically, but the “why” layer requires human input. Start with a simple file containing the build, lint, and test commands plus a top level architecture description. Add rules incrementally when the agent repeats a mistake.
Q: What happens if the context is wrong or stale?
The model will make decisions based on incorrect types or nonexistent functions. You might see code that compiles but fails integration, or silent reimplementations of shared utilities. Staleness is one of the most common causes of broken imports and repeated work.
Q: How do I measure whether retrieval is working?
Track the Dependency Invocation Rate for tasks where dependencies exist. Also monitor how often generated code follows project specific patterns instead of generic ones. If the agent frequently produces functions that already exist, the retrieval pipeline is not providing the right context.
Test yourself
Your team’s AI agent adds logging to a service but does not use the standard structured logging wrapper from the shared library. The wrapper injects correlation IDs and is required across all Go services. The agent writes log.Printf instead. What likely went wrong, and how would you fix it?
Answer: The agent likely did not have the shared library in its indexing scope or did not see a memory file rule that enforces the wrapper. Start by adding the library’s repository to the indexing pipeline so its symbol graph and documentation become queryable. Then create a memory file, something like AI_CONTEXT.md in the service’s root, that states: “All logging must use logging.Wrap(…).Info(…) from the shared logging package. Never use log.Printf directly.” After the next re-index, the retrieval layer will surface the wrapper’s signature and the memory file will bias the model toward using it. Track DIR for logging calls over the next few tasks to confirm the fix.
If you want deep dives on how real tools actually work under the hood every week, subscribe to Internals Decoded at internalsdecoded.com.
In the next episode we will look at how coding agents handle long running, multi-step tasks across a workweek. How they maintain state, memory, and a to do list over hundreds of actions.
Sources
- Sourcegraph indexing docs
- Kythe documentation
- Packmind context engineering
- RepoCoder paper (arXiv)
- CodeRAG paper (arXiv)
- RepoExec paper (arXiv)
- Cody remote repository context docs
- Kythe crossref service