For the last couple of years, "add RAG" became the default answer to almost every AI product question.
Need the model to understand docs? Add RAG.
Need it to answer questions over a repo? Add RAG.
Need it to stop hallucinating? Add RAG and pray a little.
RAG is still useful. I am not here to bury it. But for codebases, the default is changing. Modern AI coding agents do not always need a vector database to find the right context. A lot of the time, they need the same things a good developer uses every day: file names, grep, symbols, imports, tests, and exact source reads.
That shift matters because code is not just text. Code has names, paths, references, call graphs, package boundaries, tests, config files, and error strings. Treating it like a pile of semantically similar paragraphs can work, but it can also lose the structure that makes code understandable.
Classic RAG usually looks like this:
Repository
-> split files into chunks
-> create embeddings for each chunk
-> store vectors in a vector database
-> retrieve similar chunks for a query
-> send those chunks to the model
That flow is solid for many kinds of unstructured knowledge: support docs, PDFs, internal wiki pages, research notes, policies, transcripts. If the user asks a conceptual question and the answer may be hidden across lots of prose, semantic search helps.
But code retrieval has different pressure points.
If I ask an agent, "Why is checkout failing when the coupon is expired?", I do not only need something semantically close to "checkout" and "coupon". I may need:
A vector search might find some of that. A good coding agent will usually search more like a developer.
A practical code-search loop often looks closer to this:
User asks about a bug
-> glob for likely files
-> grep exact names, strings, routes, errors, flags
-> read promising files
-> follow imports and references
-> inspect tests
-> run the code or test suite
-> refine the search
That is not anti-RAG. It is agentic retrieval. The model does not receive one static bundle of chunks at the start. It keeps asking for better evidence as it learns more.
Example:
rg "expired coupon|coupon expired|CouponExpired" .
rg "validateCoupon|applyCoupon|coupon" src tests
rg "checkout" src/routes src/app tests
Then the agent reads the actual files instead of guessing from snippets:
Read src/services/coupons.ts
Read src/routes/checkout.ts
Read tests/checkout/coupon-expiry.test.ts
This is boring. That is the point. Boring retrieval is often better than clever retrieval when the answer depends on exact symbols.
Embeddings are great when words are fuzzy. Code often is not fuzzy.
If the bug mentions STRIPE_WEBHOOK_SECRET
, the agent should search for that exact string. If the stack trace says calculateFinalPrice
, the agent should jump to the function. If the failing test is should_reject_expired_coupon
, the agent should read that test.
Semantic similarity can miss these because it is trying to answer a softer question: "What chunk is conceptually close to this query?"
Code search often asks a harder, more literal question: "Where is this symbol defined, used, mutated, mocked, or tested?"
That is why tools like grep, glob, file reads, and language-server navigation are so useful. They preserve evidence. They give paths and line numbers. They let the agent verify what it found.
Chunking is one of the weirdest parts of RAG for code.
A function may start in one chunk and end in another. A class may depend on imports that got chopped off. A route handler may look harmless until you read the middleware above it. A test may only make sense with the fixture defined 80 lines earlier.
When chunks break structure, retrieval can return technically relevant but practically incomplete context.
This is why repository-level code retrieval research is moving toward more structure-aware methods. Some approaches combine lexical search with post-processing. Others use dependency-aware retrieval or repository graphs. The common theme is simple: code needs structure, not just similarity.
Another reason the RAG reflex is weakening: context windows got bigger.
If the useful part of a repo fits in context, the best retrieval system might be no retrieval system. Just read the files. If the agent can inspect the relevant source directly, a vector database may add more moving parts than value.
This does not mean "throw the whole repo into the prompt." That is lazy and expensive. But it does mean the agent can use a different strategy:
Search narrowly -> read complete files -> keep only what matters -> continue
That is closer to how developers work. We do not embed the repo in our brain before debugging. We search, open files, follow clues, and build a mental model as we go.
RAG is not dead. It is just not the automatic first move for every code problem.
Use vector RAG when:
For code agents, RAG can still help with:
The better pattern is usually hybrid. Let lexical search and symbol navigation handle source code. Let semantic retrieval handle messy human text. Let the agent decide which tool fits the question.
Instead of asking, "Should I use RAG?", ask this:
What kind of evidence does the agent need?
If the answer is exact evidence, use exact tools:
file paths
symbols
imports
tests
error strings
config keys
logs
If the answer is semantic evidence, use semantic tools:
docs
notes
tickets
research
policies
discussion threads
If the answer needs both, combine them.
A production-ready code agent should not be a chatbot with a vector database attached. It should be closer to a junior developer with a terminal, editor, search tools, test runner, and enough judgment to know when it has weak evidence.
If you are building AI coding tools in 2026, do not start by wiring up embeddings. Start with the boring tools:
glob
to find likely filesgrep
or ripgrep
for exact searchThen add semantic search where it earns its keep.
That last part is important. RAG is infrastructure. Every index needs chunking, syncing, invalidation, permissions, ranking, evaluation, and debugging. If grep plus file reads solve the problem, that is not primitive. That is good engineering.
RAG used to feel like the magic layer that made LLMs useful over private data. For many use cases, it still is.
But codebases are not just private data. They are executable systems with structure. The best AI agents are starting to treat them that way.
So no, RAG is not always the answer anymore.
Sometimes the answer is:
rg "the thing that broke"
And honestly, that feels very developer-coded.