# Embeddings vs. Grep: Rethinking the Dense Advantage in Code Search

> Source: <https://aiflash.com/news/154723/>
> Published: 2026-11-20 06:30:17+00:00

**Recommendation:** Benchmark code retrieval on realistic repository-level queries with a properly tuned BM25 baseline before adopting dense embeddings; curated-query benchmarks overstate the dense advantage.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

Code-search benchmarks show dense encoders well ahead of BM25, yet coding agents do well with grep. What does this dense advantage measure? Existing benchmarks cannot tell: they use curated queries, old code and a handicapped BM25. To find out, we build Repo-Premium, a realistic repository-level cod
