{"slug": "embeddings-vs-grep-rethinking-the-dense-advantage-in-code-search", "title": "Embeddings vs. Grep: Rethinking the Dense Advantage in Code Search", "summary": "A new repository-level code-search benchmark, Repo-Premium, finds that the reported advantage of dense embedding encoders over BM25 may be overstated because existing benchmarks use curated queries, old code and a handicapped BM25, even as coding agents perform well with plain grep. The work questions what the dense advantage in code search actually measures.", "body_md": "Code-search benchmarks show dense encoders well ahead of BM25, yet coding agents do well with grep. What does this dense advantage measure? Existing benchmarks cannot tell: they use curated queries, old code and a handicapped BM25. To find out, we build Repo-Premium, a realistic repository-level cod", "url": "https://wpnews.pro/news/embeddings-vs-grep-rethinking-the-dense-advantage-in-code-search", "canonical_source": "https://aiflash.com/news/154723/", "published_at": "2026-11-20 06:30:17+00:00", "updated_at": "2026-11-20 06:42:19.388531+00:00", "lang": "en", "topics": ["ai-research", "ai-agents", "ai-tools", "machine-learning"], "entities": ["Repo-Premium", "BM25"], "also_reported_by": [], "takeaway": "Existing code-search benchmarks overstate dense retrieval's edge over BM25 because they rely on curated queries, old code and a weakened BM25 baseline, so evaluate retrieval on realistic repository-level queries before choosing embeddings over grep.", "recommendation": {"advice": "Benchmark code retrieval on realistic repository-level queries with a properly tuned BM25 baseline before adopting dense embeddings; curated-query benchmarks overstate the dense advantage.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/embeddings-vs-grep-rethinking-the-dense-advantage-in-code-search", "markdown": "https://wpnews.pro/news/embeddings-vs-grep-rethinking-the-dense-advantage-in-code-search.md", "text": "https://wpnews.pro/news/embeddings-vs-grep-rethinking-the-dense-advantage-in-code-search.txt", "jsonld": "https://wpnews.pro/news/embeddings-vs-grep-rethinking-the-dense-advantage-in-code-search.jsonld"}}