Search Engines vs. LLMs: Why Lexical Search Still Wins

Lexical search using inverted indexes still outperforms large language models for exact-match queries like product SKUs, achieving 100% accuracy versus variable results from vector embeddings, according to a technical analysis of search architectures. The author argues that production-grade systems should implement hybrid search combining BM25 lexical indexes with vector databases and LLM reranking, with a configuration bias toward lexical results for short queries and semantic results for long-tail queries to balance precision and relevance.

Search Engines vs. LLMs: Why Lexical Search Still Wins The Heavy Lifting of the Inverted Index The core of traditional search is the inverted index. Think of it as the index at the back of a massive textbook. Instead of scanning every page for the word "transformer," the engine looks up the term and immediately gets a list of every document containing it. This is the only reason we can query petabytes of data in under 50ms. But it's not just about matching strings. A production-grade search engine performs complex preprocessing during indexing: Tokenization & Normalization: Ensuring "running" and "run" are treated as the same concept. Stop-word Filtering: Ignoring "the" or "and" to prioritize high-signal terms. Weighting BM25/TF-IDF : Calculating the rarity of a term. If a document contains the word "ergonomic" five times, it's significantly more relevant than a document that contains the word "computer" five times. Databases are built for transactional correctness ACID compliance . Search engines are built for relevance. Trying to "just add search" to a relational database using LIKE %query% is a recipe for a system crash once your dataset hits a certain scale. Benchmarking Lexical vs. Semantic Search To understand why we need both, look at how they handle different query types. In my experience benchmarking these workflows, the failure points are predictable. Exact Match e.g., "SKU-99281-X" : Lexical: 100% accuracy. It finds the exact string. - Vector/Embedding: Variable. It might find "SKU-99281-Y" because the vectors are mathematically close, which is a failure in a logistics context. Conceptual Match e.g., "portable workstation" vs "laptop" : Lexical: 0% accuracy unless you have a manually maintained synonym list . - Vector/Embedding: High accuracy. It understands the semantic relationship. The Hybrid AI Workflow The real "pro" move isn't choosing one or the other; it's implementing a hybrid search architecture. This is the only way to achieve production-grade reliability. A typical high-performance deployment looks like this: 1. Parallel Retrieval: The system sends the query to both a BM25 lexical index and a vector database like Milvus or Pinecone . 2. Reciprocal Rank Fusion RRF : The results are merged using a scoring algorithm to balance exact matches and semantic relevance. 3. LLM Reranking: A small subset of the top 20-50 results is passed to a cross-encoder or an LLM to determine the final order based on the specific user intent. For those building an LLM agent or a RAG /en/tags/rag/ Retrieval-Augmented Generation pipeline, relying solely on embeddings often leads to "hallucinations by retrieval"—where the model gets a semantically similar but factually wrong document. Integrating a lexical step fixes this. If you're configuring a hybrid search system, avoid the mistake of giving equal weight to both scores. I've found that biasing toward lexical results for short queries 1-2 words and biasing toward semantic results for long-tail queries natural language questions yields the best precision. Example Hybrid Search Configuration Logic search params: lexical weight: 0.7 High weight for keyword precision semantic weight: 0.3 Supplemental context rerank model: "cross-encoder/ms-marco-MiniLM-L-6-v2" top k retrieval: 100 final top n: 5 The "magic" of the modern search experience isn't that the old tech disappeared; it's that we've finally found a way to layer LLMs on top of a foundation of inverted indices. Next Claude Code vs GPT-4o: Adversarial Review Workflow → /en/threads/2999/