{"slug": "what-hybrid-search-looks-like-when-it-has-to-run-on-your-laptop", "title": "What Hybrid Search Looks Like When It Has to Run on Your Laptop", "summary": "A developer built ctx 1.3.0, a TypeScript command-line tool published on npm as yodaman-ctx, that indexes local codebases into SQLite and retrieves code using BM25 and dense vectors fused with reciprocal rank fusion, answering questions through a local Ollama-served model with no cloud provider in the path. The tool routes files by type to tree-sitter, section-based or size-based chunkers, and its changelog reports that fixing an indexing gap so every line lands in exactly one segment raised top-five answer coverage from 21% to 69% across 42 code questions, per the project's own measurements.", "body_md": "Suppose you are working in a codebase you did not write, and you type the following question:\n\nWhere is `ERR_CONN_RESET` handled?\n\nThe answer exists. One file contains a `catch` block that checks for exactly that constant and retries the request.\n\nA purely semantic search may not put that file first. It may return a connection-pool module, a general error-handling utility and a retry helper. All three are about the right topic. None of them necessarily contains the string you asked for.\n\nNow consider a second question:\n\nHow do we stop a user from logging in too many times?\n\nThe relevant function is called `throttleAuthAttempts`. Its body talks about a counter and a lockout window. It never uses the words \"logging in\" or \"too many times\". A keyword search may miss it. An embedding model has a much better chance of placing it near the question.\n\nThis is the trade-off that hybrid retrieval exists to address, and it has been described well elsewhere: lexical search preserves exact identifiers, dense search preserves meaning, and neither preserves both. What is described less often is what happens after you accept that argument and actually build the system. Which BM25 parameters do you use? How does the tokenizer treat `getUserById`? What does a fused score of 0.03 mean to a user? Where do the chunks come from in the first place?\n\n[ctx](https://github.com/Yoda-Man/ctx) is a useful place to look for answers, because it is small enough to read and has to make all of those decisions in code. It is a TypeScript command-line tool (published on npm as `yodaman-ctx`, a maintained fork of an earlier project called Context Expert) that indexes local projects into SQLite, searches them with BM25 and dense vectors fused by RRF, and answers questions through a local model served by Ollama. There is no cloud provider in the code path.\n\nEverything below describes ctx 1.3.0; file paths are relative to the repository root. This article walks through that pipeline in the order data moves through it: chunking, embedding, the two retrievers, fusion, optional reranking, context assembly and evaluation. At each stage, the same two questions apply: what does this step preserve, and what does it give up?\n\n*A retriever can only return the units you gave it. In code, choosing those units is most of the work.*\n\nRetrieval papers usually start with a corpus of passages. A code-search tool has to create those passages first, and the choice determines what can be found at all.\n\nctx routes each file by type in `src/indexer/chunker/chunker.ts`. Config files become a single segment. Markdown goes to a section-based extractor. TypeScript, JavaScript and Python go to a tree-sitter extractor. Every other language is treated as one segment and split by size.\n\nThe tree-sitter extractor (`code-extractor.ts`) walks the top-level statements of the file in order. Each definition — a function, class, interface, type alias, enum, or a `const` bound to an arrow function — becomes its own segment, together with its `export` keyword, decorators, and any comments that sit directly above it with no blank line in between. Everything that is not a definition (imports, constants, module setup, a Python `if __name__ == \"__main__\":` block) is grouped with its neighbours into a `module` segment. The comment at the top of the file states the invariant directly: every line of the file lands in exactly one segment.\n\nThat invariant is worth noticing because, according to the project's changelog, it was not always true. Version 1.2.0 records that earlier indexing kept only functions and classes, so constants and module-level code were in no chunk and could not be found. The maintainers report that on 42 code questions, 23 answers were previously in no chunk at all, and that after the change the answer appeared in the top five results for 69% of questions instead of 21%. Those are the project's own measurements, not independent benchmarks, but they illustrate a point that is easy to forget: no ranking function can retrieve text that was never indexed.\n\nMarkdown follows the same principle at a different granularity. `markdown-extractor.ts` uses the `marked` lexer to find headings — so a `#` inside a code fence is not mistaken for one — and makes each heading plus everything under it, up to the next heading, one segment. A table stays attached to its title. A heading with nothing under it is merged into the next section.\n\nSegments that fit within a size limit are kept whole. The limits are set per content type in `chunker/config.ts`:\n\n| Content type | Target size | Overlap | \n|---|---|---|\n| code | 512 tokens | 50 tokens | \n| docs | 1,024 tokens | 100 tokens | \n| config | 256 tokens | 25 tokens | \n\n\"Tokens\" here is an estimate, not a tokenizer count: `estimateTokens` returns `Math.ceil(text.length / 4)`. Oversized code segments are split by a recursive character splitter. Oversized documentation segments are split by a semantic chunker that embeds sentences and starts a new chunk where similarity between neighbours drops below 0.5. If that embedding call fails, the code catches the error, records a warning, and falls back to the size-based splitter instead of aborting the index.\n\n**What this step preserves:** the boundaries a reader would recognise — a whole function with its doc comment, a whole section with its heading.\n\n**What it gives up:** precision on very large units. A 2,000-line class is still one definition, so it will be split by size, and the four-characters-per-token estimate means the real token count of a chunk can differ from the configured limit.\n\n*ctx uses a single local embedding model and compares every vector on every query.*\n\nThe default embedding model is `BAAI/bge-large-en-v1.5`, which the provider rewrites to `Xenova/bge-large-en-v1.5` so that it can run through `@huggingface/transformers` in the Node process. It produces 1,024-dimensional vectors, L2-normalised. For BGE models the vector is taken from the first ([CLS]) token, which is how BAAI trained them; other models use the mean of all tokens. Until version 1.3.0, ctx mean-pooled BGE as well, so indexes built with earlier versions have to be rebuilt. The README states that the model is about 340 MB and is cached in `~/.ctx/models`. An Ollama embedding model can be configured as a fallback, but the provider code refuses a fallback whose dimension differs from the primary model's, because the stored vectors would no longer be comparable.\n\nAt query time, the vector store is not an approximate nearest-neighbour index. In `src/search/store.ts`, chunks are loaded from SQLite into an in-memory store with `distanceMetric: 'cosine'`, and `memory-store.ts` scores the query against every stored `Float32Array`, sorts, and slices the top *k*. In other words, it is a brute-force scan.\n\nThe comment above that code explains the choice. ctx builds this store afresh in every process, so the cost of building an HNSW graph would be paid on every search. The comment records 8.3 seconds to build HNSW over 3,714 vectors, compared with 43 ms for a scan of 58,728 vectors. The changelog adds that HNSW also held a second copy of every vector and could return few or no results under narrow metadata filters, and that it was removed in 1.2.0.\n\nThis is a decision that only makes sense in context. For a long-running server with millions of vectors, an approximate index is the normal choice. For a CLI that starts, answers one question and exits, the build cost dominates, and an exact scan over tens of thousands of vectors is both simpler and fast enough.\n\nTwo loading details matter at this scale. Chunks are read in batches paged by `rowid > ?` rather than by `OFFSET`, which the comment explains turned a 16-second load of a 23,000-chunk project into an index seek per page. And every loaded embedding is checked against the expected dimension; a mismatch throws an error that tells the user to re-index with `--force`, rather than silently comparing incompatible vectors.\n\n**What this step preserves:** paraphrase and intent. A question about \"logging in too many times\" can land near `throttleAuthAttempts`.\n\n**What it gives up:** any explicit guarantee that a rare exact string will dominate. The identifier has to compete with everything else in the chunk for influence over a single 1,024-dimensional vector.\n\n*The formula is the textbook one. The interesting decisions are in the tokenizer.*\n\nctx's BM25 retriever lives in `src/sdk/rag/retrieval/bm25.ts`, inside a vendored copy of the parts of the ContextAI SDK the tool uses. It has no external dependencies. The parameters are the common defaults, `k1 = 1.2` and `b = 0.75`, and the IDF is the Robertson–Spärck Jones variant with a `+1` inside the logarithm so it never goes negative:\n\n``` js\n// IDF formula: log((N - df + 0.5) / (df + 0.5) + 1)\nconst idf = Math.log((docCount - docFreq + 0.5) / (docFreq + 0.5) + 1);\n```\n\nThe per-term score is the standard saturated, length-normalised term frequency:\n\n``` js\nconst numerator = tf * (this.k1 + 1);\nconst denominator =\n  tf + this.k1 * (1 - this.b + this.b * (doc.length / this.avgDocLength));\nscore += idf * (numerator / denominator);\n```\n\nHere, a \"document\" is a chunk. Candidate chunks are found from the union of the query terms' posting lists, filtered by file type or language when asked, scored, sorted, cut to *k*, and finally divided by the top score so the best BM25 hit is 1.0.\n\nThe part that most affects code search is not the formula but the tokenizer:\n\n``` js\nconst CAMEL_BOUNDARY = /(?<=\\p{Ll})(?=\\p{Lu})|(?<=\\p{Lu})(?=\\p{Lu}\\p{Ll})/u;\n\nfor (const word of text.split(/[^\\p{L}\\p{N}]+/u)) {\n  const parts = word.split(CAMEL_BOUNDARY);\n  if (parts.length > 1) tokens.push(word.toLowerCase());\n  for (const part of parts) tokens.push(part.toLowerCase());\n}\nreturn tokens.filter((token) => token.length >= 2);\n```\n\nEvery run of characters that is not a Unicode letter or digit becomes a separator. A camelCase word is kept whole and also split at its case boundaries. Tokens shorter than two characters are dropped. Running it on a few identifiers shows what it does:\n\n| Input | Tokens | \n|---|---|\n| `ERR_CONN_RESET` | `err` ,`conn` ,`reset` | \n| `ZX-4821` | `zx` ,`4821` | \n| `getUserById` | `getuserbyid` ,`get` ,`user` ,`by` ,`id` | \n| `HTTPServer` | `httpserver` ,`http` ,`server` | \n| `get user by id` | `get` ,`user` ,`by` ,`id` | \n\nKeeping both forms serves two kinds of query. A query that contains `getUserById` verbatim still matches the whole-word token, which is rare and so carries a high IDF — exactly what BM25 is for. A query that says \"get user by id\" in plain words now matches the parts. Before version 1.3.0, the tokenizer kept camelCase as one token only, and that second query had no lexical match on the identifier at all.\n\nOther limits remain. `ZX-4821` becomes two tokens, so a chunk containing `ZX-4812` still matches on `zx`, and only the numeric token distinguishes the two. The parts of a camelCase word are also common words (`get`, `user`, `id`), so their IDF is low; they add a modest amount to a chunk's score rather than deciding it. It is still worth running a retriever's tokenizer on your own identifiers before assuming what \"exact match\" means.\n\n**What this step preserves:** exact rare strings — constants, error codes, function names typed verbatim — and, through the split parts, a lexical link between an identifier and the words it is made of.\n\n**What it gives up:** any connection between words that do not share a token, such as \"authenticate\" and `checkLogin`.\n\n*Reciprocal Rank Fusion solves the incompatible-scores problem. It creates a readability problem of its own.*\n\n`FusionService.search` in `src/search/fusion.ts` runs the dense and BM25 searches in parallel with `Promise.all`, then fuses the two ranked lists. The underlying function is the standard one, with `k = 60`:\n\n$$\n\n\\text{RRF}(d) = \\sum_{r \\in R} \\frac{1}{60 + \\text{rank}_r(d)}\n\n$$\n\nA chunk that is missing from one list receives zero from that list. The implementation also records each retriever's contribution, so a fused result carries a record of which list it came from and at what rank.\n\nRaw RRF scores are small. A chunk ranked first by both retrievers scores 2/61 ≈ 0.033. The changelog for 1.2.0 describes the practical consequence: results showed fusion values near 0.03, which looked like poor matches, and any `minScore` above that value removed every result. The fix is three lines:\n\n```\n// Raw RRF tops out near 0.03 (2/61), which read as a poor match and made\n// any 0-1 minScore drop every result; divide by the best possible score.\nconst bestPossible = (denseWeight + bm25Weight) / (k + 1);\n```\n\nAfter this division, 1.0 means \"ranked first by both\", and about 0.5 means \"ranked first by one and absent from the other\". The order of results does not change; only the scale does. Applying it to a small example:\n\n| Chunk | BM25 rank | Dense rank | Scaled score | \n|---|---|---|---|\n| A: the `catch` block for`ERR_CONN_RESET` | 2 | 3 | 0.976 | \n| B: a changelog line mentioning `ERR_CONN_RESET` | 1 | 8 | 0.949 | \n| C: a general retry helper | — | 1 | 0.500 | \n\nChunk A is first in neither list but first after fusion, because both retrievers placed it near the top. This is the behaviour hybrid search is designed for.\n\nThere is one practical constraint on this example. Each retriever is called with the same `topK` as the final result count. `ctx search` defaults to 10, so a dense rank of 8 is inside the list. `ctx ask` uses `final_k`, which defaults to 5, so each retriever contributes at most five candidates and chunk B's dense rank of 8 would not be seen at all. In that case, B would score 1/61 ÷ 2/61 = 0.5. The candidate depth of each retriever directly bounds how much agreement fusion can detect.\n\nIt is tempting to conclude that deeper lists are always better. A comment in `fusion.ts` records a test that points the other way. Fetching 50 candidates from each retriever lost an expected file on the project's eight-question fixture, with recall falling from 0.94 to 0.88. With *k* = 60, a chunk ranked seventh by both retrievers scores 2/67 ≈ 0.030, while a chunk ranked first by one and absent from the other scores 1/61 ≈ 0.016. When the lists are deep enough to contain most of a small corpus, middling agreement can outrank a strong single signal. The fixture is too small to settle the question for large projects, so the code keeps candidate depth equal to `topK`.\n\n`minScore` is also applied only after fusion. Before version 1.3.0, the same threshold was also passed to the dense search, where it was compared with raw cosine similarity, and to BM25, where it was compared with a score scaled to the best hit — three different scales for one number.\n\nThe code also supports per-retriever weights; when either weight differs from 1.0, contributions are multiplied and the list is re-sorted. Nothing in the CLI's configuration exposes these weights, so at the default settings both retrievers count equally.\n\n**What this step preserves:** agreement. A chunk supported by both kinds of evidence rises above a chunk supported by only one.\n\n**What it gives up:** the magnitude of the original scores. A BM25 match that is far stronger than everything else and a BM25 match that barely beats second place both count as rank 1.\n\n*A cross-encoder is often presented as a free improvement. In this project, it was measured and made optional.*\n\nctx includes a cross-encoder reranker (`src/search/reranker.ts`), using `Xenova/bge-reranker-base`. When enabled, it takes up to 50 fused candidates, scores each query–document pair, and returns the top *k*. In practice the limit is lower: each retriever contributes `topK` results, so the reranker sees at most 20 candidates at the default of 10. If the model cannot load, the code warns once and returns the results in their fused order rather than failing the search.\n\nIt is not enabled by default. It runs only with `ctx search --rerank`; `ask` and `chat` do not rerank.\n\nThe changelog gives the reason. On 42 questions over two projects, each scored by whether the chunk containing the answer came back, the maintainers report that hybrid search alone put the answer first 45% of the time against 26% with reranking, with mean reciprocal rank 0.56 against 0.45. Their note is that the cross-encoder helped with prose but ranked Markdown tables and code lower. The same release also records that the reranker had previously been passed the query without the document, so every result received the same score — a bug that would have been easy to miss without measuring.\n\nThis is a small sample on two projects, and it should not be generalised to a claim about cross-encoders. What it does show is that a stage that improves results on one kind of content can make them worse on another, and that the only way to know which applies is to measure on your own content.\n\n*Retrieval ends with a ranked list. The model sees something else.*\n\nFor `ctx ask`, retrieved chunks pass through an assembler (` src/sdk/rag/assembly/base-assembler.ts`) before reaching the model. The steps run in a fixed order:\n\n`final_k` results (default 5).`sandwich`).`<source>` carrying its file path and line range.\nThe sandwich ordering is a response to the observation that models tend to use information at the start and end of a long context more reliably than information in the middle. With *n* results and the default start count of ⌈*n*/2⌉, the top half goes first and the remainder goes at the end in reverse order. For five results, the order becomes 1, 2, 3, 5, 4.\n\nThe order of steps 3 and 4 matters, and until version 1.3.0 it was the other way round. When the budget ran *after* ordering, the chunks it considered last were those at the end of the sandwich — the positions the ordering reserves for the fourth- and fifth-ranked results — so a tight budget cut those first, while the least relevant chunks in the middle survived. Now the budget sees chunks in relevance order and drops the least relevant, and the sandwich is built from what remains. A test in `src/agent/__tests__/assembler.test.ts` covers this: with room for four of five chunks, the result is 1, 2, 4, 3, where the old order would have kept 1, 2, 3, 5.\n\nThe budget loop still includes any chunk that still fits rather than stopping at the first that does not, so a short, lower-ranked chunk can be included after a long one is skipped. With five documentation chunks near the 1,024-token limit, a 4,000-token budget holds three of them.\n\nIt is worth inspecting the final context rather than only the ranked list, because stages like these interact. ctx makes that straightforward: `ctx ask --context-only --json` returns the assembled context without calling the model.\n\n*Searching every indexed project is the safe default until there are a dozen of them.*\n\nctx can index many projects and search one or all of them. In `ctx chat`, a question first goes through `QueryIntentClassifier` in `src/agent/query-router.ts`, which uses string matching and no model call. Its rules, in order:\n\n| Condition | Intent | Confidence | \n|---|---|---|\n| Two or more project names mentioned | multi-project | 0.95 | \n| A multi-project keyword (e.g. \"compare\", \"across\") and at least one project name | multi-project | 0.85 | \n| Exactly one project name mentioned | single project | 0.9 | \n| No name, but a phrase like \"this project\" or \"this repo\" | single project | 0.85 | \n| A multi-project keyword with no project name | general | 0.6 | \n| Anything else | general | 0.5 | \n\nA comment in the code explains the second-to-last rule: \"compare error handling\" with no project named is deliberately not classified as multi-project, to avoid false positives.\n\nWhat happens to a \"general\" question depends on state. If a project is focused (with `/focus`), the question stays in that project. Only when nothing is focused and at least two projects exist does the router ask the local model, with a 5-second timeout and a confidence threshold of 0.7; if the call fails or the confidence is too low, it searches every project. The changelog explains the ordering: when the model was consulted first, a local model often exceeded the 5-second limit, the fallback searched every project, and with a dozen projects that took about 50 seconds — past the agent's 30-second tool limit — so answers arrived with no sources.\n\nWhen multiple projects are searched, the multi-project service applies RRF at two levels: the dense and BM25 managers each merge results across projects, and the service then fuses the dense and BM25 lists.\n\n*If the right chunk never enters the context, no prompt can recover it.*\n\nctx ships an evaluation command, `ctx eval`, with metrics implemented directly in `src/eval/metrics.ts`: reciprocal rank, precision@k, recall@k, hit rate, NDCG@k with binary relevance, and average precision. Each golden entry pairs a question with the file paths expected in the results:\n\n```\n{\n  \"id\": \"fixture-002\",\n  \"query\": \"Where is the JWT token verified?\",\n  \"expectedFilePaths\": [\"src/auth/middleware.ts\"],\n  \"tags\": [\"auth\"],\n  \"source\": \"manual\"\n}\n```\n\nThe repository includes a small fixture project (eight source files) and a golden set of eight queries in `fixtures/eval/`. A GitHub Actions workflow runs this evaluation on pull requests that touch search, indexing, agent or eval code, and weekly on a schedule. `scripts/check-eval-thresholds.js` fails the build if MRR falls below 0.6, precision@k below 0.5, or hit rate below 0.8. The script's comment describes these as a minimum quality floor, intentionally lower than the configuration defaults of 0.7, 0.6 and 0.85.\n\nEight queries over eight files is a regression check, not a benchmark, and the project does not present it as one. Relevance is also judged at the file level, so a result counts as relevant if it comes from the right file even if it is the wrong chunk of that file. Within those limits, it does the job it is designed for: a change that breaks retrieval on a known set of questions fails before it is merged.\n\nThe core of ctx's retrieval is short. RRF is a loop over two lists. BM25 is two formulas and an inverted index. Cosine similarity over normalised vectors is a dot product. The original argument for hybrid search — that lexical and semantic retrieval preserve different evidence — is fully visible in a few hundred lines.\n\nMost of the remaining code, and most of the changelog, is about everything around those lines: which units get indexed, how identifiers are tokenised, what a score of 0.03 means to a person reading it, how many candidates each retriever contributes, whether a reranker helps on this kind of content, and what survives a token budget. Several of the fixes recorded in the changelog were not ranking improvements at all. They were cases where the right text was not indexed, where a score was misleading, or where a stage was silently doing nothing.\n\nThat is the practical lesson from reading this codebase. Choosing hybrid search is a decision that takes a paragraph to justify. Making it work takes attention to the steps before and after it.\n\nIf you are building retrieval over source code, a few checks follow directly from the code above:\n\nctx is MIT-licensed and available at [github.com/Yoda-Man/ctx](https://github.com/Yoda-Man/ctx). The files referenced in this article are under `src/search/`, `src/indexer/chunker/`, `src/sdk/rag/` and `src/eval/`.\n\nRobertson, S., & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. *Foundations and Trends in Information Retrieval*, 3(4), 333–389. [https://doi.org/10.1561/1500000019](https://doi.org/10.1561/1500000019)\n\nCormack, G. V., Clarke, C. L. A., & Büttcher, S. (2009). Reciprocal Rank Fusion outperforms Condorcet and individual rank learning methods. *Proceedings of SIGIR 2009*, 758–759. [https://doi.org/10.1145/1571941.1572114](https://doi.org/10.1145/1571941.1572114)\n\nLiu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. *Transactions of the Association for Computational Linguistics*, 12, 157–173. [https://doi.org/10.1162/tacl_a_00638](https://doi.org/10.1162/tacl_a_00638)\n\nXiao, S., Liu, Z., Zhang, P., Muennighoff, N., Lian, D., & Nie, J.-Y. (2024). C-Pack: Packed Resources For General Chinese Embeddings. *Proceedings of SIGIR 2024*. arXiv:2309.07597. (The paper introducing the BGE embedding models.)\n\nYoda-Man. *ctx* source code and CHANGELOG.md, version 1.3.0. [https://github.com/Yoda-Man/ctx](https://github.com/Yoda-Man/ctx)", "url": "https://wpnews.pro/news/what-hybrid-search-looks-like-when-it-has-to-run-on-your-laptop", "canonical_source": "https://dev.to/trust_mutemasango_c24e776/what-hybrid-search-looks-like-when-it-has-to-run-on-your-laptop-12b3", "published_at": "2026-11-06 09:06:18+00:00", "updated_at": "2026-11-06 09:23:29.460787+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-search", "large-language-models"], "entities": ["ctx", "yodaman-ctx", "Ollama", "SQLite", "Context Expert", "tree-sitter", "TypeScript"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/what-hybrid-search-looks-like-when-it-has-to-run-on-your-laptop", "markdown": "https://wpnews.pro/news/what-hybrid-search-looks-like-when-it-has-to-run-on-your-laptop.md", "text": "https://wpnews.pro/news/what-hybrid-search-looks-like-when-it-has-to-run-on-your-laptop.txt", "jsonld": "https://wpnews.pro/news/what-hybrid-search-looks-like-when-it-has-to-run-on-your-laptop.jsonld"}}