Ghost Search — MCP-native, Tor-routed search layer for RAG (seeking research collaborators)

Researcher sookoothaii has released Ghost Search, a Tor-routed, privacy-preserving meta-search aggregator with 14 onion search engines, BM25 ranking, and an MCP server interface, as an open community research project on Codeberg. The tool aims to cut LLM search costs by eliminating paid APIs and persistence, offering RAM-only operation, STIX 2.1 export, and an Android app in F-Droid review. Ghost Search is positioned as a direct integration for LLM agent workflows, with 7 MCP tools and an Ollama fallback to avoid commercial dependencies.

I’m a researcher who got tired of paying 60% of my LLM budget for search access. So I built Ghost Search. I’m opening it as a community research project and looking for collaborators. Search is the bottleneck for RAG, not the LLM. A 7B model with good retrieval beats a 400B model with bad retrieval. But building a search layer that handles IP blocks, captcha walls, and cookie tracking requires infrastructure most researchers don’t have. I’m not the first to address this. OnionSearch megadose, 1.7k stars , darkdump josh0xA , darker saadejazz , and Robin apurvsinghgautam all scrape onion search engines. LibreX hnhx, 841 stars is a privacy-respecting meta-search with Tor and I2P support. Dark Watchdog IEEE 2025 uses RAG for dark web forum monitoring with a fine-tuned BERT classifier. IntelShed posted here on HF Forums combines hybrid BM25 + pgvector + cross-encoder reranking for OSINT. So the building blocks exist. What’s missing is the combination that makes it directly usable in an LLM agent workflow without paid APIs, without persistence, and without a backend. Ghost Search is a Tor-routed, privacy-preserving meta-search aggregator with 14 onion search engines, BM25 ranking, and an MCP server interface. No API keys for the search layer, no per-query fees, no commercial dependency. What makes it different from the projects above: MCP server, not just a CLI. 7 tools ghost search, ghost health, ghost engines, ghost reset engine, ghost classify, ghost stix export, ghost load blacklist , stdio transport, works with any MCP-compatible client. OnionSearch and darkdump are CLI tools. Robin has a Streamlit UI. None expose an MCP interface. If you use Claude Desktop, VS Code, or any MCP client, ghost search drops into your workflow directly. RAM-only, no persistence. No PostgreSQL, no Redis, no Elasticsearch. Search state lives in memory and is purged on screen-off, process exit, or panic wipe. darkscraper, voidaccess, and m-lally/dark-web-search all persist to databases. Ghost doesn’t. This matters for the privacy use case — there’s nothing to seize, nothing to subpoena. Android app in F-Droid review. No backend, no accounts, no telemetry. Links open in Tor Browser via intent. I’m not aware of another onion search aggregator that targets F-Droid with a RAM-only architecture. BM25 ranking with engine agreement boost. Okapi BM25 k1=1.2, b=0.75 replaces heuristic frequency sorting. m-lally/dark-web-search uses RediSearch BM25 but on a crawled index. Ghost ranks live meta-search results — no crawl, no index, no storage. STIX 2.1 export. OASIS-standard Observable bundles url, ipv4-addr, cryptocurrency-wallet, vulnerability, email-addr with relationships. voidaccess also supports STIX 2.1, but their pipeline is heavier 13-stage, PostgreSQL, relationship graphs . Ghost’s export is stateless — one tool call, one bundle, nothing stored. LLM integration with Ollama fallback. NVIDIA NIM Nemotron-3-Super- 120B primary, Ollama qwen3:8b local fallback with 60s cooldown on primary failure. Robin supports OpenAI/Claude/Gemini/Ollama but requires you to send queries through a third-party API. Ghost’s fallback means you can run the whole thing on a laptop with Ollama and never touch a paid API. Entity extraction is ReDoS-safe. BTC, XMR, PGP, email, .onion, CVE, IPv4 — linear-time regex, bounded quantifiers. darkscraper extracts similar entities but stores them in PostgreSQL. Ghost extracts for display only, nothing persisted. All filters OFF by default. Ahmia abuse blacklist, snippet liability filter, STIX export — every feature is opt-in. Results are marked, never removed. This was important for F-Droid acceptance. 180 tests, TypeScript, tsc clean. Repo: https://codeberg.org/sookoothaii/ghost-search-mcp https://codeberg.org/sookoothaii/ghost-search-mcp There are public dark web datasets — Dizzy 32,555 onion domains with category labels, arXiv 2209.07202 , DUTA 26-class illegal activity classification, EACL 2017 , CoDA 10,000 documents for linguistic analysis, NAACL 2022 . These are classification datasets. They tell you what a domain IS. What doesn’t exist is a search ranking benchmark: “given query X, which results are most relevant, and does BM25 rank them correctly?” That’s the gap I want to fill. I can collect raw results through Tor. I can’t annotate them alone and call it rigorous. I have a workstation — i9, RTX 3080 Ti, 16GB VRAM. I can run local models, fine-tune, experiment. I use that hardware to build the infrastructure. But the infrastructure isn’t for me. It’s for the researcher in a developing country with a laptop and no GPU. It’s for the student who can’t afford GPT-5 search API calls. It’s for the journalist who needs anonymous search without exposing their identity to a commercial API provider. Free, anonymous, secure, no paid dependency. That’s the design goal. I build with my hardware so others can use it without hardware. I’m one person. I can build infrastructure. I can’t do all of this: Search ranking benchmark dataset — I’m building an annotated dataset of onion search results: query, results, relevance scores. I need help with annotation guidelines and the actual annotation. Multiple annotators per result, agreement metrics, published rubric. I can collect the data. I can’t annotate it alone. Ranking evaluation against alternatives — BM25 is my baseline. Cross-encoder re-ranking is a well-established next step BEIR benchmark, MS MARCO, sentence-transformers CrossEncoderRerankingEvaluator . The game-retrieval benchmark by kimeyu showed BM25 beats dense retrieval in-domain, and reranking value is domain-dependent. Onion search might be another domain where BM25 is hard to beat. I need someone who has done retrieval evaluation to design the experiment properly. Query refinement with a fine-tuned model — I’m currently using Nemotron via API for query expansion. The INTERS dataset ACL 2024 showed instruction tuning on 20 IR tasks significantly boosts LLM performance on query understanding. SAIL EMNLP 2023 fine-tuned LLaMA-7B with search-augmented training and outperformed ChatGPT on fact-checking. FOLLOWIR NAACL 2025 fine-tuned a 7B model to follow complex IR instructions. I want to know: can a fine-tuned 7B model replace my Nemotron dependency for query refinement? I have the pipeline and the hardware to train. I need someone who has done instruction tuning to guide the methodology. MCP integration testing — If you use Claude Desktop, VS Code, or any MCP client, I want to know: does ghost search work in your RAG workflow? What’s missing? I’ve tested it locally but I haven’t seen it in someone else’s pipeline. Privacy audit — I believe the architecture is anonymity-preserving Tor-only, RAM-only, no telemetry, no accounts . I’m a developer, not a security researcher. I’d welcome a review from someone with OPSEC/Tor expertise. This is a FOSS project. MIT licensed. No commercial dependency, no surveillance, no data sale. Privacy is the product, not the cost. I’m one person. I can build the infrastructure. I can’t build the community alone. If this is useful to you, tell me. If it’s not useful yet, tell me what’s missing.