{"slug": "show-hn-gsearch-real-google-search-for-ai-agents-via-a-real-firefox-mcp", "title": "Show HN: Gsearch – Real Google Search for AI Agents, via a Real Firefox (MCP)", "summary": "A developer released gsearch, an open-source MCP server that runs a real Firefox browser via Camoufox to return Google's live results page as typed JSON, including resolved links, the AI Overview, People also ask and related searches, with no API key required. The tool installs through `uv tool install gsearch-agent` or `pipx install gsearch-agent`, runs on the user's own IP at a default cap of 20 searches per hour, and supports any proxy format for higher volume. The project positions itself against SerpApi and Bright Data (per-query pricing) and Brave, Tavily, Exa and DuckDuckGo (non-Google indexes), and warns that automated access to Google Search may breach Google's Terms of Service, noting Google's official Custom Search API closes on 1 January 2027.", "body_md": "**[Quick start](#-quick-start)** ·\n  **[How it compares](#-how-it-compares)** ·\n  **[What you get](#-what-your-agent-gets)** ·\n  **[How it works](#-how-it-works)** ·\n  **[Proxies](#-proxies)** ·\n  **[Performance](#-performance)** ·\n  [FAQ](#-faq)\n\nYour agent calls `google_search` and gets Google's actual results page as typed JSON: every link resolved to\nits real URL, plus the AI Overview, People also ask and related searches. A real Firefox\n([Camoufox](https://github.com/daijro/camoufox)) runs Google's page as it is, on your own IP or through any\nproxy you already have.\n\n**Why:** plain HTTP gets a JavaScript wall, automated Chrome got a CAPTCHA 2 out of 2 times in our test, and\nGoogle's official Custom Search API closes on 1 January 2027. Other search MCP servers either aren't Google's\nindex (Brave, Tavily, Exa, DuckDuckGo) or charge per query (SerpApi, Bright Data).\n\n<sub>A real, live call on a home IP with a warm session, rendered from the recorded result; the wait shown is the measured time. [MP4](https://github.com/RayenMlayeh/gsearch/blob/main/docs/assets/demo.mp4)</sub>\n\n```\nuv tool install gsearch-agent                         # or: pipx install gsearch-agent\ngsearch setup                                         # downloads the browser once (~1 GB), prints your MCP config\nclaude mcp add gsearch --scope user -- gsearch-mcp    # or any MCP client, below\n```\n\nThen just ask: *\"Search Google for what asyncio is and give me the top three links\"*, *\"What do people also\nask about asyncio?\"*, *\"Find asyncio pages on python.org from the past year.\"*\n\nWithout a proxy, gsearch uses your own IP, at most 20 searches an hour. For more, add a proxy in any format:\n\n```\nclaude mcp add gsearch --scope user -e GSEARCH_PROXY=\"gate.example.com:7000:user-sid-{sid}:password\" -e GSEARCH_DATA_LIMIT_MB=1000 -- gsearch-mcp\n```\n\n## **Claude Desktop, Cursor and other MCP clients**\n\n`gsearch setup` prints this block with the full path of `gsearch-mcp` filled in. Use the full path if your\nclient doesn't find the command.\n\n```\n{\n  \"mcpServers\": {\n    \"gsearch\": {\n      \"command\": \"gsearch-mcp\",\n      \"env\": { \"GSEARCH_PROXY\": \"optional, any format\", \"GSEARCH_DATA_LIMIT_MB\": \"1000\" }\n    }\n  }\n}\n```\n\n**Claude Desktop:** `claude_desktop_config.json`. **Cursor:** `~/.cursor/mcp.json`, or `.cursor/mcp.json` in a\nproject.\n\n**No install, with uv:** use `\"command\": \"uvx\", \"args\": [\"gsearch-agent\"]`, after fetching the browser once\nwith `uvx --from gsearch-agent gsearch setup`.\n\n**Settings in a file:** put them in a `.env` (see [`.env.example`](https://github.com/RayenMlayeh/gsearch/blob/main/.env.example)) and set\n`GSEARCH_ENV_FILE` to its absolute path instead.\n\nNote\n\nAutomated access to Google Search may breach Google's Terms of Service. Read\n[Legal and responsible use](#-legal-and-responsible-use) before running live searches.\n\n## **The same comparison as text**\n\n|  | **gsearch** | SerpApi, Bright Data | Brave, Tavily, Exa | DuckDuckGo, SearXNG MCPs | Playwright + Chromium Google MCPs | \n|---|---|---|---|---|---|\n| Google's own results | ✅ | ✅ | ❌ their own index | ❌ / depends on the instance | ✅ | \n| API key or account | **none** | required | required | none | none | \n| Cost | **free** (+ your proxy, optional) | per query | usage-based plans | free | free | \n| Runs on your machine | ✅ | ❌ hosted | ❌ hosted | ✅ | ✅ | \n| AI Overview, People also ask | ✅ | ✅ (SerpApi) | ❌ | ❌ | varies | \n| Google's redirect links resolved to real URLs | ✅ | ✅ | n/a | n/a | varies | \n| Blocks reported as errors with a retry time | ✅ | n/a | n/a | varies | varies | \n| Terms-of-Service risk is yours | **yes** ([Legal](#-legal-and-responsible-use) ) | the provider's | no | low | yes | \n\n<sub>\"Varies\": we did not test those projects. Our only Chromium measurement is\n[F-0001](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0001-cold-start-chrome-vs-camoufox.md): Playwright Chrome was sent to a CAPTCHA 2 of 2\ntimes on an IP where gsearch was served. Read [Legal](#-legal-and-responsible-use) before running live searches.</sub>\n\n<sub>Measured in October 2026:\n[F-0014](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0014-eh1-hybrid-session-reuse.md),\n[F-0006](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0006-fixture-provenance-and-serp-anatomy.md),\n[F-0011](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0011-goto-resolution-leak-and-fix.md),\n[F-0012](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0012-headless-vs-headed-fingerprint.md),\n[F-0001](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0001-cold-start-chrome-vs-camoufox.md);\n14 s and 0.6 MB on a clean proxy IP: [F-0015](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0015-b1-residential-proxy-benchmark.md).\nFlagged proxy pools are slower: see [Performance](#-performance).</sub>\n\nGoogle's official search API is closed to new customers and shuts down on 1 January 2027\n([F-0005](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0005-custom-search-api-sunset.md)). gsearch is built for AI agents instead:\n\n- 🦊 **A real browser, not a scraper.** Camoufox, a hardened Firefox, runs Google's page as it is, with a\nreal-device fingerprint and a locale that matches its IP.\n- 🔗 **Real URLs.** Google hides result links behind`/goto` redirects. gsearch resolves every one to the\npage it points to, without loading that page.\n- 🤖 **Built for agents.** One MCP server, two tools, typed JSON. Blocks come back as clear errors with`retry_after_s` , never as empty results, and every string is flagged as untrusted web content.\n- 🌍 **Any proxy, any format.** Rotating residential, static VPS or datacenter lists, HTTP, HTTPS or SOCKS5,\nin any common layout. Every byte is counted, with a hard data limit.\n- 🎯 **Nothing to tune.** Two settings. Everything else is fixed to what worked best in testing.\n- 📊 **Honest numbers.** Every claim in this README links to a write-up of how it was measured.\n\nReal output from a saved results page (offline demo mode, where `url` stays `null`; a live search fills it):\n\nA block is never an empty list. It comes back as an error that says what happened and how long to wait:\n\n## **Every field explained**\n\n| Field | What it holds | \n|---|---|\n| `organic[].url` | The real destination of each result, resolved from Google's `/goto` redirect in the same session. | \n| `organic[].title` ,`snippet` ,`date` ,`domain` ,`display_url` | What the result shows, as text. `link` keeps the exact link Google served. | \n| `organic[].sitelinks` ,`source_meta` | A result's sub-links, and what Google shows instead of a URL on forum and video results ( `2 years ago` ). | \n| `ai_overview` | The AI Overview text and its sources, when Google shows one. | \n| `people_also_ask` ,`related_searches` | The questions and related queries on the page. | \n| `ads` | Text ads, when the market serves them. | \n| `pagination` | Current page and next page. Ask for more results with `page` . | \n| `meta.untrusted_content` | Always `true` : every string is third-party web text (see[prompt injection](#prompt-injection) ). | \n\nError codes: `captcha`, `soft_block`, `hard_block`, `consent_wall`, `js_gate`, `rate_limited`,\n`parse_drift`, `network`, `proxy`, `backend_unavailable`, `invalid_request`. The full JSON Schema is in\n[`serp.schema.json`](https://github.com/RayenMlayeh/gsearch/blob/main/packages/gsearch-parse/src/gsearch_parse/serp.schema.json).\n\n- **Like a person.** A visible window, because headless sessions were challenged far more often on proxy IPs\n([F-0015](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0015-b1-residential-proxy-benchmark.md) ). The query is typed into the address\nbar. WebRTC is off, so your own IP can't leak around the proxy.\n- **Never forged.** gsearch never generates or alters Google's tokens: the browser runs the page as it is.\n- **Cheap links.** Links are resolved on the same IP with the same cookies and a TLS fingerprint matching the\nbrowser's ([F-0011](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0011-goto-resolution-leak-and-fix.md) ,[F-0013](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0013-tls-camoufox-vs-curl-cffi.md) ).\n- **Runs on servers.** On a Linux machine without a screen, the browser uses a virtual display\nautomatically (`sudo apt install xvfb` ).\n\n```\n# rotating residential: {sid}, or your provider's sid-…, session-…, sessid-… part, gives a new IP per search\nGSEARCH_PROXY=gate.example.com:7000:user-country-us-sid-{sid}:password\n\n# static proxies (VPS, datacenter, ISP): one or many, separated by commas, spaces or new lines\nGSEARCH_PROXY=socks5://user:password@203.0.113.10:1080, http://203.0.113.11:3128\n\n# or a file with one proxy per line\nGSEARCH_PROXY=/path/to/proxies.txt\n```\n\n- **Rotating:** a new IP and a new browser for every search, up to 3 at once. A blocked IP is replaced, up to\n4 IPs per search.`country-xx` or`cc-xx` in the login is checked, and exits elsewhere are dropped.\n- **Static:** each IP keeps one warm session, and the IPs take turns. Each IP has its own budget (20 searches\nan hour, 150 a day) and rests until midnight after a CAPTCHA, so ten IPs allow up to about 200 searches an\nhour.\n- **Data:** every byte goes through a local relay that counts it, and`GSEARCH_DATA_LIMIT_MB` stops\neverything at your limit, across restarts.\n\nTip\n\nCheck your proxy provider's acceptable-use policy. Many forbid using their network on sites whose terms prohibit automated access, and forbid solving CAPTCHAs.\n\n**Proxy quality decides speed and cost.** Many residential IPs are already flagged by Google. Each flagged\nIP costs a browser launch (about 0.7–1 MB and 20 s) before the next one is tried.\n\n|  | Expected with a high-quality rotating proxy | Measured on a low-quality pool ( [F-0015](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0015-b1-residential-proxy-benchmark.md) ) | \n|---|---|---|\n| Fresh IPs served by Google | ~85% or more (assumed) | 1 in 3 to 1 in 5 | \n| Time per search | ~15–20 s (14 s measured on a clean IP) | 40–75 s | \n| Proxy data per search | ~0.6–1 MB (0.6 MB measured on a clean IP) | 4.7–6.3 MB | \n| Searches per hour (3 browsers) | ~500–600 | ~72 | \n| 1,000 searches | ~0.6–1 GB, under 2 hours | ~5–6.5 GB, ~14 hours | \n\nWithout a proxy, a warm session on your own IP takes **~1.2 s per search**, with every link resolved and 0\nCAPTCHAs in testing ([F-0014](https://github.com/RayenMlayeh/gsearch/blob/main/docs/findings/F-0014-eh1-hybrid-session-reuse.md)), capped at 20 searches an\nhour to protect your IP. Static proxies have not been benchmarked yet.\n\n**Measure your own proxy** within your data limit:\n\n```\npython tools/benchmark.py -n 10\n```\n\n## **Settings** (five in total; most people need the first two)\n\n| Setting | What it does | \n|---|---|\n| `GSEARCH_PROXY` | Your proxies, in any format above, or a file listing them. Unset: your own IP. | \n| `GSEARCH_DATA_LIMIT_MB` | Stop all proxy traffic at this many MB, counted across restarts. Unset: no limit. | \n| `GSEARCH_ENV_FILE` | Absolute path of your `.env` . Needed in MCP client configs. If set, the file must exist. | \n| `GSEARCH_FIXTURES_DIR` | Answer from saved pages instead of Google (demos and tests). | \n| `GSEARCH_STATE_DIR` | Where budgets, the data meter and saved block pages live. Default: `%LOCALAPPDATA%\\gsearch` or`~/.local/share/gsearch` . | \n\nEverything else is fixed to what worked best in testing: a new IP per search with rotating proxies, a visible browser, address-bar search, real-device fingerprints, 4 attempts per search, always resolving URLs, and a 10-minute cache for repeated queries.\n\n## **MCP tools, CLI and Python**\n\n| MCP tool | Arguments | Returns | \n|---|---|---|\n| `google_search` | `query` , and optionally`page` (1–50),`lang` (`en` ,`fr` , …),`country` (`us` , …),`time_range` (`h` ,`d` ,`w` ,`m` ,`y` ),`site` (a domain) | The JSON above plus a short text summary, or an error result on a block. | \n| `google_search_status` | none | Proxy data used against the limit, IPs used and blocked, and the cache. Never contacts Google. | \n\n```\ngsearch search \"query\" [--page N] [--lang xx] [--country xx] [--time-range h|d|w|m|y] [--site domain]\ngsearch search \"query\" --fixtures fixtures/gecko     answer from saved pages (offline, from a clone of this repo)\ngsearch parse page.html                              parse any saved Google results page\ngsearch schema                                       print the JSON Schema\ngsearch identity check                               check the browser's fingerprint matches its IP (no Google traffic)\ngsearch setup                                        download the browser once and print the MCP config\n```\n\nExit codes: `0` results, `2` usage or settings error, `3` blocked or other error, `4` Google changed its\nlayout.\n\n``` python\nimport asyncio\nfrom gsearch_core.client import SearchClient\nfrom gsearch_core.config import fetcher_from_env\nfrom gsearch_parse.models import SearchRequest\n\nasync def main() -> None:\n    fetcher, _live = fetcher_from_env()      # reads the settings and .env\n    result = await SearchClient(fetcher).search(SearchRequest(query=\"what is asyncio in python\"))\n    print(result.model_dump_json(indent=2))  # SearchResponse or SearchError\n    await fetcher.aclose()\n\nasyncio.run(main())\n```\n\nTo call it over MCP from your own code, the way an agent does, see\n[`examples/mcp_client.py`](https://github.com/RayenMlayeh/gsearch/blob/main/examples/mcp_client.py).\n\n## **Troubleshooting**\n\n| You see | What it means | What to do | \n|---|---|---|\n| `captcha` on most searches | Your proxy's IPs are flagged by Google. | Try another pool or country. Measure with `tools/benchmark.py` . | \n| `captcha` with hours to wait, no proxy | Your own IP was challenged and rests until midnight. | Wait, or add a proxy. | \n| `rate_limited` | Every IP is at its hourly or daily budget (own IP or static proxies). | Wait `retry_after_s` , or add proxies. | \n| `proxy: no usable exit in 4 draws` | Four IPs in a row were dead or in the wrong country. | Check the country part of the login, and the provider. | \n| `proxy: proxy data limit reached` | `GSEARCH_DATA_LIMIT_MB` is used up. | Raise it, or delete `proxy_bytes.json` in the state folder to count from zero. | \n| `parse_drift` | Google changed its page layout. | Update gsearch, or see [CONTRIBUTING.md](https://github.com/RayenMlayeh/gsearch/blob/main/CONTRIBUTING.md#when-google-changes-its-page) . | \n| `GSEARCH_ENV_FILE=… does not exist` | The settings path is wrong. | Use an absolute path. gsearch refuses to start rather than fall back to your own IP. | \n| `the Camoufox browser is not installed` | The browser has not been downloaded yet. | Run `gsearch setup` once. | \n| Browser windows open | Expected with a proxy: visible sessions are challenged far less. | On a server without a screen, install Xvfb and they run on a virtual display. | \n\nWhen the browser hits a block or an error, the page is saved with a screenshot in the state folder's\n`evidence/`, and the error's `evidence_ref` points to it.\n\n**Do I need a proxy?** No. Without one, gsearch uses your own IP with a warm session (~1.2 s per search),\ncapped at 20 searches an hour and 150 a day to keep your IP clean. For volume, use a proxy.\n\n**Which proxy should I buy?** A rotating residential proxy with sticky sessions (a session id in the\nusername). Quality matters more than price: run `tools/benchmark.py` on a small plan before buying more.\n\n**How much proxy data does a search use?** About 0.6 MB on a clean IP. Flagged IPs cost more, because each\none is a wasted browser launch. Set `GSEARCH_DATA_LIMIT_MB` so you can never overspend.\n\n**Does it solve CAPTCHAs?** No. A blocked IP is dropped and a fresh one is tried. gsearch ships no CAPTCHA\nsolver and never forges Google's tokens.\n\n**Can it run on a server?** Yes. On Linux without a screen, install Xvfb and the browser runs on a virtual\ndisplay.\n\n**Which agents work with it?** Any MCP client: Claude Code, Claude Desktop, Cursor and others. You can also\nuse the CLI or call it from Python.\n\n- **Google's Terms of Service restrict automated access to Google Search.** Running this tool against Google\nmay breach those terms, whatever this project's license says.\n- **Litigation risk is real and current.** In*Google v. SerpApi* (N.D. Cal., filed 2025-12-19), Google argues\nthat solving its SearchGuard JavaScript challenge by automated means is circumvention under DMCA §1201.\ngsearch runs that challenge in a real browser and does not forge or alter tokens, but the complaint also\ntargets automated browsers. See[docs/DESIGN.md](https://github.com/RayenMlayeh/gsearch/blob/main/docs/DESIGN.md) (D-004).\n- **No CAPTCHA solver is shipped, wired in or recommended.**`gsearch_core.captcha` defines an interface only;\nanything you plug into it, and its legality, is your responsibility.\n- **Get legal advice** for your jurisdiction and use case before running live searches. The parser\n(`gsearch_parse` ) only reads HTML you already have.\n- **Limits are on by default and cannot be bypassed from the CLI or MCP.** Don't use proxies to hammer\nGoogle.\n- **No personal-data harvesting.** Don't profile people, scrape people-search results or collect personal\ndata at scale. Keep caches short-lived and don't republish raw results.\n- **On a block, back off.** CAPTCHAs and rate limits come back as errors with`retry_after_s` . gsearch never\ntries to defeat the challenge.\n\nEvery title, snippet, URL and question in a result is **untrusted third-party text** from the open web, and\nmay contain instructions aimed at your agent. The tool descriptions say so, every response carries\n`meta.untrusted_content: true` and an MCP `untrusted_content` flag, and the text summary ends with a\nwarning. Treat results strictly as data.\n\nTests never contact Google: the parser is tested against saved pages, and the browser against a local fake\nGoogle. Pull requests are welcome: pick a\n[good first issue](https://github.com/RayenMlayeh/gsearch/labels/good%20first%20issue), see the\n[roadmap](https://github.com/RayenMlayeh/gsearch/issues?q=is%3Aissue+label%3Aroadmap), and start with\n[CONTRIBUTING.md](https://github.com/RayenMlayeh/gsearch/blob/main/CONTRIBUTING.md) and [docs/DESIGN.md](https://github.com/RayenMlayeh/gsearch/blob/main/docs/DESIGN.md).\n\n**If gsearch saves you time, a ⭐ helps other people find it.**\n\n[MIT](https://github.com/RayenMlayeh/gsearch/blob/main/LICENSE). Camoufox (MPL-2.0) is installed from PyPI as a dependency and is never vendored.", "url": "https://wpnews.pro/news/show-hn-gsearch-real-google-search-for-ai-agents-via-a-real-firefox-mcp", "canonical_source": "https://github.com/RayenMlayeh/gsearch", "published_at": "2026-10-19 10:17:12+00:00", "updated_at": "2026-10-19 10:46:05.888200+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-tools", "ai-search", "ai-crawlers"], "entities": ["gsearch", "Camoufox", "Google", "SerpApi", "Bright Data", "Brave", "Tavily", "Exa"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-gsearch-real-google-search-for-ai-agents-via-a-real-firefox-mcp", "markdown": "https://wpnews.pro/news/show-hn-gsearch-real-google-search-for-ai-agents-via-a-real-firefox-mcp.md", "text": "https://wpnews.pro/news/show-hn-gsearch-real-google-search-for-ai-agents-via-a-real-firefox-mcp.txt", "jsonld": "https://wpnews.pro/news/show-hn-gsearch-real-google-search-for-ai-agents-via-a-real-firefox-mcp.jsonld"}}