Google is shutting off the Custom Search JSON API on 2027-01-01. This keeps your code running by changing one base URL.
"The Custom Search JSON API is closed to new customers. Existing Custom Search JSON API customers have until January 1, 2027 to transition to an alternative solution." — Google, Custom Search JSON API overview
Google's suggested replacement is Vertex AI Search — a different API, a different response shape, and a paid product. Every line you wrote against customsearch/v1 has to be rewritten.
cse-bridge is the other option: a small self-hosted HTTP service that speaks Google's customsearch/v1 wire format on top of your own SearXNG instance. Your client library, your parsing code, your pagination loop and your cx values all stay exactly as they are.
- customsearch({version: 'v1'})
+ customsearch({version: 'v1', rootUrl: 'http://localhost:8080/'})
No API keys. No per-query fees. No account. It is your machine talking to your SearXNG.
git clone https://github.com/Booyaka101/cse-bridge.git
cd cse-bridge
docker compose up -d
That is the whole install. Now make the call you were already making:
(A prebuilt image is also on GHCR — ghcr.io/booyaka101/cse-bridge — which the compose file uses automatically once pulled; docker compose up -d builds locally either way.)
curl 'http://localhost:8080/customsearch/v1?key=k&cx=default&q=rust%20async%20runtime&num=3'
{
"kind": "customsearch#search",
"url": {
"type": "application/json",
"template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&cx={cx?}"
},
"queries": {
"request": [
{
"title": "Google Custom Search - rust async runtime",
"totalResults": "6",
"searchTerms": "rust async runtime",
"count": 3,
"startIndex": 1,
"inputEncoding": "utf8",
"outputEncoding": "utf8",
"safe": "off",
"cx": "default"
}
],
"nextPage": [ { "startIndex": 4, "count": 3, "...": "..." } ]
},
"searchInformation": {
"searchTime": 1.409902324,
"formattedSearchTime": "1.41",
"totalResults": "6",
"formattedTotalResults": "6"
},
"items": [
{
"kind": "customsearch#result",
"title": "The Async Ecosystem - Asynchronous Programming in Rust",
"htmlTitle": "The Async Ecosystem - Asynchronous Programming in Rust",
"link": "https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html",
"displayLink": "rust-lang.github.io",
"snippet": "The Async Ecosystem Rust currently provides only the bare essentials for writing async code. Importantly, executors, tasks, reactors, combinators, and low-level I/O futures and traits are not yet provided in the standard library. ...",
"htmlSnippet": "The Async Ecosystem Rust currently provides only the bare essentials for writing async code. ...",
"formattedUrl": "https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html",
"htmlFormattedUrl": "https://rust-lang.github.io/async-book/08_ecosystem/00_chapter.html"
}
]
}
That is a real, unedited response from the stack above. Real results, from real engines, in Google's shape.
Port 8080 already taken? Put CSE_BRIDGE_HOST_PORT=8081 in a .env next to docker-compose.yml.
These four are verified end to end against a live stack. Full recipes in docs/migrating-from-google-cse.md.
Node — @googleapis/customsearch
import { customsearch } from '@googleapis/customsearch';
const client = customsearch({ version: 'v1', rootUrl: 'http://localhost:8080/' });
const res = await client.cse.list({ q: 'test', cx: 'default', auth: 'k' });
console.log(res.data.items.length); // 10
Python — google-api-python-client
from google.api_core.client_options import ClientOptions
from googleapiclient.discovery import build
service = build("customsearch", "v1", developerKey="k",
client_options=ClientOptions(api_endpoint="http://localhost:8080"))
res = service.cse().list(q="test", cx="default").execute()
print(len(res["items"])) # 10
LangChain — GoogleSearchAPIWrapper
search = GoogleSearchAPIWrapper(google_api_key="k", google_cse_id="default")
search.search_engine = build("customsearch", "v1", developerKey="k",
client_options=ClientOptions(api_endpoint="http://localhost:8080"))
search.run("test")
curl / anything else — swap https://www.googleapis.com for http://localhost:8080.
You need a SearXNG instance with JSON output enabled (see below).
npm install -g cse-bridge
SEARXNG_URL=http://localhost:8888 cse-bridge
cse-bridge 1.1.0
listening http://localhost:8080
endpoint http://localhost:8080/customsearch/v1
backend http://localhost:8888
profiles default, docs, news, code (from profiles.yml)
auth disabled (any key accepted)
Requires Node 22 or newer. The package has zero runtime dependencies.
SearXNG does not serve JSON unless you turn it on. From the SearXNG search API docs: "Format needs to be activated in search:". In settings.yml:
search:
formats:
- html
- json
The bundled searxng/settings.yml already does this, so docker compose up just works. If you point at your own instance and forget, the bridge tells you exactly what to fix instead of failing mysteriously:
{"error":{"code":503,"message":"The service is currently unavailable.","errors":[{"message":"SearXNG at http://localhost:8888 returned HTML, not JSON. Enable it in settings.yml:\n search:\n formats:\n - html\n - json","domain":"global","reason":"backendError"}],"status":"UNAVAILABLE"}}
All configuration is environment variables. Every one has a working default.
| Variable | Default | What it does |
|---|---|---|
SEARXNG_URL |
http://localhost:8888 |
Your SearXNG instance. Must have json insearch.formats . |
PORT |
8080 |
Listen port. |
HOST |
0.0.0.0 |
Bind address. |
CSE_BRIDGE_KEYS |
(unset) | Comma-separated accepted key values.Unset means the key param is not checked at all. |
PROFILES_FILE |
profiles.yml |
cx → backend profile map. A missing file is fine. |
CSE_BRIDGE_TIMEOUT_MS |
20000 |
Per-request backend timeout. |
CSE_BRIDGE_CACHE_TTL_MS |
300000 |
How long a query's result set stays stable. 0 disables caching — seePagination . |
CSE_BRIDGE_CACHE_MAX |
256 |
Max distinct queries held in the cache. |
CSE_BRIDGE_PAGEMAP |
off |
on rebuildsitem.pagemap by fetching the result pages. SeeStructured data . A profile'spagemap: key overrides this. |
CSE_BRIDGE_PAGEMAP_MAX |
10 |
Result pages fetched per request; results sharing a page count once. 0 disables fetching. |
CSE_BRIDGE_PAGEMAP_TIMEOUT_MS |
3000 |
Deadline for one page. |
CSE_BRIDGE_PAGEMAP_BUDGET_MS |
9000 |
Deadline for the whole enrichment pass. Defaults to 3x the per-URL timeout. Items not reached in time come back bare. |
CSE_BRIDGE_PAGEMAP_TTL_MS |
3600000 |
How long a fetched page stays cached. |
CSE_BRIDGE_PAGEMAP_ALLOW_PRIVATE |
off |
on lets pagemap fetch loopback and private addresses. Only needed for an intranet index. |
Google's cx identified a Programmable Search Engine. Here it selects a block in profiles.yml, so a client you cannot edit keeps sending its existing cx and you decide server-side what it searches:
default:
description: General web search across the instance's enabled engines.
categories: [general]
docs:
categories: [general]
site: docs.rs # every query on this cx gets an implicit site: filter
news:
categories: [news]
pagemap: true # rebuild item.pagemap for this cx only
An unknown cx falls back to default — never an error, because a migrating client cannot change the cx it sends.
| Endpoint | Purpose |
|---|---|
GET /customsearch/v1 |
The Google-shaped search endpoint. |
GET /healthz |
Liveness plus backend reachability. |
GET /healthz?deep=1 |
Also runs a real query, proving format=json is enabled. |
/healthz deliberately does not search — a 30-second container healthcheck firing real queries would get your instance rate-limited by upstream engines.
key, cx, q, num, start, hl, lr, safe, siteSearch, siteSearchFilter, dateRestrict, fileType, exactTerms, excludeTerms, sort, searchType, imgSize, imgType, imgColorType, imgDominantColor.
A few behaviours are worth knowing:
numabove 10 clamps to 10 instead of erroring. Google rejects it; clamping is friendlier and keepsstart=1,11,21loops walking.startabove 91 returns Google's exact error envelope , includingstatus: "INVALID_ARGUMENT"anderrors[0].reason: "badRequest".dateRestrict(d7,m6, …) maps onto SearXNG's coarserday/week/month/yearbuckets, always roundingup — you get a superset of what you asked for, never a subset.siteSearch,fileType,exactTerms,excludeTermsbecome search operators in the backend query, since SearXNG has no dedicated parameters for them.sort=datereorders by thepublishedDateSearXNG attaches to news and paper results; undated results keep their relevance order and sit last.searchType=imageswitches to SearXNG'simagescategory — seeImage search below.imageis the only accepted value, exactly as on Google.imgSize,imgType,imgColorType,imgDominantColorare validated against Google's exact enums (an out-of-enum value gets Google's 400, because Google rejects it too) and then accepted for compatibility — SearXNG has no size/type/color parameters to map them onto, so they do not filter anything. Same posture assortexpressions beyonddate.
searchType=image works with the same one-line base-URL change as everything else. The link of each item is the image file itself (what Google promises — clients hotlink it into <img> tags), and the page it was found on is image.contextLink:
curl 'http://localhost:8080/customsearch/v1?key=k&cx=default&q=red%20panda&searchType=image&num=1'
{
"kind": "customsearch#search",
"url": {
"type": "application/json",
"template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&cx={cx?}"
},
"queries": {
"request": [
{
"title": "Google Custom Search - red panda",
"totalResults": "2",
"searchTerms": "red panda",
"count": 1,
"startIndex": 1,
"inputEncoding": "utf8",
"outputEncoding": "utf8",
"safe": "off",
"cx": "default",
"searchType": "image"
}
],
"nextPage": [ { "startIndex": 2, "count": 1, "...": "..." } ]
},
"searchInformation": {
"searchTime": 0.0000863,
"formattedSearchTime": "0.00",
"totalResults": "2",
"formattedTotalResults": "2"
},
"items": [
{
"kind": "customsearch#result",
"title": "50 Adorable Facts About The Red Pandas You Have To Know | Facts.net",
"htmlTitle": "50 Adorable Facts About The Red Pandas You Have To Know | Facts.net",
"link": "https://facts.net/wp-content/uploads/2020/08/AdobeStock_209028852.jpeg",
"displayLink": "facts.net",
"snippet": "50 Adorable Facts About The Red Pandas You Have To Know | Facts.net",
"htmlSnippet": "50 Adorable Facts About The Red Pandas You Have To Know | Facts.net",
"formattedUrl": "https://facts.net/wp-content/uploads/2020/08/AdobeStock_209028852.jpeg",
"htmlFormattedUrl": "https://facts.net/wp-content/uploads/2020/08/AdobeStock_209028852.jpeg",
"image": {
"contextLink": "https://facts.net/nature/animals/red-panda-facts",
"thumbnailLink": "https://ts1.mm.bing.net/th?id=OIP.I_aIcVvl98DbktQmP297ugHaE7&pid=15.1",
"width": 4000,
"height": 2666
}
}
]
}
That is a real response from the compose stack (only nextPage is elided; the 0.00 searchTime is the result-set cache answering — see Pagination). Worth knowing:
width/heightare parsed from theresolutionSearXNG reports;byteSizefrom its human-readablefilesize(1 KB = 1024);mime/fileFormatfromimg_format(jpg→image/jpeg). When an engine does not report one of these, the field isomitted — never guessed. The example above has nomimebecause that engine sent no format.- A result whose image URL is missing is dropped entirely rather than emitted with a page URL as
link— an item that claims to be an image but links to an HTML page breaks hotlinking clients silently. searchType=imagesupersedes the profile'scategoriesrather than merging with them: acxpinned tocategories: [news]cannot also be an image engine, and the client asking for images is the stronger signal. Everything else about the profile (itssite:restriction, language, engines) still applies.- Image and web result sets for the same query are cached separately , so alternating between them never leaks results across.
Google's item.pagemap carried the structured data it had scraped from each result page: metatags, schema.org objects, cse_image, cse_thumbnail. SearXNG returns none of that, so code that read item["pagemap"]["metatags"][0]["og:image"] breaks on migration.
The bridge can rebuild it. Off by default, because it makes the bridge fetch the result pages:
CSE_BRIDGE_PAGEMAP=on cse-bridge
Or per cx, which wins over the environment either way:
docs:
categories: [general]
pagemap: true
A real result from the compose stack with it on:
{
"link": "https://users.rust-lang.org/t/async-await-and-multi-thread-tokio-runtime/110107",
"title": "Async/await and multi-thread Tokio runtime - help - The Rust Programming Language Forum",
"pagemap": {
"metatags": [
{
"description": "Hey guys! I'm trying to grasp async/await usage with Tokio runtime. ...",
"generator": "Discourse 2026.9.0-latest",
"og:site_name": "The Rust Programming Language Forum",
"og:type": "website",
"twitter:card": "summary",
"og:image": "https://us1.discourse-cdn.com/flex019/uploads/rust_lang/original/2X/8/83e41956eccfd67ad6ff76f15a2c22e58db31d4f.svg",
"og:title": "Async/await and multi-thread Tokio runtime",
"article:published_time": "2024-04-17T19:15:04+00:00"
}
],
"qapage": [
{ "name": "Async/await and multi-thread Tokio runtime", "datepublished": "2024-04-17T19:15:04.626Z" }
],
"question": [
{ "answercount": "10", "upvotecount": "0", "name": "Async/await and multi-thread Tokio runtime", "...": "..." }
],
"person": [
{ "name": "frozenspider", "url": "https://users.rust-lang.org/u/frozenspider" },
{ "name": "parasyte", "url": "https://users.rust-lang.org/u/parasyte" }
],
"cse_image": [ { "src": "https://us1.discourse-cdn.com/flex019/uploads/rust_lang/original/2X/8/83e41956eccfd67ad6ff76f15a2c22e58db31d4f.svg" } ],
"cse_thumbnail": [ { "src": "https://us1.discourse-cdn.com/flex019/uploads/rust_lang/original/2X/8/83e41956eccfd67ad6ff76f15a2c22e58db31d4f.svg" } ]
}
}
(Elided for length: the metatags object had 20 keys, person had 11 entries, and there was an answer array.)
| Key | Comes from |
|---|---|
metatags |
<meta name=...> and<meta property=...> . The key isexactly as the page wrote it :og:title keeps its case, a barename is lowercased. That is what Google did. |
Lowercased schema.org type ( qapage ,product ,newsarticle , ...) |
<script type="application/ld+json"> , including every node in an@graph , and microdataitemtype /itemprop . |
cse_image ,cse_thumbnail |
[{ "src": ... }] from the first ofog:image ,og:image:secure_url ,twitter:image ,twitter:image:src that resolves to an http(s) URL. |
Anything in a literal <PageMap> block |
Pages that publish Google's own <PageMap> markup. It wins over a DataObject of the same name, since the site meant it literally. |
When nothing parses, the pagemap key is absent, not an empty object. Nothing is ever synthesized to fill a gap.
Per request the bridge fetches up to CSE_BRIDGE_PAGEMAP_MAX (10) result pages, 4 at a time, each with a 3s deadline, under a 9s deadline for the whole pass. Results that share a page share one fetch, and the fragment is ignored, so ten images from one gallery or three #section links cost a single request and all of them get the pagemap. A page that is slow, unreachable, not HTML, or redirects more than twice leaves that item bare and the search still returns 200. Bodies are capped at 512 KB and reading stops at </head>. Pages are cached by URL for an hour, so a repeated query costs nothing.
Because these are URLs a search backend chose, pagemap refuses loopback, RFC1918, link-local and CGNAT addresses and .internal/.local names. Set CSE_BRIDGE_PAGEMAP_ALLOW_PRIVATE=on if you are indexing an intranet. That check resolves the hostname you were sent, not the socket, so it is not proof against DNS rebinding. Do not point a public bridge at a private network.
- Data Google built rather than read. Its index held objects the page never published, and those cannot be recovered from the page.
cse_thumbnail`` widthandheight. Those described Google's own thumbnail crop. The bridge emitssrconly, rather than inventing dimensions.- Anything below
</head>. Microdata in the body is not scanned, so a page that puts its only schema.org markup in the footer parses to nothing. - Anything past the 512 KB read cap. A few big sites (YouTube, for one) put a megabyte of inline script above their
og:tags, so the read stops before reaching them and the item comes back with almost nothing. Google crawled the whole page; the bridge deliberately does not. - Google's 50-attribute and 1024-character caps are applied; a value longer than that is dropped, not truncated. Google also documented dropping
descriptionand a few other tags, but its live API returned them, so the bridge keeps them.
This is the part that quietly breaks naive implementations.
SearXNG merges several engines into each page, and those engines have varying latency — some drop out entirely. Ask the same query twice, seconds apart, and the results come back in a different order. Page straight through and start=11 re-serves links that start=1 already showed.
Measured against a live instance without a cache: start=1,11,21 returned 30 links, only 28 unique.
So cse-bridge resolves a query to a stable, de-duplicated result set and pages within it for CSE_BRIDGE_CACHE_TTL_MS — the way Google behaves. With the cache on, the same three requests return 30 links, 30 unique. A cached deep page also costs zero backend calls.
Set CSE_BRIDGE_CACHE_TTL_MS=0 to disable it, and expect overlapping pages.
SearXNG's JSON payload is exactly {query, results, answers, corrections, infoboxes, suggestions, unresponsive_engines} — verified in get_json_response. There is no result-count field. Anything a bridge reports as totalResults is synthesized.
cse-bridge reports a lower bound: everything it has actually walked past, plus one more page's worth only when a next page genuinely exists. It grows monotonically as you page (20 → 30 → 40) so while start < totalResults loops keep advancing, and it is never "0" while items exist.
It is not a real total, and it does not pretend to be. What it will never do is invent one — the obvious wrong answer is len(results) * 100, which sends clients paging into empty space.
Worth knowing before you migrate:
totalResultsis a lower bound, not an estimate of the web. If your code displays "about 1,240,000 results", it will now show a much smaller honest number.- 100 results maximum per query (
start≤ 91), same as Google. pagemapis reconstructed from the page, not from Google's index. SearXNG extracts none of it, so withCSE_BRIDGE_PAGEMAP=onthe bridge fetches the result pages itself and rebuildsmetatags, schema.org DataObjects from JSON-LD and microdata,cse_image/cse_thumbnail, and any literal<PageMap>block. It is off by default. What it cannot give you: DataObjects that only ever existed becauseGoogle built them, andcse_thumbnail`` width/height, which were the dimensions of Google's own crop. Those are omitted rather than guessed. SeeStructured data .image.thumbnailWidthandimage.thumbnailHeightare omitted on image results — SearXNG does not report thumbnail dimensions, and inventing them would be worse than leaving them out (the same posture astotalResults).width,height,byteSize,mimeandfileFormatappear whenever the engine reports the underlying data.- The image filters (
imgSize,imgType,imgColorType,imgDominantColor) validate but do not filter — SearXNG has no backend for them. - No
spellingunless SearXNG produces a correction ; it is thinner than Google's. - Result quality is your SearXNG's , not Google's. Which engines are enabled, and whether they are being rate-limited, decides what you get. Check
unresponsive_engineson your instance if results look thin. sortbeyonddateis accepted and validated but has no backend to act on.- Not a Google account substitute. Nothing here talks to Google.
npm install
npm test
Node's built-in test runner, no framework. The suite runs fully offline against a fake backend. To also run the live checks against a real instance:
docker compose up -d
CSE_BRIDGE_LIVE=1 SEARXNG_URL=http://localhost:8888 npm test
brcrusoe72/agent-search also wraps SearXNG, but exposes its own /search API for AI agents — a different shape, requiring code changes. cse-bridge exists for the opposite case: you have code you do not want to change.
rondeo-balos/tp-custom-search-api is an abandoned, unlicensed 270-line prototype of this same idea. It is worth reading as a list of what to get right: it computes totalResults from a number_of_results key that does not exist in SearXNG's JSON (falling back to a fabricated len(results) * 100), emits no nextPage/ previousPage, copies htmlTitle/ htmlSnippet through unescaped, and validates neither num nor start. All four are covered by tests here.
- Why
totalResultsand pagination are the hard parts — a longer write-up of the two problems above, with the measurements. - Discussion on r/selfhosted .
- Background: the Hacker News thread on the shutdown is worth reading for what people are migrating to. Note it is archived, so you cannot reply to it.
The most useful thing you can report: a client library that will not accept an endpoint override. The whole premise of this project is that yours will — Node, Python and LangChain are verified, and the others in the migration guide follow the same documented mechanism but are not covered by the acceptance checks. If you hit one that can't be repointed, please open an issue; that is the case that breaks the premise and I want to know about it.
Bug reports, missing CSE parameters, and SearXNG engine configurations that produce noticeably better results are all welcome.
MIT — see LICENSE.