Google Trends ties its data tokens to your IP and it broke my scraper in a way I didn't expect

I just shipped a Google Trends scraper as an Apify Actor (https://apify.com/swiftscrape/google-trends-fast-scraper), and the most interesting part wasn't building it — it was a blocking bug that only showed up in production, on the platform, that I couldn't reproduce locally no matter what I tried.

If you've ever tried to scrape Google Trends and had it work on your laptop but mysteriously fail behind proxies, this one's for you. The root cause is non-obvious and I couldn't find it documented anywhere, so here's the whole thing.

Google Trends has no official API. There are scrapers out there, but the most-used one on the market is slow (runs take minutes), it charges you even when it returns nothing, and its multi-term comparison breaks when called from AI agents. Its reviews are full of "ran for 20 minutes, returned nothing, cost me money."

That's a wedge. Same data, but: fast, and you only pay for terms that actually return data — failed, blocked or empty terms are free. If a scraper is unreliable, that's the developer's problem, not the user's bill.

Google Trends' UI is backed by internal JSON endpoints. You don't need a headless browser — you can talk to them directly:

GET /trends/api/explore

with your terms → returns GET /trends/api/widgetdata/multiline?token=…

→ interest over time.GET /trends/api/widgetdata/relatedsearches?token=…

→ related queries.No Chrome, no Puppeteer. Direct HTTP is why it runs in ~1.5s locally instead of minutes.

Locally (running from my home IP, no proxy): flawless. Timeline, related queries, everything.

On Apify (behind a datacenter proxy): explore

succeeded, timeline

succeeded... but related queries silently failed for every term. Then when I switched to residential proxies to "fix" it, it got worse — now even the timeline failed with blocked

.

That combination made no sense to me at first:

If residential IPs are "better," why did they break more?

Here's the thing nobody tells you: the widget tokens Google hands you in step 1 are tied to the IP address that made the explore call. When you then call the

multiline

/ relatedsearches

endpoints with those tokens, Google checks that the request comes from the blocked

.Now the table makes sense:

explore

and timeline

sometimes landed on the same IP by luck — but the parallel related calls hit fresh IPs and got rejected.explore

came from a different IP. Even the timeline broke.The "better" proxy broke things precisely because it rotated IPs more aggressively.

The fix is to force every request in a single run through the same IP by pinning a proxy session:

const proxy = await Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'] });
const sessionId = `gt${Date.now()}`;

// Every request in this run reuses the same IP:
const proxyUrl = await proxy.newUrl(sessionId);

One sticky session → explore

, timeline

, and related

all come from the same IP → tokens stay valid. Result after the fix, on the platform:

chatgpt

: 53 weekly data points, 16 top related queries, 3 risinggemini

: 53 data points, 25 top, 18 risingThe lesson generalizes beyond Google Trends: any two-step "get a token, then use the token" API can be IP-bound. If step 2 fails behind a proxy but step 1 succeeds, suspect IP-pinned tokens before you suspect rate limiting.

Because I only charge for terms that return validated data, I had to make the billing code honest: a term with no search volume comes back from Google as all-zeros with a hasData: false

flag, not an empty array. If you don't check hasData

, you'll "successfully" return a series of zeros and charge for a term that had no data. That check is the difference between "no result = no charge" being a slogan and being true.

hasData

flags or your "free on failure" promise is a lie.The scraper is live here if you want to try it (pay-per-result, failed terms are free): ** Fast Google Trends Scraper on Apify**.

Happy to answer questions about the internal endpoints or the token behavior in the comments.