Two months ago I gave an agent a simple job: log into a vendor portal, download last month's invoice PDF, rename it, drop it in a folder. It worked on my laptop. It failed on the server, silently, in a way that took me a full day to understand — the page loaded, the DOM was there, the login form was there, and the credentials were rejected with a generic error. No CAPTCHA. No block page. Just "something went wrong."
The site had decided my agent was a bot several layers before any of that.
I build a browser for this problem, so I've now read a lot of block pages. The thing I keep having to explain is that "bot detection" is not one check that you either pass or fail. It's a stack, and the layers fire in order, and the layer that kills you is almost never the one you're looking at.
Before a single line of your JavaScript runs, the server already has:
crypto/tls
. That's not a heuristic, it's a contradiction.Sec-Fetch-Site
, `Sec-Fetch-Mode`
, `Sec-CH-UA`
and friends are sent by real Chrome in a specific pattern that depends on how the navigation started. The University of Bamberg's Only after all of that does the page get to run the fingerprinting script everybody writes blog posts about.
This ordering is why "I patched navigator.webdriver
and it still doesn't work" is the single most common thing I hear. Of course it doesn't. You fixed layer 4 of a stack that rejected you at layer 1.
Your desktop Chrome passes not because it's trusted, but because everything about it agrees with everything else about it. That's the whole game.
platformVersion
, architecture
, model
).navigator.platform
.navigator.platform
matches the GPU strings that WebGL reports.Intl.DateTimeFormat().resolvedOptions()
reports.An agent stack breaks this by accident, constantly. You set a UA string but not the client hints. You route through a proxy in Frankfurt while your container's timezone is UTC and your navigator.languages
is en-US
. You run headless in a container with software rendering, so WebGL reports SwiftShader or llvmpipe while the UA claims a Windows desktop with an RTX card. Each of those is individually harmless-looking and collectively a signed confession.
The detector doesn't need to know what a "correct" machine looks like. It only needs to find two of your claims that can't both be true.
1. Half-overrides. Someone overrides navigator.platform
in an injected script but leaves the CDP-level UA metadata untouched, or vice versa. Now the main thread says one thing and the browser's own protocol-level state says another. Same for the connection info: navigator.connection
exposes rtt
, downlink
and effectiveType
, and real Chrome derives all three from one measurement. If you spoof effectiveType: '4g'
and leave rtt: 0
, you've published a value combination Chrome never emits. (When we derive that object, we measure the actual round-trip through the proxy and run it through Chromium's own effective-connection-type thresholds, so the three fields stay consistent by construction. Deriving them separately is how you get a contradiction.)
2. Environment leaking through the disguise. This is the one that bit me hardest. Suppress the font enumeration surface all you like — if the host OS is a Chinese Windows install, some CSS system font keywords (menu
, small-caption
, status-bar
) can still resolve through a code path that isn't the one you patched, and the measured metrics come back as a CJK UI font on a persona that claims to be a US English machine. The fingerprint surface was clean. The rendering wasn't. Detectors measure rendering.
3. Automation artifacts nobody thinks of as fingerprint surface. A proxy-auth browser extension is the classic. It works, it's easy, and it puts an enumerable entry in the extension list of a browser that is supposed to look like a stock consumer install. Same category: an extra tab your launcher opened, a window size no human has, a --disable-*
flag that changes an observable default.
In rough order of return on effort:
requests
" plateaus fast.page.evaluate
or an init script runs Function.prototype.toString
, property descriptors, prototype identity across realms. If you can push the change into the browser itself, do that instead. If you can't, at least make the JS layer and the protocol layer say the same thing.None of this makes you undetectable, and I'd distrust anyone who tells you otherwise. The Bamberg data also found that enterprise-grade bot management (the stuff that does behavioral ML per customer) sits on well under 1% of the web overall — but around 15% of the sites people actually want to automate. The hard targets are hard on purpose, and against the top tier the deciding factors turn out to be exit IP quality, account age, and behavior, not which browser build you used.
What the browser layer gets you is the removal of cheap reasons to block you. That's not nothing. Most of the failures I debug are cheap reasons — a header, a timezone, a WebGL string. Fix those, and you find out whether you had a real problem in the first place.
If your agent is failing today, don't start with the fingerprint. Start with a request log and find out how far you got before it went wrong. Nine times out of ten the answer is: not as far as you assumed.