{"slug": "the-dark-arts-of-web-automation-openai-threatened-to-ban-my-account", "title": "The Dark Arts of Web Automation. OpenAI threatened to ban my account", "summary": "OpenAI threatened to ban the account of a developer preparing a talk on web automation, according to the developer. The talk, titled 'The Dark Arts of Web Automation,' argues that AI agents can appear human by driving a browser through the Chrome DevTools Protocol (CDP) rather than using an MCP server, citing a study by Arize AI showing CLI and MCP both achieve about 83% accuracy but CLI is faster and cheaper. The developer built a tool called chrome-agent to facilitate CDP-based browser control.", "body_md": "# The Dark Arts of Web Automation\n\nOminous, right?\n\nSounds like I'm about to teach you something you'll need a lawyer for.\n\n…We'll come back to the lawyer in a bit.\n\nHere's a little bit of background…\n\nSo…\n\nI was getting ready for this talk -- and OpenAI threatened to ban my account ...\n\nJust for the work that I was doing in preparing it…\n\nI checked my email the other day, and I found this.\n\nWhat a **shocker** this was!\n\nSo -- what does one have to do to earn the banhammer?\n\nFor cyber abuse?\n\nWith a web browser?\n\nThis. This is what I was doing.\n\nEvery one of these is being solved by an AI agent using a web browser\n\n... and in 15 minutes or so, you'll understand exactly how.\n\nI want my agents to be able to use the web the way that I do -- book the things, send the emails, fill in the forms -- so I don't have to.\n\nBut, the moment something that isn't a person starts clicking, many webpages fight back.\n\nSo -- this is a talk about winning that fight. And it starts with one slightly unconventional idea.\n\nHere it is. The premise of the whole talk.\n\n'A CDP browser is just like a meatbag with a mouse.'\n\n...\n\nAt least as far as Google, Cloudflare, and the rest can tell\n\nNo joke -- if you have your agent drive a browser through the Chrome DevTools Protocol, your agent's clicks and keystrokes travel the exact same path inside Chrome that yours do.\n\nThat's the big idea. The rest of the talk is how you pull it off. And that comes down to three things.\n\nA CLI -- not an MCP.\n\nThe Chrome DevTools Protocol -- which is where a tool I built, called **chrome-agent** comes in.\n\nAnd a loop, run on a ladder.\n\nSo, let's start with the one that picks a fight.\n\nFirst thing: give your agent a command line interface (shell-based tools), not an MCP server. And before some of you knuckle-up on this debate, there are specific reasons for this.\n\nIt's worth noting that capability is a wash -- CLI and MCP both got to the right answer about 83% of the time in a recent study by Arize AI.\n\nHOWEVER, a CLI beats an MCP in reuse, speed and cost.\n\nReuse first. A CLI sequence can be programmed: write it once, run it a thousand times, without a model in the loop. Whereas MCP hits the model on every turn.\n\nThe CLI is FASTER for a similar reason -- because there's no model in the middle of every step. In that same study -- MCP took 71 round-trips and 8 minutes for the same task that was completed with just 7 calls and under a minute using the CLI. Hold onto this one -- speed comes back at the very end.\n\nAnd token cost -- Anthropic reported that executing code instead of running MCP can be 75x cheaper in terms of token usage.\n\nSo -- what are we actually running on the command line?\n\nWe drive the browser with the Chrome DevTools Protocol -- the second thing your agent needs to appear human.\n\nYou already know of this protocol, even if you've never heard its name. That developer panel that opens in Chrome when you hit F12 drives the browser using it. Your agents can speak it, too, using the chrome-agent tool. Chrome Agent also makes it easy for your agent to write code to replay CDP interactions.\n\nThe surface area of CDP is enormous. 57 domains, hundreds of methods and events, covering everything the browser can do. I've grouped these domains into 8 buckets to make it easier to hold in your head.\n\nBut the good news is that you don't need all 57 domains. To interact with a page like a human, you usually need just a small subset of these.\n\nIt's easiest to think about the subset of CDP domains you'll frequently use in terms of the 'digital senses' they provide your agent.\n\nYou SEE the page -- read its structure from the DOM, its semantics from the accessibility tree, or just take a screenshot when you want the pixels.\n\nYou HEAR what the page reports about itself -- the network data, its console, and its logs.\n\nAnd you OPERATE the page -- with clicks, keystrokes, and navigation.\n\nThe third and final thing you need for your agent to appear human when using the browser is what I call \"a loop on a ladder.\"\n\nSo here's the loop:\n\nSense, act, verify -- and you repeat it until the page gives in.\n\nSense -- perceive the page through one or more channels -- the DOM, the accessibility tree, a screenshot.\n\nThen Act -- do ONE thing -- click something, type something, select something.\n\nAnd then Verify -- sense again, but through a different channel than the action. e.g. If you clicked something, don't ask the click whether it worked -- look at the screen, or the network instead.\n\nSense where you are. Make one move. Confirm it landed. Then iterate again.\n\nAnd when the loop won't close -- so when you sense, act, verify, and the page still won't do what you want -- that's the page fighting back and telling you to climb the ladder.\n\nThe Meatbag Ladder, that is! This is a ladder of techniques that are increasingly more human as you climb it.\n\nIt has 3 rungs and you climb only as high as the page forces you. Said differently, you climb to the lowest rung that works.\n\nOn Rung 1 you don't act human at all. If you can just call the API exposed within the page, or fire a synthetic JavaScript click, do that -- it's easy, it's free, it's instant, and it's the right default.\n\nClimb to Rung 2 when faking it stops working: e.g. when you need a real click using CDP's Input domain. This is agent input that the page cannot tell apart from your own.\n\nClimb to Rung 3 when you need human input plus human behavior -- a real mouse path, a little dwell and some jitter, or vision to actually see and interpret things.\n\nSo you start cheap, and climb one rung at a time -- only when the page makes you. Climb the ladder, run the loop on each rung, then write down the path that worked.\n\nAnd that's how you arrive at fully automated AI agent-driven browsing.\n\nFirst you explore -- you run the loop by hand, climbing rungs, until the thing actually works.\n\nAnd then you automate -- you write that solution down so you never have to discover it again.\n\nAnd you write it down as CODE, as an AGENT SKILL or, quite often, as both.\n\nSo let me show you how this all comes together -- starting with a word from my attorney.\n\nI told you we'd come back to the lawyer.\n\nEverything you're about to see is real -- all of this comes from real agent browser use, in the wild.\n\nHOWEVER, On the advice of counsel, everything you're about to see runs only on infrastructure and accounts I own.\n\nNOW ...\n\nWho's ready to see some agents getting BUSY with the browser?\n\nAlright, so here's some simple, everyday use. Let's send a batch of personalized emails -- each one different -- from your Outlook web client.\n\nWe've got some lovely pseudocode here to make it easy to get the general sense of how this all works in code.\n\nOutlook's compose box has nothing to defeat, so this is Rung 1 of the Meatbag Ladder -- you don't act human at all. A synthetic click opens compose, you fill it in programmatically, and a synthetic click sends it.\n\nAnd then you just … let it rip!\n\nThe reason this is going so smoothly is that it's executing a program. You capture the sequence once then loop it -- 20 emails, or 200, from one command. Solve it once, run it forever.\n\nThe agent is riffing on the content to personalize it, whilst running a program for all the interactions.\n\nYou may ask \"Why drive the web UI at all -- why not use the API?\" Because in a corporate environment the official API needs an app registration and an admin's sign-off -- which, as an employee, you often can't get. HOWEVER, The logged-in web session needs nothing but the login you already have. The web UI becomes a universal API in this pattern.\n\nThat's Rung 1 ... But what happens when the page pushes back?\n\nSo -- let's say you're shopping at your favorite online megastore. Let's just call them … Demazon. They're a crafty bunch over there at Demazon, and their pages don't quite care for your bots. Run that exact same fake click that just worked on Outlook, point it at an 'Add to Cart' button, and … nothing. No error, no complaint. The page just ignores it.\n\nBecause the page is checking: did this click come from a real person, or from JavaScript? Chrome stamps every event with that answer -- whether an input is trusted, or untrusted. A click you fire from code is stamped untrusted, and, in this case, the page quietly drops that input.\n\nBUT ... No worries -- you just climb the Meatbag Ladder! On Rung 2 you run the click through Chrome's Input domain which uses the same input path your actual mouse uses. Now it's stamped trusted, the page can't tell the difference, and the item drops into the cart.\n\nNow, we've got quite the inside view of Demazon.com. Watch the bottom panel and let's imagine that's what the server's logs might look like. Every fake click is rejected, but the real ones are accepted.\n\nAs a heads-up -- when you see the mouse pointer in these screen recordings, that has been added programmatically to show what mouse inputs the agent is simulating using chrome-agent / CDP. The agent isn't actually using my mouse.\n\nRung 2 is where the real meatbag inputs begin BUT … that, alone, is not enough to let your agent replace you in the browser.\n\nSo we climb to the top of the Meatbag Ladder. Rung 3 -- this is the narrow frontier where pages are actively hunting for bots. There are a variety of techniques we deploy here, though, so let's talk through a few of them.\n\nI'm sure this guy looks familiar. This is Cloudflare Turnstile and it appears deceptively simple, but it's the hardest target we've hit yet. You can't easily get a hold of that checkbox via typical web automation programming.\n\nIt's buried behind three nested boundaries -- a closed shadow root, then a cross-origin iframe, then another shadow root inside that. To every cheap trick, that checkbox simply does not exist. There is no element to grab.\n\nSOOOO ... you stop trying to grab it! You ask the browser where the iframe sits on screen, do a bit of maths to calculate the position of the checkbox, and fire a trusted click at that point on the glass. And Chrome does the rest!\n\nA real click lands right on the checkbox.\n\nNo human in the loop -- all agent. That's one level, cleared, and mostly the trick here was just figuring out how to interact with it. The next levels make you prove you can actually see.\n\nSo here's MTCaptcha -- remember these ones? These types are still around ...\n\nThe agent has to actually read this guy. So it simulates what you'd do -- it takes a screenshot of the challenge and looks at it, and its own vision capability picks the characters out of the noise.\n\nThen it types the answer back using real, trusted keystrokes, routed into the challenge's cross-origin iframe, one character at a time. The same inputs your keyboard sends.\n\nAnd the server agrees: text correct, token issued!\n\nNow there's one more level before the final boss -- and this one is won or lost based on how you move like a meatbag.\n\nSo this one's by Lemin -- it's a little jigsaw puzzle where you have to spot where the piece belongs, and drag it into the gap -- and there's an entire class of CAPTCHAs like this.\n\nThis one's tricksy in different ways. There's no shadow root and no cross-origin iframe to pierce -- the piece is sitting right there in the page. The hard part is the drag itself.\n\nWhen you drop the puzzle piece, these types of CAPTCHAs sample the mouse movement into a trail of points the whole way across, including jitter and changing speed and all. So it's not just solving the puzzle, but solving it with Moves like Jagger!\n\nSOOOO … the agent drags the way a hand drags -- eased in, gently curved, a small overshoot and then settles. Just like a meatbag with a mouse! It uses Vision to see the gap and human-like motion to cross it.\n\nSo that's Turnstile. MTCaptcha. and Lemin -- three gates built to keep agents out, and we've just beat each of them. Which leaves only one boss standing ...\n\nAnd here he is -- The final boss of The Internet. reCAPTCHA v2.\n\nThe little checkbox, the grid of blurry traffic lights and crosswalks -- everyone in this room has squinted at one.\n\nBUT we've got the whole kit for how to beat this guy, too!\n\nThe digital senses … the loop to deploy them in … the Meatbag Ladder … everything you need to take down this cheeky bastard!\n\nSo let's go!\n\nHere's the whole machine, and it comes in two halves.\n\nOn one side we have The Solver. Pure code -- no agent, no model. It does everything programmatically:\n\n- the trusted click on the checkbox,\n- piercing into the challenge iframe,\n- and then every round it screenshots the tile grid;\n- if a round expires, it just re-arms and goes again.\n\nThis bit is deterministic, fast, free.\n\nThere's one step in that loop the code can't do and that's look at a grid of fuzzy photos and know which ones have a bus in them.\n\nThat's vision and thinking, and it needs eyes and a brain.\n\nSo that's the only job the agent gets. And we call that The Operator. The solver taps the agent on the shoulder -- and the agent:\n\n- takes one look at the grid\n- picks the tiles with the thing in them\n- hands the answer back to the Solver\n- then stands by until the next lap.\n\nThis is really the entire talk, running as one system. Code does the deterministic driving; the agent does only the part that takes eyes and a brain.\n\nWho wants to see it go?\n\nAllllllright!\n\nSo while that's playing, by the way, some of these are really hard! The agent spotted bicycles in photos that I didn't!\n\nBut look at that -- it's solved and server-verified and IT'S FAST!\n\nAnd fast is the whole point! This big bad boss is on the clock! Every round expires, and a full solve can be many rounds back to back. An agent that round-trips a model on every click and every look BURNS THAT CLOCK and loses -- the challenge resets before it ever finishes.\n\nThe only thing I've found that defeats this is what you're watching: deterministic code running at machine speed, with one quick AI look per round.\n\nRemember when I told you to hold onto speed? This is why.\n\nAnd this is why it had to be a CLI, on CDP -- never a model sitting in the middle of every single interaction step.\n\nIn case you're wondering, solving this wasn't a fluke …\n\nHere it is, again and again ...\n\nA reliable, repeatable solution, every time.\n\nBut HERE's what I actually want you to walk out of here with …\n\nThe big takeaway is the methodology enabling this. The CAPTCHAs were just a tricksy test of it.\n\nThis came down to careful, disciplined engineering -- it's the engineering that enabled an agent to do something it simply could not do on its own.\n\nAnd the method is simple:\n\n-\nGive your agent a CLI, so you can program it.\n\n-\nDrive the whole browser through CDP -- using its digital senses.\n\n-\nRun it as a loop on the Meatbag Ladder -- climbing only as high as the page forces you.\n\n-\nThen Explore until you solve it and write the solution down.\n\nThat's what makes this durable and truly useful: Figure it out once. Do it forever.\n\nWhich brings us all the way back … to the crew at OpenAI and the impending banhammer.\n\nAfter a quiet word, they kindly rescinded the threat.\n\nSo, I've still got access to Codex, which is nice!\n\nSo, you, too, can use Chrome Agent. It's installable in the Python ecosystem.\n\nIt's also open-source and available on GitHub: [github.com/captivus/chrome-agent](https://github.com/captivus/chrome-agent)\n\nThat's the tool I wrote to do all of this with, and I use it all day, every day!\n\nOR ... build your own! We live in the age of unbounded, personalized software.\n\nPlease do follow me on X and let's chat -- I want to hear how you're automating the web with AI.\n\nHappy hacking!", "url": "https://wpnews.pro/news/the-dark-arts-of-web-automation-openai-threatened-to-ban-my-account", "canonical_source": "https://gallon.me/the-dark-arts-of-web-automation.html", "published_at": "2026-07-24 18:39:45+00:00", "updated_at": "2026-07-24 18:52:35.436527+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "artificial-intelligence"], "entities": ["OpenAI", "Chrome DevTools Protocol", "chrome-agent", "Arize AI", "Anthropic"], "alternates": {"html": "https://wpnews.pro/news/the-dark-arts-of-web-automation-openai-threatened-to-ban-my-account", "markdown": "https://wpnews.pro/news/the-dark-arts-of-web-automation-openai-threatened-to-ban-my-account.md", "text": "https://wpnews.pro/news/the-dark-arts-of-web-automation-openai-threatened-to-ban-my-account.txt", "jsonld": "https://wpnews.pro/news/the-dark-arts-of-web-automation-openai-threatened-to-ban-my-account.jsonld"}}