How to use pyppeteer_stealth for web scraping

Pyppeteer_stealth patches automation signals in Pyppeteer to evade anti-bot detection systems. The plugin hides the WebDriver flag, replaces the HeadlessChrome user agent string, and populates empty navigator properties that expose headless browsers. Developers can apply the stealth patches by importing the library and calling it on a page instance before navigating to any target website.

Pyppeteer on its own is straightforward to detect. It leaks the WebDriver flag, uses the HeadlessChrome user agent string, returns an empty navigator.vendor, and fails other fingerprinting checks that anti-bot systems run on every request. Any serious bot detection will catch a bare Pyppeteer session before your scraping logic even runs. pyppeteer stealth https://pypi.org/project/pyppeteer-stealth/ is a plugin that patches those leaks. It applies a set of evasion techniques directly to the Pyppeteer page instance to make headless Chrome look more like a real browser session. In this tutorial you will learn what pyppeteer stealth patches, how to use it, how to fix the most common setup issue you will run into, and where its limits are against modern anti-bot systems. What is pyppeteer stealth? pyppeteer stealth is the Python implementation of the Puppeteer Stealth plugin, built as an add-on for Pyppeteer, which is Python's unofficial port of Puppeteer. Where base Pyppeteer exposes clear automation signals, pyppeteer stealth patches the most obvious ones. Here is what it patches: User Agent. Changes the HeadlessChrome flag in the user agent to a standard Chrome string so the browser does not announce it is running headless. WebDriver. Sets navigator.webdriver to false . This is the most commonly checked automation flag and one of the first things anti-bot systems look for. Chrome Runtime. Modifies the Chrome runtime object to make headless Chrome look like it is running in standard GUI mode. Hardware Concurrency. Overrides the CPU core count to match a realistic machine rather than the default value headless environments often return. Plugins. Populates navigator.plugins with real browser plugin data. An empty plugin list is a strong automation signal. Vendor. Overrides navigator.vendor with a real vendor string. Headless Chrome returns an empty string here by default. WebGL. Spoofs GPU properties to return realistic hardware values rather than the generic software renderer headless environments use. Media Codecs. Replaces bot-like codec values with realistic MIME types that match a real browser installation. How to use pyppeteer stealth Install the libraries pip3 install pyppeteer stealth pyppeteer Step 1: Run base Pyppeteer as a baseline Before adding the stealth plugin, run a fingerprinting test to see what base Pyppeteer exposes. This gives you a clear before-and-after comparison. python pip3 install pyppeteer import asyncio from pyppeteer import launch async def scraper : browser = await launch headless=True page = await browser.newPage await page.goto "https://bot.sannysoft.com/" await page.screenshot {"path": "baseline.png"} await browser.close asyncio.run scraper The screenshot shows multiple red flags on the fingerprinting test. WebDriver exposed, HeadlessChrome in the user agent, empty plugins list. Base Pyppeteer fails a significant portion of the checks that anti-bot systems run. Step 2: Add pyppeteer stealth Import the stealth function and call it on the page instance after creating it but before navigating anywhere. The plugin patches the page context before any page code runs: python pip3 install pyppeteer-stealth pyppeteer import asyncio from pyppeteer import launch from pyppeteer stealth import stealth async def scraper : browser = await launch headless=True page = await browser.newPage apply stealth patches to the page before navigating await stealth page await page.goto "https://bot.sannysoft.com/" await page.screenshot {"path": "stealth.png"} await browser.close asyncio.run scraper With the plugin applied, the fingerprinting test passes. The WebDriver flag is hidden, the user agent looks normal, plugins are populated, and the other patched properties return realistic values. Two lines of change and the surface-level fingerprint looks clean. Step 3: Scrape real data Here is a complete example that uses pyppeteer stealth to extract product data from an e-commerce page: python import asyncio from pyppeteer import launch from pyppeteer stealth import stealth async def scrape products url: str - list: browser = await launch headless=True page = await browser.newPage await stealth page await page.goto url, {"waitUntil": "networkidle0"} products = await page.querySelectorAll ".product" results = for product in products: name = await product.querySelectorEval ".product-name", "el = el.innerText" price = await product.querySelectorEval ".price", "el = el.innerText" results.append {"name": name, "price": price} await browser.close return results data = asyncio.run scrape products "https://www.scrapingcourse.com/ecommerce/" print data Output {"name": "Abominable Hoodie", "price": "$69.00"}, {"name": "Adrienne Trek Jacket", "price": "$57.00"}, ... {"name": "Artemis Running Short", "price": "$45.00"}, That works on an open page. Now test it against something with actual protection. The common setup issue: Chromium not found When you run Pyppeteer for the first time, you may hit this error: OSError: Chromium downloadable not found at https://storage.googleapis.com/chromium-browser-snapshots/Win x64/1181205/chrome-win.zip This happens because Pyppeteer targets a specific Chromium revision that is no longer available at that URL. The fix is to override the revision number before Pyppeteer tries to download it. Find the chromium downloader.py file in your Pyppeteer installation. If you are using a virtual environment, it is at venv/Lib/site-packages/pyppeteer/chromium downloader.py . Add this line before the REVISION variable: python import os os.environ "PYPPETEER CHROMIUM REVISION" = "1181217" REVISION = os.environ.get "PYPPETEER CHROMIUM REVISION", chromium revision You can also set the environment variable before running your script without touching the library files: Linux / macOS export PYPPETEER CHROMIUM REVISION=1181217 Windows set PYPPETEER CHROMIUM REVISION=1181217 The limitations of pyppeteer stealth pyppeteer stealth passes fingerprinting tests. That is a meaningful improvement over base Pyppeteer. The limits become clear when you point it at a page with real anti-bot protection. 1. It fails against modern anti-bot systems python import asyncio from pyppeteer import launch from pyppeteer stealth import stealth async def scraper : browser = await launch headless=True page = await browser.newPage await stealth page await page.goto "https://www.scrapingcourse.com/antibot-challenge" await page.screenshot {"path": "blocked.png"} await browser.close asyncio.run scraper The screenshot shows the block page. pyppeteer stealth patches surface fingerprints but it does not address the deeper JavaScript challenges, timing analysis, and behavioral checks that modern systems like Cloudflare run in the background. 2. It has not been updated since 2021 This is the most significant limitation. Anti-bot systems update frequently. A stealth library that has not changed in years is working against detection techniques from years ago. Many anti-bot vendors have specifically added detection for pyppeteer stealth's patches since 2021 because its bypass mechanisms are public and well-documented in the open-source code. 3. Navigation patterns are still predictable Even with fingerprints patched, the way Pyppeteer navigates pages, the timing between requests, the absence of natural reading pauses, and other behavioral signals can still look automated to systems that analyze traffic patterns rather than just browser properties. 4. No proxy infrastructure pyppeteer stealth has no built-in proxy rotation or geo-targeting. IP bans, rate limits, and geo-restrictions are entirely your problem to handle separately. Going beyond pyppeteer stealth When the plugin is not enough, you have two paths. You can add proxy rotation, switch to a more recently maintained stealth library, and keep tuning the setup manually. Or you can move the anti-bot handling out of your code entirely. Spidra handles the full stack at the API level https://spidra.io/products/spidra-api . Every request runs through a real browser with residential proxy rotation across 50 countries, CAPTCHA solving, and fingerprinting maintained against current detection techniques. It also replaces the HTML parsing step entirely: instead of returning raw HTML you still need to parse, it extracts exactly what you describe and returns clean structured JSON. Here is the same anti-bot challenge page that blocked pyppeteer stealth, using Spidra's Python SDK https://docs.spidra.io/sdks/python : pip install spidra python from spidra import SpidraClient, ScrapeParams, ScrapeUrl import os spidra = SpidraClient api key=os.environ "SPIDRA API KEY" job = spidra.scrape.run sync ScrapeParams urls= ScrapeUrl url="https://www.scrapingcourse.com/antibot-challenge/" , prompt="Extract the main heading", use proxy=True, proxy country="us", print job.result.content { "heading": "You bypassed the Antibot challenge :D" } No browser to launch. No plugin to apply. No revision number to fix. The same request works on open pages and protected ones without any changes. Here is the same e-commerce scraping task without any selectors or parsing: job = spidra.scrape.run sync ScrapeParams urls= ScrapeUrl url="https://www.scrapingcourse.com/ecommerce/" , prompt="Extract all product names and prices", output="json", print job.result.content {"name": "Abominable Hoodie", "price": "$69.00"}, {"name": "Adrienne Trek Jacket", "price": "$57.00"}, {"name": "Artemis Running Short", "price": "$45.00"} If you want a guaranteed output shape for downstream pipelines, add a schema: job = spidra.scrape.run sync ScrapeParams urls= ScrapeUrl url="https://www.scrapingcourse.com/ecommerce/" , prompt="Extract all products", output="json", schema={ "type": "array", "items": { "type": "object", "required": "name", "price" , "properties": { "name": {"type": "string"}, "price": {"type": "string"}, "image": {"type": "string", "null" }, } } } Required fields always appear in every record, as null if the page does not have that value. pyppeteer stealth vs. Spidra | pyppeteer stealth | Spidra | | |---|---|---| | Fingerprint patching | Yes, 8 properties patched | Handled at infrastructure level | | Last updated | 2021 | Actively maintained | | Cloudflare bypass | Fails on JS challenges | Built in, automatic | | DataDome / PerimeterX | Not reliable | Built in, automatic | | Proxy rotation | Not included | Built in, 50 countries | | Structured output | Raw HTML, you parse it | AI extraction, optional schema | | Chromium setup issues | Yes, revision fix required | Not applicable | | Maintenance as anti-bots evolve | Manual | Handled by Spidra | | Language | Python | Python, Node.js, Go, PHP, Ruby, and 5 more | | Best for | Light scraping, basic fingerprint patching | Protected sites, production pipelines | Conclusion pyppeteer stealth does what it says. It patches the most visible automation signals in Pyppeteer and a fingerprinting test looks much cleaner with it applied. For light scraping on sites without serious bot protection, it is a simple and low-effort improvement over base Pyppeteer. The limitation is age. It has not been updated since 2021 and modern anti-bot systems have had years to study and specifically detect its patches. Against Cloudflare, DataDome, and similar systems it is not reliable, and the behavioral patterns Pyppeteer produces beyond the browser fingerprint are still detectable. If you need to scrape sites that are actively trying to stop you, maintaining a patched Pyppeteer setup is ongoing work. Spidra handles the full anti-detection stack automatically so you can focus on the data rather than the browser setup. Get started free at spidra.io https://spidra.io/ . No credit card required.