Webclaw, an open-source Rust web data extraction toolkit positioned as a drop-in Firecrawl alternative, has drawn over 2.3k GitHub stars and is available both self-hosted and as a managed cloud servic…
Web scraping remains necessary for AI data pipelines because LLM web search tools cannot replace it, according to an article on Substack's The Web Scraping Club. The article explains that large langua…
A new analysis of DataDome's anti-bot JavaScript Tag 5.9.4 on leboncoin.fr found the sensor is 124 KB of minified JavaScript with readable method names and a plain base64 string table, a simpler obfus…
The September 2026 web scraping event season begins with Oxycon 2026, a free online conference starting in 8 days with 10 sessions and 18 speakers, and FuelTurin, a networking aperitivo organized by D…
AntsData, a data provider offering pre-compiled datasets, SERP APIs, Web Search APIs, and a forthcoming Web Unlocker API, has been reviewed in a hands-on tutorial. The company's Google SERP API allows…
Google has begun replacing destination URLs in search results with redirect links for automated queries, requiring 500 to 1,000 requests to resolve a five-page ranking, according to a report by PPC La…
An autonomous agent from Manus AI collected a 412-entry fingerprint inventory from Kasada's anti-bot system on nike.com, revealing the browser signals the system reads to build user fingerprints. The …
Obscura, a from-scratch Rust browser engine that embeds V8 and claims to run in about 30 MB of RAM, was tested against Akamai Bot Manager on Radisson's booking pages, where it failed to clear the anti…
The Web Scraping Club newsletter marked its fourth anniversary with more than 7,000 subscribers, founder Pierluigi Vinciguerra wrote, noting the publication has run with no marketing efforts and 0 ads…
Microsoft's open-source MarkItDown Python library, with over 165k GitHub stars and 12 million monthly downloads, converts web pages, PDFs, Word documents, Excel spreadsheets, and other formats into Ma…
Bright Data launched Scraper Studio, a cloud-hosted scraping environment with an AI agent mode that generates production-ready scrapers from plain-English descriptions, plus a full IDE mode and an AI-…
Decodo, a web scraping API provider, argues in a new article that Retrieval Augmented Generation (RAG) pipelines are only as effective as their data acquisition layer, and that web scraping—not built-…