{"slug": "how-to-scrape-react-next-js-sites-with-claude-desktop-without-puppeteer", "title": "How to Scrape React & Next.js Sites with Claude Desktop (Without Puppeteer)", "summary": "A developer has released x402-cluster, an MCP server that lets Claude Desktop scrape client-rendered React and Next.js sites without running Puppeteer or Docker locally. The server routes requests through a serverless headless browser on Cloudflare's infrastructure, waits for the React DOM to render, and returns JSON matching a supplied schema. It is configured by adding an MCP server entry with a bearer API key to claude_desktop_config.json.", "body_md": "If you’ve tried to build an AI agent that scrapes modern websites, you’ve probably hit this wall: standard `fetch()` tools return blank HTML.\n\nWhy? Because sites built with React, Vue, or Next.js render on the client. To actually get the data, you need a headless browser.\n\nHistorically, this meant writing a Puppeteer or Playwright script, containerizing it in Docker, and hosting it on a VPS. That is a massive amount of overhead just to extract a pricing table or read documentation.\n\nHere is how to solve this instantly in Claude Desktop using the **x402-cluster** MCP server—zero Docker or Puppeteer config required.\n\nInstead of running a heavy local browser, you can route the request through a serverless Edge browser using the x402 API.\n\nYou send it a URL and a JSON schema, it spins up a headless browser on Cloudflare's infrastructure, waits for the React DOM to render, and returns perfectly typed JSON.\n\nOpen your `claude_desktop_config.json` file and add this block:\n\n```\njson\n{\n  \"mcpServers\": {\n    \"x402-cluster\": {\n      \"url\": \"[https://x402-cluster.ian-944.workers.dev/v1/mcp.json](https://x402-cluster.ian-944.workers.dev/v1/mcp.json)\",\n      \"headers\": {\n        \"Authorization\": \"Bearer sk_live_YOUR_API_KEY\"\n      }\n    }\n  }\n}\n```\n\n", "url": "https://wpnews.pro/news/how-to-scrape-react-next-js-sites-with-claude-desktop-without-puppeteer", "canonical_source": "https://dev.to/ianshelp/how-to-scrape-react-nextjs-sites-with-claude-desktop-without-puppeteer-1dcc", "published_at": "2026-11-22 18:33:52+00:00", "updated_at": "2026-11-22 18:41:44.683244+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-tools", "developer-tools"], "entities": ["Claude Desktop", "x402-cluster", "Cloudflare", "Puppeteer", "React", "Next.js", "Playwright"], "also_reported_by": [], "takeaway": "Adding the x402-cluster MCP server to claude_desktop_config.json lets Claude Desktop fetch client-rendered React/Next.js pages as schema-typed JSON via a serverless Cloudflare browser, avoiding a local Puppeteer or Docker setup.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/how-to-scrape-react-next-js-sites-with-claude-desktop-without-puppeteer", "markdown": "https://wpnews.pro/news/how-to-scrape-react-next-js-sites-with-claude-desktop-without-puppeteer.md", "text": "https://wpnews.pro/news/how-to-scrape-react-next-js-sites-with-claude-desktop-without-puppeteer.txt", "jsonld": "https://wpnews.pro/news/how-to-scrape-react-next-js-sites-with-claude-desktop-without-puppeteer.jsonld"}}