Show HN: Pay-per-Crawl – make AI crawlers pay per page in USDC, for 10 stacks

ForgeMesh released Pay-per-Crawl, a pay-per-crawl gate that charges AI crawlers, SEO crawlers and scripts in USDC on the Base network, across ten stacks including WordPress, Drupal, Laravel, Next.js, Node, Cloudflare, nginx, Caddy, Traefik and Python. Each edition settled two real $0.01 payments on Base mainnet through the PayAI facilitator before release, for ten settlements in one day, and the packages are published on npm, PyPI, Packagist and drupal.org. Humans and reverse-DNS-verified search engines read free, while paying crawlers receive an HTTP 402 with an x402 v2 header, a v1 body and an MPP challenge, and the page is served only after the settlement transaction hash returns; site owners keep 100% of payments directly in their own wallets.

One gate for the whole web: Pay-per-Crawl now ships for ten stacks Last night Pay-per-Crawl was a WordPress plugin that started as a comment on TikTok, with two real payments behind it. This afternoon the same gate runs on WordPress, Drupal, Laravel, Next.js, Node, Cloudflare, nginx, Caddy, Traefik and Python, and it is published on npm, PyPI, Packagist and drupal.org. Every edition settled two real payments on Base mainnet before we released it. We are proud of this one, so here is exactly what shipped and what it took. The same gate everywhere The behaviour is identical on every stack. Humans and reverse-DNS-verified search engines read free. AI crawlers, SEO crawlers and scripts get an HTTP 402 carrying an x402 v2 header, a v1 body and an MPP challenge. They pay USDC on Base, and the page is served only after the settlement transaction hash comes back. The owner keeps 100%, paid directly to their wallet. There is no custody, no ForgeMesh account and no API key. PayAI is the default facilitator and any facilitator URL works. Every edition carries the same 85-agent crawler catalog and serves the same discovery files: /.well-known/x402.json, llms.txt and agents.txt. stacks, one behaviour real $0.01 settlements on Base mainnet in one day crawlers in the shared catalog Pick your stack WordPress: free GPL plugin from forgemesh.io/wordpress https://forgemesh.io/wordpress , a GitHub release. It is submitted to wordpress.org and under review, so it is not listed there yet. Next.js middleware: npm i @forgemeshlabs/pay-per-crawl-next Express, Fastify, Hono, Koa: npm i @forgemeshlabs/pay-per-crawl-node Cloudflare Worker in front of any origin: npm i @forgemeshlabs/pay-per-crawl-worker . The shared engine is @forgemeshlabs/pay-per-crawl-core , and there is an nginx njs snippet too. Source: github.com/forgemeshlabs/pay-per-crawl-js https://github.com/forgemeshlabs/pay-per-crawl-js . Python FastAPI/Starlette, Flask/WSGI, Django : pip install forgemesh-pay-per-crawl , repo pay-per-crawl-py. Go: go get github.com/forgemeshlabs/pay-per-crawl-go . It includes a Caddy module http.handlers.pay per crawl, built with xcaddy and a Traefik plugin. PHP: composer require forgemesh/pay-per-crawl-laravel or composer require forgemesh/pay-per-crawl-drupal , both on the shared forgemesh/pay-per-crawl-core . Drupal has a project page at drupal.org/project/pay per crawl https://www.drupal.org/project/pay per crawl . How we know it works Every edition passed its own test suite and a 37 to 44 check HTTP matrix. Then each code base settled two real $0.01 payments on Base mainnet through PayAI before release. That is ten real settlements in one day. A few of the transactions: WordPress v1 https://basescan.org/tx/0xe764d2affaca73fa225aecd383aff5969b5d9ff4e8035933b7561983f23ba7f5 , Python v2 https://basescan.org/tx/0x15d95ad90c9530e4816016f65681ddd4a79730373634c991c57afa2b88a8b44b , Go v1 https://basescan.org/tx/0x948aa9522f9feed31a1a8bc77e7c1de88699b1e629759f5677bd6a2e3117c4e8 , Laravel v2 https://basescan.org/tx/0xff1dfa134999dc4ad19bd8a9f9ae05e286e5c9e9714bc13c4979a3c073d7b63d . All public on Basescan. We would rather show you a transaction hash than say the word tested. Two security bugs, found and fixed before publishing Porting the gate turned up two real problems, and we fixed both in every edition before anything went to a registry. A forwarded-IP spoof. A crawler could prepend a Googlebot address to X-Forwarded-For and walk through as a search engine. All editions now use the right-most hop, the one our own proxy appended. Unanchored path exemptions. Free paths like /robots.txt were matched loosely enough that /robots.txt.html or /.well-known/../post could slip past the gate. The exemptions are now exact files, a sitemap regex, and normalised dot segments. We are telling you because that is how we ship: attack our own gate first, then publish. What porting taught us Drupal: the page cache answers before normal event subscribers ever run, so the gate had to become an HTTP middleware that sits above it. Laravel: Sanctum drops middleware groups unless they are pushed after boot. Traefik: its plugin interpreter cannot load assembly-backed keccak, so the Go edition carries a pure-Go keccak. Edge editions pass static assets free by default, since nobody is scraping your favicon for training data. Pro works with every edition Pro is $4.99 a month for up to 5 sites at proxy.forgemesh.io/wp https://proxy.forgemesh.io/wp , and it connects to any of the editions above: a hosted dashboard, a Monday digest, paywall monitoring with alerts, a crawler-intel feed of 104 agents, and a listing in the paid-sites directory https://proxy.forgemesh.io/wp/sites with a badge. The gate itself is complete without it, and payments never pass through us. How this differs from Cloudflare Pay Per Crawl Cloudflare Pay Per Crawl bills a few enrolled AI companies by card, with Cloudflare as merchant of record. Ours lets any x402 or MPP agent pay the site owner's wallet directly. If you also use Cloudflare, set its AI crawler control to Allow, otherwise crawlers are blocked before they get the chance to pay. Thank you One TikTok comment, one night, one afternoon. Thank you to everyone who asked for it. Tell us which stack we missed. Bill the bots on any stack Pick your edition above or start at forgemesh.io/wordpress. Pro $4.99/mo, up to 5 sites is at proxy.forgemesh.io/wp. Related reading: Your WordPress site gets 175,000 bot visits a day. Now they pay. https://forgemesh.io/blog/wordpress-pay-per-crawl-plugin . - ai-crawlers - x402 - wordpress - opensource All posts → https://forgemesh.io/blog