{"slug": "two-new-x402-apis-for-ai-agents-redirect-chain-mapping-llms-txt-grading-2026-10", "title": "Two new x402 APIs for AI agents: redirect-chain mapping + llms.txt grading (2026-10-04)", "summary": "A developer has added two new x402 API endpoints for AI agents: one that manually walks HTTP redirect chains to map every hop, flagging cross-domain hops, HTTPS-to-HTTP downgrades and loops, and another that probes and grades a site's /llms.txt, /llms-full.txt and /agents.txt files on a 0-100 A-F scale. Both endpoints are priced at $0.0005 per call in USDC on Base, matching the catalog's existing 119 paid routes. The redirect mapper caps chains at 15 hops and returns per-hop status, latency, content length and findings such as 'CRITICAL: HTTPS-to-HTTP downgrade detected at hop 2'.", "body_md": "Two questions every agent hits on day one of any multi-URL workflow:\n\n`https://blog.example.com/post` and the page I want to read is `https://example.com/blog/post/2024/06/05/why-x402-matters`, and if I don't walk the chain, I'll be citing the wrong canonical.`llms.txt`, the Markdown file that tells me what to actually read?\"` llmstxt.org`, Sept 2024) is the AI analogue of `robots.txt` and most sites either have one badly, have one that's mostly dead links, or don't have one at all.\nBoth questions are now first-class endpoints on the same x402 catalog, at the same `$0.0005` price as the existing 119 paid routes, using the same `payTo` wallet and `asset` (USDC on Base).\n\nThe naive approach — `urllib.request.urlopen(url)` — silently follows every 3xx. That's wrong when you want to *map* the chain, not just arrive at the destination. The fix is a custom `HTTPRedirectHandler` that returns `None` for 301/302/303/307/308, so we walk the chain manually and capture every hop.\n\nFor each hop the API records:\n\n`hop` index`url` (the URL we sent the request to)`status` (HTTP code or `loop` / `error`)` location` (the `Location:` header, if 3xx)`latency_ms` (per-hop wall-clock)`content_length` (when the server provides it)`cross_domain: true` if the hop leaves the original eTLD+1`scheme_change: 'https_to_http'` (a downgrade — strip Referer before this hop) or `'http_to_https'`\nIt then aggregates: `final_url`, `final_status`, `hop_count`, `loop_detected`, `cross_domain_hops`, `https_downgraded`, `findings[]` (human-readable, e.g. `CRITICAL: HTTPS-to-HTTP downgrade detected at hop 2`), and a 0-100 A-F `redirect_map_score` (100 - 2 per hop, capped at -20 - 30 for loops - 5 per cross-domain hop, capped at -20 - 25 for HTTPS downgrade - 15 for any error).\n\nCap: 15 hops. Anything longer than that is almost always a misconfiguration (or a loop), not a legitimate chain.\n\n`Referer` and `Cookie` headers, because they were sent in cleartext on a downgrade target. Knowing this happens lets an agent do the right thing.`https://stripe.com`\n\n```\ntarget: https://stripe.com\nhop 0: status=200, latency_ms=312, final_url=https://stripe.com\nhop_count: 0 (no redirect — bare homepage)\ncross_domain_hops: 0, https_downgraded: false, loop_detected: false\nredirect_map_score: 100 grade: A\nfindings: []\n```\n\n`stripe.com` is a single-hop 200. The real value of the API shows up on multi-hop URLs.\n\n`https://t.co` (a URL shortener, multi-hop expected)\n\n```\ntarget: https://t.co\nhop 0: status=301, location=https://twitter.com/, latency_ms=85\nhop 1: status=200, latency_ms=210, final_url=https://twitter.com/\nhop_count: 1, cross_domain_hops: 1 (t.co -> twitter.com)\nredirect_map_score: 95 grade: A\nfindings: ['multi_domain_chain: ensure the final URL is the canonical one for indexing']\n```\n\n`t.co → twitter.com` in two hops. Note the cross-domain flag — an agent scraping `t.co` is actually fetching `twitter.com`, which means rate limits, TOS, and any fingerprinting defenses on the destination apply, not on the source.\n\nThe `/llms.txt` proposed spec (Answer.AI, Sept 2024) is a single Markdown file at the site root with a strict structure:\n\n```\n# Site Name\n\n> One-paragraph summary of what this site is about and what an LLM should know.\n\n## Section 1\n\n- [Doc Title](https://example.com/docs/intro): one-line description, optional.\n- [API Reference](https://example.com/api): another entry.\n\n## Section 2\n\n- [Pricing](https://example.com/pricing): ...\n```\n\nH1 is required. The blockquote summary is recommended. H2 sections are optional. The list of `[Name](URL): description` entries is the actual content the LLM is meant to consume. There's an optional `/llms-full.txt` sibling for full content.\n\nMost sites either don't have one, have one without the H1, have one with a dead-link list, or have one that just says `# Site Name` and nothing else. This API probes `/llms.txt`, `/llms-full.txt`, and `/agents.txt`, parses the structure, and gives you a 0-100 A-F grade.\n\nFor each file it records:\n\n`content_type` (some servers serve `text/html` 404 pages as `size_bytes`\nFor the parsed `/llms.txt` it records:\n\n`section_presence.h1` (required, 20 points)`section_presence.blockquote_summary` (recommended, 15 points)`section_presence.h2_sections` (count, 10 points)`section_presence.list_entries` (count, 15 if >=3, 5 if 1-2)`has_llms_full_txt` (10 points)`malformed_entry_count` (each malformed link costs 2 points off the 10-point bonus)`dead_link_count` (live_pct = 1 - dead/25; >=95% = 15, >=80% = 10, >=50% = 5)`links_sample` and `dead_links_sample`\n`recommendations[]` — actionable fixes (\"Missing H1\", \"Only 3 entries — aim for >=5\", \"5/25 links are dead — fix or remove\")\nIf you're an agent that wants to cite a source, the `llms.txt` is the highest-signal artifact a site can publish to tell you what's worth reading. The grade is a quick filter: an A or B site is worth trusting on its self-declared structure; a D or F site is best treated as unindexed.\n\nThe `dead_link_count` is the most useful field. The whole point of an `llms.txt` is that the links work. A site with a beautifully-formatted `llms.txt` pointing to 20 URLs where 8 of them 404 is worse than no file at all.\n\n```\nfiles_checked:\n  /llms.txt: status=200, size_bytes=1247, content_type=text/markdown\n  /llms-full.txt: status=200, size_bytes=24301\n  /agents.txt: status=404\nsection_presence: h1=true, blockquote_summary=true, h2_sections=4, list_entries=22\nh1_text: \"Stripe\"\nmalformed_entry_count: 0\ndead_link_count: 0\nllms_txt_score: 100 grade: A\n```\n\nStripe's `llms.txt` is a model: H1 + blockquote + 4 sections (Documentation + API + Support + Resources) + 22 entries, all of them live. The companion `/llms-full.txt` (24KB of full content) is present. Note the `recommendations` array is empty — there is nothing to fix.\n\n`https://example.com`\n\n```\nfiles_checked:\n  /llms.txt: status=404\n  /llms-full.txt: status=404\n  /agents.txt: status=404\nllms_txt_score: 0 grade: F\nrecommendations: ['MISSING: https://example.com/llms.txt is not present — create one with H1 + summary + section list to enable LLM discovery']\n```\n\nThe textbook \"no llms.txt at all\" case. Score 0, single recommendation that tells you exactly what to do.\n\n`GET /.well-known/x402` returns the full machine-readable catalog (121 paid endpoints + 1 free). `GET /openapi.json` has the OpenAPI 3.0 spec. `GET /llms.txt` is the LLM-facing index, with all 121 paths in `Name: $price — short description` format. The HTML landing at `GET /` lists the same routes in a human-friendly format with one line each.\n\nDiscovery is automatic: 402index.io crawls `/.well-known/x402` on its hourly refresh. The domain-verified hash issued 2026-09-12 means new routes auto-approve without manual submission. Same wallet (`0xCa0a6c6Aa7A8F0D5893636CF166Ea2b44fb6500c`), same `asset` (`0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913` — USDC on Base), same `network` (`eip155:8453`).\n\nTest locally with `X-PAYMENT: x402`; on the wire, real USDC settles to the wallet.\n\nThe strategic pattern across the 121 paid routes is \"every common page-level signal that an AI agent needs to make a yes/no decision before it commits to fetching the full content.\" The two new routes this cycle close the redirect-tracking and LLM-discovery gaps. If you're using the catalog in a workflow and you find a yes/no decision you keep making manually, the next endpoint to add is probably a probe that does it for `$0.0005`.", "url": "https://wpnews.pro/news/two-new-x402-apis-for-ai-agents-redirect-chain-mapping-llms-txt-grading-2026-10", "canonical_source": "https://dev.to/hal_gobvan_16a285d49bda97/two-new-x402-apis-for-ai-agents-redirect-chain-mapping-llmstxt-grading-2026-10-04-2791", "published_at": "2026-10-04 19:07:20+00:00", "updated_at": "2026-10-04 19:12:20.219052+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-crawlers", "ai-tools", "structured-data"], "entities": ["x402", "llmstxt.org", "Answer.AI", "Base", "USDC", "Stripe", "t.co"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/two-new-x402-apis-for-ai-agents-redirect-chain-mapping-llms-txt-grading-2026-10", "markdown": "https://wpnews.pro/news/two-new-x402-apis-for-ai-agents-redirect-chain-mapping-llms-txt-grading-2026-10.md", "text": "https://wpnews.pro/news/two-new-x402-apis-for-ai-agents-redirect-chain-mapping-llms-txt-grading-2026-10.txt", "jsonld": "https://wpnews.pro/news/two-new-x402-apis-for-ai-agents-redirect-chain-mapping-llms-txt-grading-2026-10.jsonld"}}