Is Your API Ready for AI Agents? A 6-Point Checklist: llms.txt, agent-card.json, MCP Server Card, ai-catalog.json, robots.txt, x402

A six-point self-audit published by an unnamed technical blog gives developers pass/fail curl checks for the files AI agents from Claude, ChatGPT and Gemini read to discover and call services, covering llms.txt, the A2A agent-card.json at /.well-known/agent-card.json, an MCP endpoint and server card, the ARD ai-catalog.json, robots.txt and x402. The checklist warns that a SPA returning index.html with a 200 status for llms.txt, an agent card pointing at a 404 endpoint, or a 401 without a usable WWW-Authenticate header will cause agents to move to a competitor whose files work. The MCP check passes on either a 200 JSON-RPC result with serverInfo or a 401 carrying a WWW-Authenticate header pointing to OAuth protected resource metadata, and the server card is described as optional and experimental under SEP-2127.

Is Your API Ready for AI Agents? A 6-Point Checklist: llms.txt, agent-card.json, MCP Server Card, ai-catalog.json, robots.txt, x402 Audit your domain for AI agents in ten minutes: six pass/fail curl checks for llms.txt, A2A agent-card.json, MCP, ARD ai-catalog.json, robots.txt and x402. AI agents from Claude, ChatGPT, and Gemini now find and call services without a human pasting in a URL. They do that by reading a handful of files and endpoints on your domain. If those are missing, or present but broken, the agent moves on to a competitor whose files work. This is a self-audit. Six checks, each with a curl command and a clear pass condition. For background on what each file is and why it exists, the longer guide is How to Make Your Website Discoverable to AI Agents https://contextiq.trango-compute.com/blog/make-website-discoverable-ai-agents-llms-txt-agent-cards ; this post is the pass/fail version. Replace example.com with your domain and run each command from a machine outside your network. Check 1: llms.txt curl -si https://example.com/llms.txt | head -20 Pass: 200 , content-type: text/plain or text/markdown , and a body that starts with a Your Product heading followed by a one-line description and linked sections. Common fail: your SPA returns index.html with a 200 . The status looks fine and the body is HTML. Check the first line of the body, not just the status. Quick fix: a Markdown file at your site root with your product name, a sentence naming what you do, and links to docs and pricing. Name specific entities frameworks, providers, protocols because that's what answer engines index. Check 2: A2A Agent Card if you expose an agent curl -si https://example.com/.well-known/agent-card.json Pass: 200 , application/json , valid JSON with a name , a service URL, and a non-empty skills array whose descriptions say what the agent does in concrete terms. Common fails: the old card path the spec's location has moved once, so check which your client expects , JSON syntax errors, and a card whose endpoint returns 404 . A card that points at nothing is worse than no card. Skip this check if you don't operate an agent. Field-by-field details are in A2A Agent Cards Explained https://contextiq.trango-compute.com/blog/a2a-agent-card-well-known-agent-card-json . Check 3: MCP Endpoint and Server Card if you expose tools Test that the endpoint answers an initialize request: curl -si -X POST https://example.com/mcp \ -H 'content-type: application/json' \ -H 'accept: application/json, text/event-stream' \ -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}' Then check for the server card: curl -si https://example.com/.well-known/mcp/server-card Pass: either a 200 JSON-RPC result with serverInfo , or a 401 carrying a WWW-Authenticate header that points to OAuth protected resource metadata. A correct auth challenge counts as a pass because it tells clients exactly how to get in. The server card is optional and experimental SEP-2127 https://contextiq.trango-compute.com/blog/mcp-server-discovery-well-known-server-card-sep-2127 but cheap to publish. Common fail: a 401 with no usable header. If your server is auth-protected and clients can't connect, see MCP Server Returns 401? https://contextiq.trango-compute.com/blog/mcp-server-401-oauth-discovery-debug-protected-resource-metadata . Check 4: ARD Catalog curl -si https://example.com/.well-known/ai-catalog.json Pass: 200 , JSON, and entries that list each agentic capability on your domain your A2A agent, your MCP server with a type and descriptive tags. The catalog is the index registries crawl, so every capability from checks 2 and 3 should appear here. Background: the ARD specification https://contextiq.trango-compute.com/blog/agentic-resource-discovery-ard-specification-ai-catalog . Common fail: the catalog lists an endpoint that no longer exists, or omits the MCP server you ship. Check 5: robots.txt — Agentmap Line and Crawler Access curl -s https://example.com/robots.txt Pass, part one: it contains an Agentmap: directive pointing to your catalog: Agentmap: https://example.com/.well-known/ai-catalog.json Pass, part two: it doesn't block the crawlers you want to reach you. Look for Disallow: / under user agents such as GPTBot , ClaudeBot , Google-Extended , PerplexityBot , or CCBot . Blocking some of these is a legitimate policy choice for training data, but a blanket User-agent: / Disallow: / copied from a staging config will hide your discovery files from everything. Make the choice deliberately. Check 6: x402 Payment Challenge if you charge per request If your API is paid per call, agents need to know the price before they send money. The x402 protocol does this with an HTTP 402 Payment Required response: curl -si https://example.com/api/paid-endpoint Pass: 402 with a machine-readable payment requirements body naming the network, asset, amount, and payTo address, and nothing ambiguous an agent has to guess at. Skip this check if your API is free or uses conventional API keys. To validate the challenge itself, use x402 Inspector https://contextiq.trango-compute.com/x402-inspector ; the pre-payment checks an agent should run are in the x402 payment checklist https://contextiq.trango-compute.com/blog/x402-payment-checklist-what-ai-agents-verify-before-signing . Scoring Yourself | Passes of the checks that apply | Where you stand | |---|---| | All | Discoverable and callable. Re-check quarterly. | | Checks 1 and 5 only | Crawlable by answer engines, invisible to agent registries. Add the protocol files that match what you ship. | | Protocol files present, check 5 fails | Files exist but registries won't find them. Add the Agentmap line, fix crawler rules. | | Any 200 returning HTML | A catch-all route is masking failures. Fix this first. | Check only the items that match what you actually offer. Publishing a card or catalog entry for a capability you don't run creates a trap for agents. Do the Whole Sweep in One Scan The six curl checks take about ten minutes by hand and need repeating whenever you change your CDN or routing rules. Agent Readiness Detector https://contextiq.trango-compute.com/agent-readiness-detector runs the protocol checks in one scan: agent card, ARD catalog, robots.txt Agentmap line, MCP initialize handshake, and MCP discovery metadata. It reports each protocol as confirmed, indicated, or not detected, with the evidence lines behind each verdict. llms.txt and the x402 challenge are the two checks you run separately with the commands above. Run it from the outside, because that's where agents sit. Follow Trango Compute on LinkedIn We post updates on new tools, context engineering patterns, and LLM cost research. Follow on LinkedIn https://www.linkedin.com/company/trango-compute