{"slug": "the-web-is-becoming-agentic-what-happens-to-api-discovery", "title": "The Web Is Becoming Agentic. What Happens to API Discovery?", "summary": "As AI agents replace humans as API consumers, discovery shifts from search engines to machine-readable layers like llms.txt, MCP, and A2A protocols, which are not competitors to search but a new discovery stack built for software. The article argues that agents need structured, parseable data to complete the discovery-to-execution pipeline autonomously, and that existing tools are layers, not competitors.", "body_md": "# The Web Is Becoming Agentic. What Happens to API Discovery?\n\nSearch engines solved discovery for humans. Agentic systems need a machine-readable discovery layer for software. Here's what that means — and why existing tools are layers, not competitors.\n\nAs AI agents replace humans as API consumers, discovery shifts from search engines to machine-readable layers like llms.txt, MCP, and A2A protocols. These are not competitors to search — they are a new discovery stack built for software, not humans.\n\n## Search engines solved discovery for humans. Agentic systems need a machine-readable discovery layer for software.\n\nFor 20 years, the web was built around one discovery model.\n\nA human searches Google. Finds documentation. Reads through API descriptions. Compares options in a marketplace. Makes a decision. Integrates.\n\nEvery step of this flow was designed for human judgment — the ability to read prose, infer context, compare unstructured descriptions, and fill in gaps with intuition.\n\nNow a new consumer is emerging: the AI agent.\n\nAn agent receiving the instruction \"find an API for international payments and execute a transaction\" must do everything a human developer would do — but autonomously, using only machine-readable signals.\n\nAnd the infrastructure that made APIs discoverable for humans? It wasn't built for this.\n\n## The old discovery model: human as primary consumer\n\nIn the old model, API discovery looked like this:\n\n```\nHuman\n  ↓\nGoogle / docs / marketplace\n  ↓\nAPI\n```\n\nA developer would:\n\n- Google \"best API for payments\"\n- Find a marketplace (RapidAPI, AWS Marketplace)\n- Read documentation\n- Compare options side by side\n- Make a decision based on features, pricing, and reputation\n- Integrate\n\nEach step required human judgment. Understanding context. Comparing unstructured descriptions. Making decisions with incomplete information. Filling in gaps by reading between the lines.\n\nThis model worked because the consumer was always a human who could **guess**.\n\n## The new discovery model: agent as primary consumer\n\nNow imagine the consumer is an AI agent.\n\n```\nHuman\n  ↓\nAI Agent\n  ↓\n???\n  ↓\nAPI\n```\n\nA user says: \"Find an API for international payments and execute a transaction.\"\n\nThe agent must independently:\n\n**Discover** which providers exist**Understand** what each API can do**Compare** options against the user's requirements**Check pricing**— is this transaction cost-effective?** Handle authentication**— how do I get access?** Select**a provider** Call**the API** Handle errors**— what if something goes wrong?\n\nAt each step, the agent needs **machine-readable information**. Not prose documentation. Not a landing page. Not a marketing description.\n\nStructured, parseable, actionable data.\n\nIf the information exists only in human-readable documentation — scattered across prose, hidden behind JavaScript-rendered pages, described only in natural language — the agent cannot complete the path autonomously.\n\nThe question is not whether agents can read documentation.\n\nThe question is: **where is the machine-readable information that lets an agent complete the full discovery-to-execution pipeline without a human?**\n\n## What an agent needs to know about an API\n\nIt's not enough for an agent to know:\n\napi.example.com exists\n\nAn agent must understand:\n\n**What the API can do**— capabilities, operations, available actions** Which operations are available**— endpoints, methods, parameters** How to authenticate**— auth flow, token endpoint, scopes, API keys** How much it costs**— machine-readable pricing, per-call cost, tier limits** What the limits are**— rate limits, quotas, usage caps** How reliable the documentation is**— is the OpenAPI spec in sync with the actual API?** Whether to trust the description**— self-declared vs verified metadata** Whether the endpoint actually matches the claimed behavior**— evidence, not claims\n\nThis is where **Agent Readiness as discovery infrastructure** enters the picture.\n\n## Existing mechanisms: not competitors, but layers\n\nIt would be easy to say: \"our standard solves everything.\"\n\nThat would be wrong.\n\nThe truth is that several technologies already address pieces of the problem. But none of them address all of it.\n\n| Mechanism | What it solves | What it doesn't solve |\n|---|---|---|\nSearch | Discovery for humans | Machine-readable context, execution |\nOpenAPI | Interface description | Discovery, trust, verification, pricing |\nllms.txt | Context for LLMs | Execution, auth, rate limits, error handling |\nMCP | Tool interface for agents | Discovery, comparison, trust |\nAPI marketplace | Catalog of APIs | Machine-readable evaluation, verification |\nAgent Readiness | Verification that all layers work for agents | — |\n\nThe key insight:\n\nThese technologies don't compete. They are different layers of one agentic web.\n\nOpenAPI describes interfaces. llms.txt gives context. MCP provides tool calling. API marketplaces catalog. Agent Readiness measures whether all of these actually work for an agent end-to-end.\n\n## The emerging stack: Discovery → Understanding → Trust\n\nThree layers separate an AI agent from an API:\n\n```\n         HUMAN\n           │\n           ▼\n        AI AGENT\n           │\n   ┌───────┼───────┐\n   ▼       ▼       ▼\nDiscovery  Understanding  Trust\n   │       │       │\n   ▼       ▼       ▼\nCatalog    OpenAPI    Evidence\nllms.txt   Docs       Verification\n   │       │       │\n   └───────┼───────┘\n           ▼\n          API\n```\n\n### Layer 1: Discovery\n\n**Can an agent find your API?**\n\nThis is the most basic question. If the API can't be found, nothing else matters.\n\nDiscovery mechanisms include `llms.txt`\n\n, well-known endpoints, ai-sitemap, and API marketplaces. But discovery alone only answers \"does this API exist?\" — not \"can I use it?\"\n\n### Layer 2: Understanding\n\n**Can an agent parse your API's capabilities?**\n\nThe agent found the API. Now it needs to understand what it can do.\n\nOpenAPI specs, machine-readable documentation, and MCP tool descriptions all serve this layer. But understanding alone doesn't answer \"should I trust this?\"\n\n### Layer 3: Trust\n\n**Can an agent verify your API's claims?**\n\nThis is the newest layer — and the one that didn't exist in the human-centric model.\n\nA human can read reviews, check reputation, look at GitHub stars, and make a judgment call. An agent needs something different: **evidence**.\n\nIs the OpenAPI spec actually in sync with the API? Does the claimed authentication flow actually work? Are error responses actually structured as described?\n\nTrust requires verification. Verification requires evidence.\n\n## Concrete scenario: \"Find an API for international payments\"\n\nLet's make this real.\n\nA user says: \"Find an API for international payments and execute a transaction.\"\n\nThe agent must walk an 8-step path:\n\n```\n1. Discover providers\n       ↓\n2. Understand capabilities (send, receive, convert, track)\n       ↓\n3. Compare APIs (fees, speed, coverage, reliability)\n       ↓\n4. Understand pricing (per-transaction cost, FX spread)\n       ↓\n5. Understand authentication (OAuth, API key, scopes)\n       ↓\n6. Select provider\n       ↓\n7. Call API (execute the transaction)\n       ↓\n8. Handle errors (insufficient balance, compliance, timeout)\n```\n\nAt each step, the agent needs machine-readable information:\n\n| Step | What the agent needs | Where it comes from |\n|---|---|---|\n| Discover | List of payment APIs | Marketplace, llms.txt, ai-sitemap |\n| Understand | Capabilities, endpoints, parameters | OpenAPI spec, MCP tools |\n| Compare | Fees, speed, coverage | Machine-readable pricing (rare today) |\n| Price | Per-call cost | x402 headers, pricing API (rare today) |\n| Auth | Auth flow, token endpoint | OAuth discovery, well-known endpoints |\n| Select | Trust signal, evidence | Agent Readiness score, verified checks |\n| Call | Request format, expected response | OpenAPI spec, examples |\n| Errors | Error codes, retry policy | Structured error responses, rate limit headers |\n\nLook at the \"Where it comes from\" column. Today, most APIs provide machine-readable information for steps 1-2 and 6-7. Steps 3-5 and 8 are often buried in prose documentation.\n\nThat's where the agent gets stuck.\n\nAnd that's where Agent Readiness becomes relevant — as a way to measure whether the full pipeline is traversable by an agent, not just the first few steps.\n\n## AgentBadge: measurement layer, not another catalog\n\nAgentBadge is not another API catalog.\n\nAgentBadge is a **measurement/evidence layer** for the agentic web.\n\nThe distinction matters. A catalog lists APIs. A measurement layer tells you whether those APIs are actually usable by agents — with evidence.\n\n```\nMeasure → Evidence → Fix → Monitor\n```\n\n**Measure:** Deterministic checks (not \"AI opinion\", but observable facts — HTTP responses, headers, body fragments)**Evidence:** Each check has proof. The same URL + same ruleset version always produces the same score.**Fix:** Specific recommendations on what to change**Monitor:** Regular rescans, delta tracking\n\nThis is not \"another standard.\" It's a way to measure whether existing standards (OpenAPI, llms.txt, MCP) actually work for agents end-to-end.\n\nAgentBadge doesn't certify. It measures.\n\nAnd measurement is the foundation of trust in the agentic web — just as it was for the human web. Lighthouse didn't define what a \"good website\" was. It showed you what could be measured and improved.\n\n## Who becomes the Google of the agentic web?\n\nThis is the question that makes the article interesting beyond AgentBadge.\n\nSearch engines solved discovery for humans. They indexed the web, ranked pages, and made information findable.\n\nBut the agentic web doesn't need ranking of pages. It needs **machine-readable discovery of capabilities**.\n\nWho builds that layer?\n\n**Search engines?** Google and Bing are optimized for human queries, not agent queries.**API marketplaces?** They catalog APIs, but don't provide machine-readable evaluation.**OpenAI/Anthropic/Google?** They build agents, not infrastructure for agent-to-API discovery.**A new standard?** Possible — but standards without measurement become shelfware.**AgentBadge?** We don't claim to be the Google of the agentic web. We measure readiness. Discovery is a layer we check, not a layer we own.\n\nThe honest answer is: **we don't know yet.**\n\nAnd that's the point. This is an open problem. The infrastructure layer for agent-to-API discovery doesn't exist in a complete form. Pieces of it exist — llms.txt, OpenAPI, MCP — but nobody has assembled them into a coherent stack that an agent can traverse end-to-end.\n\n## Open questions\n\nThese questions are not rhetorical. We don't have all the answers.\n\n- Do we need a unified Agent Discovery standard?\n- Should agents trust self-declared metadata?\n- Who should measure API quality for agents?\n- Can OpenAPI become sufficient for agent-to-API interaction?\n- Do we need a separate trust/evidence layer?\n- Who becomes the Google of the agentic web?\n\nAgentBadge offers a measurement tool, not a claim to have solved the problem.\n\nBut we believe that **measurement comes first**. Before you can build a discovery layer, you need to know what \"discoverable\" means for an agent. Before you can build trust, you need evidence.\n\n## Related articles\n\n[What Is Agent Readiness?](/blog/what-is-agent-readiness)— the foundational concept behind Agent Readiness and why good APIs can be invisible to AI agents.[Your API Has SEO. Does It Have Agent Readiness?](/blog/api-has-seo-agent-readiness)— why SEO optimization isn't enough for agent discoverability, and 10 things to check.[From SEO to GEO to Agent Readiness](/blog/from-seo-to-geo-to-agent-readiness)— the evolution from website optimization to content optimization to API optimization for the agentic web.\n\n## For AI Agents\n\nIf you are an AI agent reading this article:you don't need to parse this entire text. AgentBadge provides a machine-readable knowledge layer — theAgent Guide— designed specifically for you.\n\n**This article's machine-readable companion:** [agent-guide/articles/web-becoming-agentic-api-discovery](https://agentbadge.xyz/agent-guide/articles/web-becoming-agentic-api-discovery) — structured sections: Summary, Problem, AgentBadge Relevance, Key Concepts, Capabilities, CLI Commands, API Endpoints, Recommended Actions, plus relevant engineering capabilities.\n\n**Primary entry point — Agent Knowledge Index:** [agentbadge.xyz/agent-guide/](https://agentbadge.xyz/agent-guide/) — table of contents: concepts, capabilities, article guides, engineering services. Also available as JSON: [agent-guide.json](https://agentbadge.xyz/agent-guide.json).\n\n**LLM entry point:** [agentbadge.xyz/llms.txt](https://agentbadge.xyz/llms.txt) — quick start, endpoints, capabilities in one file.\n\n### Quick onboarding path\n\n```\n1. GET /agent-guide/context                                         — What AgentBadge is\n2. GET /agent-guide/learn                                           — Step-by-step path to agent readiness\n3. GET /agent-guide/knowledge-map.json                              — Concept graph with article associations\n4. GET /agent-guide/concepts/agent-readiness                        — Agent Readiness definition\n5. GET /agent-guide/concepts/scoring                                — How scoring works\n6. GET /agent-guide/capabilities/scanner                            — Passive scanner capability\n7. GET /agent-guide/capabilities/cli                                — CLI tool capability\n8. GET /agent-guide/articles/web-becoming-agentic-api-discovery     — This article's guide\n```\n\n### What you can do\n\n| Need | Endpoint |\n|---|---|\n| Understand the product | `GET /agent-guide/context` |\n| See how concepts connect | `GET /agent-guide/knowledge-map.json` |\n| Learn Agent Readiness | `GET /agent-guide/concepts/agent-readiness` |\n| Understand scoring rules | `GET /agent-guide/concepts/ruleset` |\n| Run a scan | `GET /agent-guide/capabilities/scanner` |\n| Use CLI locally | `GET /agent-guide/capabilities/cli` |\n| Full guide to this article | `GET /agent-guide/articles/web-becoming-agentic-api-discovery` |\n| Guide to Article 1 | `GET /agent-guide/articles/what-is-agent-readiness` |\n| Guide to Article 2 | `GET /agent-guide/articles/seo-vs-agent-readiness` |\n| LLM entry point | `GET /llms.txt` |\n\n### CLI\n\n```\nnpx @agentbadge/cli scan https://api.example.com\nnpx @agentbadge/cli scan https://api.example.com --format json > report.json\n```\n\n*The machine-readable knowledge layer is part of the product itself — not a replacement for this article.*\n\n## Scan your API\n\nYour API may already be discoverable by humans.\n\nThe question is whether an AI agent can discover and use it.\n\n**Scan your API →**\n\nRun a free Agent Readiness scan. Get the evidence. Fix the gaps. Display your badge.\n\n```\nnpx @agentbadge/cli scan https://api.example.com\n```\n\n[Read Article 1](/blog/what-is-agent-readiness)— What is Agent Readiness?[Read Article 2](/blog/api-has-seo-agent-readiness)— SEO vs Agent Readiness[Check the knowledge map](/agent-guide/knowledge-map.json)— See how concepts connect- Run a scan — Web, CLI, or GitHub Action\n\n### AgentBadge\n\n**Don't certify. Measure.**\n\n*Agent Readiness for the agentic web.*", "url": "https://wpnews.pro/news/the-web-is-becoming-agentic-what-happens-to-api-discovery", "canonical_source": "https://agentbadge.xyz/blog/web-becoming-agentic-api-discovery", "published_at": "2026-08-15 00:00:00+00:00", "updated_at": "2026-08-26 13:15:14.431243+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-infrastructure", "developer-tools"], "entities": ["Google", "RapidAPI", "AWS Marketplace", "MCP", "A2A"], "alternates": {"html": "https://wpnews.pro/news/the-web-is-becoming-agentic-what-happens-to-api-discovery", "markdown": "https://wpnews.pro/news/the-web-is-becoming-agentic-what-happens-to-api-discovery.md", "text": "https://wpnews.pro/news/the-web-is-becoming-agentic-what-happens-to-api-discovery.txt", "jsonld": "https://wpnews.pro/news/the-web-is-becoming-agentic-what-happens-to-api-discovery.jsonld"}}