Inside an Agent Readiness Scanner: Rules, Evidence and Reproducibility

AgentBadge's Agent Readiness scanner assigns scores like 87/100 through a deterministic pipeline of rules, evidence, assertions, and scoring, rejecting LLM-based opinions in favor of reproducible measurements. The scanner checks specific artifacts such as OpenAPI documents and robots.txt, with each assertion marked VERIFIED, INFERRED, CONFLICT, or MISSING, and rules versioned for reproducibility.

Inside an Agent Readiness Scanner: Rules, Evidence and Reproducibility Why 87/100? Evidence over opinions in API measurement. Rules, evidence, assertions, scoring, and reproducibility — inside the AgentBadge scanner. An Agent Readiness score is not an LLM opinion. It's the output of a deterministic pipeline: Rules → Evidence → Assertions → Score → Report. Every assertion has a status VERIFIED, INFERRED, CONFLICT, MISSING . Rules are versioned. Results are reproducible: same target + same measurement state + same ruleset = same result. For AI agents: Start with the Agent Knowledge Layer to access machine-readable concepts, scanner capabilities, scoring rules, and the complete learning path. When someone tells you that an API has an 87/100 Agent Readiness score , the first question should not be: "Is 87 a good score?" The better question is: "Why is it 87?" And the question after that is even more important: "Can I reproduce the result myself?" Agent Readiness should not be an opinion generated by an LLM. It should be a measurable property of a service, calculated from explicit rules and supported by evidence. The core idea is simple: Rules → Evidence → Assertions → Score → Report 1. A scanner should measure, not guess Imagine two tools scanning the same API. Tool A says: "Your API appears to be highly suitable for AI agents." Tool B says: "AB-004 passed because https://example.com/openapi.json returned HTTP 200 and contained a valid OpenAPI document." Which result would you trust? The second one tells you what was checked, what rule was applied, what evidence was found, why the rule passed or failed, and where the evidence came from. Every meaningful score should be explainable through evidence. The scanner should not ask an AI model: "How agent-ready does this API feel?" It should ask deterministic questions: "Does this URL exist?" "Does it return the expected content type?" "Does the response contain an OpenAPI document?" 2. Rules are the measurement instrument AgentBadge treats rules as explicit, versioned measurement definitions. AB-001 Name: OpenAPI discoverability Given: target = https://example.com Check: GET /.well-known/openapi.json Pass when: HTTP status = 200 AND response is valid OpenAPI Evidence: URL HTTP status content type content hash Severity: medium The rule is not: "The API looks well documented." It is: "This specific machine-readable artifact was found and passed these specific checks." 3. Deterministic before intelligent Deterministic before intelligent. | Question | Preferred method | |---|---| Does robots.txt exist? | HTTP request | | Does sitemap exist? | HTTP request + parser | | Does OpenAPI exist? | HTTP request + schema validation | | Is JSON valid? | JSON parser | | Does declared endpoint exist in another document? | Exact matching | | What does an undocumented endpoint mean? | AI-assisted inference | | What does an API capability actually mean? | Human confirmation / assisted review | AI is a copilot , not the authority responsible for the score. 4. Evidence is the missing layer A score without evidence is difficult to trust. Documentation: 18/25 tells you almost nothing. AB-007 OpenAPI discoverability STATUS: VERIFIED Evidence: GET https://api.example.com/openapi.json HTTP 200 Content-Type: application/json OpenAPI version: 3.1.0 Confidence: 1.00 Evidence is part of the measurement itself. 5. Assertions connect evidence and scoring { "rule id": "AB-007", "status": "VERIFIED", "target": "https://api.example.com/openapi.json", "evidence": { "http status": 200, "content type": "application/json" }, "confidence": 1.0 } Four states: VERIFIED direct evidence , INFERRED reasonable but insufficient , CONFLICT two sources disagree , MISSING not found . 6. Confidence is not the same as verification An LLM may be 94% confident an API supports refunds. Without machine-readable evidence, it stays INFERRED. The user can Confirm, Edit, or Reject. 7. Reproducibility matters { "target": "https://api.example.com", "ruleset": "agent-readiness-v1.0", "scanner version": "0.1.0", "timestamp": "...", "assertions": ... , "score": { "total": 76, "categories": { "discovery": 18, "documentation": 20, "authentication": 17, "machine readability": 21 } } } 8. Rules must be versioned Agent Readiness v1.0 must not silently become v1.1 . Same target + same measurement state + same ruleset = reproducible result. 9. Why open rules do not destroy the product The moat is not secret rules. It's the standard installed inside the developer workflow: GitHub Actions, README badges, continuous monitoring, regression alerts. 10. The score should explain itself +8 OpenAPI documentation detected -12 New authentication issue detected +0 Discovery unchanged Result: 76 → 72 11. From Measure to Prove to Improve MEASURE Scan → PROVE Evidence + Assertions → IMPROVE Fix / Review → Measure again. 12. What AgentBadge should never claim AgentBadge measures Agent Readiness. It does not certify security, business correctness, or reliability. Don't certify. Measure. 13. What this enables npx @agentbadge/cli scan https://api.example.com CI enforcement, README badges, programmatic queries, vendor comparison. 14. The bigger idea Performance has metrics. Accessibility has checks. Security has scanners. The agentic web needs the same. Define the rules. Collect the evidence. Show the reasoning. Version the rules. Make the result reproducible. Related Articles What Is Agent Readiness? /blog/what-is-agent-readiness — Article 1 Why AI Agents Fail to Use APIs /blog/why-ai-agents-fail-to-use-apis — Article 5 What Does an AI Agent Need to Understand an API? /blog/what-ai-agent-needs-to-understand-api — Article 6 Why Your OpenAPI Spec Isn't Enough /blog/why-openapi-isnt-enough — Article 7 How Do You Measure Agent Readiness? /blog/how-do-you-measure-agent-readiness — Article 8 For AI agents: the Agent Knowledge Layer provides machine-readable access to this article's concepts, capabilities, and knowledge map.