{"slug": "show-hn-evaluation-context-protocol-ecp", "title": "Show HN: Evaluation Context Protocol (ECP)", "summary": "A developer released Evaluation Context Protocol (ECP), a vendor-neutral protocol for testing AI agent outputs, tool calls, and audit context across frameworks and CI systems. ECP complements MCP by focusing on evaluation rather than tool access, and runs locally or in CI with a JSON-RPC contract.", "body_md": "# Portable evaluations\n\nfor AI agents.\n\nECP is a vendor-neutral protocol for testing agent outputs, tool calls, and evaluator-visible audit context — across frameworks, models, eval platforms, and CI systems.\n\n``` bash\n$ pip install \"ecp-runtime==0.3.3\" \"ecp-sdk==0.3.3\"\n$ ecp init\n$ ecp run --manifest ecp_eval/manifest.yaml --json\n\n# 3 scenarios · 7 graders · all passed ✓\n```\n\n## MCP is for tools. ECP is for evals.\n\nMCP gives agents a common way to use tools. ECP gives evaluators a common way to inspect what an agent returned, what tools it used, and what audit evidence it exposed — independent of the framework that built the agent or the platform that runs the test.\n\n## Beyond the final answer.\n\nMost evals start with the final answer. ECP also checks the behavior behind it.\n\n`public_output`\n\n`tool_calls`\n\n`evaluation_context`\n\n`ecp run --manifest`\n\n### Runs anywhere\n\nRun evals locally or wire ecp run into your CI. Exits non-zero on failure, so a regression breaks the build.\n\n### Framework neutral\n\nWrap agents built with plain Python, LangChain, LlamaIndex, CrewAI, or PydanticAI behind one evaluation contract.\n\n### JSON-RPC contract\n\nImplement the protocol in any language: agent/initialize, agent/step, agent/reset over stdio or Streamable HTTP.\n\n## Your existing agent stack.\n\n## Start grading agents in five commands.\n\nInstall the runtime, initialize a starter manifest, and run your first eval.", "url": "https://wpnews.pro/news/show-hn-evaluation-context-protocol-ecp", "canonical_source": "https://www.evaluationcontextprotocol.io/", "published_at": "2026-06-30 15:56:33+00:00", "updated_at": "2026-06-30 16:21:03.424289+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-research", "developer-tools", "large-language-models"], "entities": ["ECP", "MCP", "LangChain", "LlamaIndex", "CrewAI", "PydanticAI", "JSON-RPC"], "alternates": {"html": "https://wpnews.pro/news/show-hn-evaluation-context-protocol-ecp", "markdown": "https://wpnews.pro/news/show-hn-evaluation-context-protocol-ecp.md", "text": "https://wpnews.pro/news/show-hn-evaluation-context-protocol-ecp.txt", "jsonld": "https://wpnews.pro/news/show-hn-evaluation-context-protocol-ecp.jsonld"}}