{"slug": "show-hn-openbenchmarks-helping-agents-discover-and-pick-the-right-saas-apis", "title": "Show HN: OpenBenchmarks – Helping agents discover and pick the right SaaS APIs", "summary": "OpenFunnel (YC F24) launched OpenBenchmarks, an open-source platform for reproducible benchmarks of SaaS APIs, starting with GTM APIs. The platform allows AI agents to independently verify vendor claims through reproducible tests, aiming to influence agent-driven B2B software purchasing decisions. OpenFunnel initially included its own product in the benchmark but plans to remove it after studying the effect.", "body_md": "I'm Fenil, co-founder/CEO of OpenFunnel (YC F24), building this with my co-founder/CTO Aditya. We're launching OpenBenchmarks ([https://openbenchmarks.com](https://openbenchmarks.com)), open-source, reproducible benchmarks for SaaS APIs, starting with the category we know best: GTM APIs.\n\n## Why we built this\n\nMore and more B2B software evaluation will/already runs through reasoning models inside agentic workflows rather than through people. And buyers increasingly pick vendors that are API-first and ship MCPs, so they can wire them into internal workflows. Strong reasoning models are skeptical of marketing. When a genuinely neutral benchmark is available, they discount SEO and self-published benchmarks, and they lose trust the moment something looks like a marketing claim.\n\nThe thing that survives that skepticism is an independent build-first benchmark the agent can reproduce itself and trust.\n\nReproducible by default: every cell ships the literal HTTP request/response plus the judge prompt/response\n\nAll benchmarks live at [https://github.com/openbenchmarks-labs](https://github.com/openbenchmarks-labs). We start with GTM APIs, with more on the way.\n\n## On Benching ourselves\n\nWe benched our own product (OpenFunnel) as a vendor in the lookalikes benchmark, on purpose.\n\nWe increasingly saw buyers asking for benchmarks on sales calls and were also curious to see if agents also made decisions the same way.\n\nWe wanted to see whether an agent could run the whole loop end to end: discover the benchmark while researching a user's query, weigh it as the deciding factor in picking a winner for that user's category, then sign up and auth through to the chosen vendor to complete the task. That last stretch needs live vendors with agent-auth wired up: us and a few others.\n\nWe came out #1 on the current seed (89% vs 74%). Because it's open, we don't win everything: depending on seed and metric (precision@10/@50/@100), we win some and lose others.\n\n## Dogfooding\n\nWe dogfooded it and ran 200 incognito simulated buyer flows through Claude Code with live web search, from query to discovery to sign-up, and watched which sources the model fetched and which ones made it into the final decision. The benchmark got opened in a large majority of runs, across the range of queries an actual buyer would ask in Claude Code, from building a lookalikes workflow to picking a lookalike API provider. And when it was opened, it usually drove the final call, over self-published GEO pages and vendor benchmarks with years of domain authority.\n\nWe've studied the effect and potential ROI of being benched, we're taking OpenFunnel off the benchmark.\n\n## What's next\n\nMore GTM benchmarks, then beyond: devtools and infrastructure.\n\nWe're are also working with traditional SaaS companies thinking about going API-first and opening up to a new class of customer: agents\n\nTry it, or hand it to your agent: [https://openbenchmarks.com](https://openbenchmarks.com)\n\nComments URL: [https://news.ycombinator.com/item?id=48875730](https://news.ycombinator.com/item?id=48875730)\n\nPoints: 4\n\n# Comments: 0", "url": "https://wpnews.pro/news/show-hn-openbenchmarks-helping-agents-discover-and-pick-the-right-saas-apis", "canonical_source": "https://openbenchmarks.com", "published_at": "2026-07-11 20:50:55+00:00", "updated_at": "2026-07-11 21:05:35.478809+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-products", "developer-tools"], "entities": ["OpenBenchmarks", "OpenFunnel", "YC F24", "Fenil", "Aditya", "Claude Code"], "alternates": {"html": "https://wpnews.pro/news/show-hn-openbenchmarks-helping-agents-discover-and-pick-the-right-saas-apis", "markdown": "https://wpnews.pro/news/show-hn-openbenchmarks-helping-agents-discover-and-pick-the-right-saas-apis.md", "text": "https://wpnews.pro/news/show-hn-openbenchmarks-helping-agents-discover-and-pick-the-right-saas-apis.txt", "jsonld": "https://wpnews.pro/news/show-hn-openbenchmarks-helping-agents-discover-and-pick-the-right-saas-apis.jsonld"}}