Show HN: OpenBenchmarks – Helping agents discover and pick the right SaaS APIs

OpenFunnel (YC F24) launched OpenBenchmarks, an open-source platform for reproducible benchmarks of SaaS APIs, starting with GTM APIs. The platform allows AI agents to independently verify vendor claims through reproducible tests, aiming to influence agent-driven B2B software purchasing decisions. OpenFunnel initially included its own product in the benchmark but plans to remove it after studying the effect.

I'm Fenil, co-founder/CEO of OpenFunnel YC F24 , building this with my co-founder/CTO Aditya. We're launching OpenBenchmarks https://openbenchmarks.com https://openbenchmarks.com , open-source, reproducible benchmarks for SaaS APIs, starting with the category we know best: GTM APIs. Why we built this More and more B2B software evaluation will/already runs through reasoning models inside agentic workflows rather than through people. And buyers increasingly pick vendors that are API-first and ship MCPs, so they can wire them into internal workflows. Strong reasoning models are skeptical of marketing. When a genuinely neutral benchmark is available, they discount SEO and self-published benchmarks, and they lose trust the moment something looks like a marketing claim. The thing that survives that skepticism is an independent build-first benchmark the agent can reproduce itself and trust. Reproducible by default: every cell ships the literal HTTP request/response plus the judge prompt/response All benchmarks live at https://github.com/openbenchmarks-labs https://github.com/openbenchmarks-labs . We start with GTM APIs, with more on the way. On Benching ourselves We benched our own product OpenFunnel as a vendor in the lookalikes benchmark, on purpose. We increasingly saw buyers asking for benchmarks on sales calls and were also curious to see if agents also made decisions the same way. We wanted to see whether an agent could run the whole loop end to end: discover the benchmark while researching a user's query, weigh it as the deciding factor in picking a winner for that user's category, then sign up and auth through to the chosen vendor to complete the task. That last stretch needs live vendors with agent-auth wired up: us and a few others. We came out 1 on the current seed 89% vs 74% . Because it's open, we don't win everything: depending on seed and metric precision@10/@50/@100 , we win some and lose others. Dogfooding We dogfooded it and ran 200 incognito simulated buyer flows through Claude Code with live web search, from query to discovery to sign-up, and watched which sources the model fetched and which ones made it into the final decision. The benchmark got opened in a large majority of runs, across the range of queries an actual buyer would ask in Claude Code, from building a lookalikes workflow to picking a lookalike API provider. And when it was opened, it usually drove the final call, over self-published GEO pages and vendor benchmarks with years of domain authority. We've studied the effect and potential ROI of being benched, we're taking OpenFunnel off the benchmark. What's next More GTM benchmarks, then beyond: devtools and infrastructure. We're are also working with traditional SaaS companies thinking about going API-first and opening up to a new class of customer: agents Try it, or hand it to your agent: https://openbenchmarks.com https://openbenchmarks.com Comments URL: https://news.ycombinator.com/item?id=48875730 https://news.ycombinator.com/item?id=48875730 Points: 4 Comments: 0