Show HN: Oqoqo – build evals and custom benchmarks for real-world tasks

Oqoqo, a new tool launched on Hacker News, enables developers to build realistic evals and custom benchmarks for real-world tasks, measuring agent-friendly product surfaces against Codex, Claude Code, OpenClaw, Hermes, Pi, Opencode, Cursor, and GitHub Copilot. It supports regression testing for MCP, CLI, skills, SDK, and other agent-facing interfaces, and allows creating and sharing custom benchmarks for agent discovery and usage.

Most benchmarks today exist in curated environments and do not translate well to the real world. We built Oqoqo to bridge this gap. Oqoqo makes it super simple to build realistic evals and custom benchmarks for tasks users actually care about. Oqoqo can: - Reliably measure how agent friendly your product surfaces are against Codex, Claude Code, OpenClaw, Hermes, Pi, Opencode, Cursor, GitHub Copilot - Regression test MCP, CLI, skills, SDK, and any agent facing interface we are continuously using Oqoqo to dogfood and improve our own MCP/CLI - Create and share custom benchmarks for how agents discover and use your product - Compare models and harnesses for domain specific tasks - See whether new versions improve agent experience Would love to hear feedback and thoughts on what kind of evals you are running today, if you are benchmarking agent facing interfaces, and whether you have published a custom benchmark. Comments URL: https://news.ycombinator.com/item?id=49249987 https://news.ycombinator.com/item?id=49249987 Points: 1 Comments: 0