{"slug": "startup-spotlight-confident-ai-turns-production-agent-failures-into-tests-for", "title": "Startup Spotlight: Confident AI turns production agent failures into tests for the next release", "summary": "Confident AI, led by co-founder and CEO Jeffrey Ip, is building a hosted platform that converts failed production AI agent traces into new tests for the next release, connecting to the open-source evaluation framework DeepEval that Ip built earlier. Customer Finom reports that the platform let its product managers run evaluations directly, reducing waits for engineering, though Finom's case study does not measure AI quality. The company, listed on Y Combinator, is trying to turn open-source evaluation adoption into an enterprise workflow business.", "body_md": "# Startup Spotlight: Confident AI turns production agent failures into tests for the next release\n\n**Jeffrey Ip built DeepEval before Confident AI's hosted platform; Finom reports faster iteration, but its case study does not measure AI quality.**\n\n        By [RuntimeWire Staff](https://runtimewire.com/author/runtimewire-staff)\n        · Published \n\nPrimary source: [Y Combinator](https://www.ycombinator.com/companies/confident-ai)\n\n## Why it matters\n\nConfident AI is trying to turn open-source evaluation adoption into an enterprise workflow business. Finom says the platform let product managers run evaluations directly, reducing waits for engineering.\n\n[Confident AI](https://confident-ai.com/?ref=runtimewire) targets the work triggered when an AI agent fails in production: finding the cause and preventing the next release from repeating it. Co-founder and CEO [Jeffrey Ip](https://www.ycombinator.com/founders/1314014-jeffrey-ip?ref=runtimewire) ([@jeffr_yyy](https://x.com/jeffr_yyy?ref=runtimewire)) made that loop the product, connecting its open-source evaluation framework to a hosted system that can turn troublesome production traces into new tests.…\n\nWhen you purchase through links in our articles, [we may earn a small commission](https://runtimewire.com/affiliate-monetization-standards). This doesn’t affect our editorial independence.", "url": "https://wpnews.pro/news/startup-spotlight-confident-ai-turns-production-agent-failures-into-tests-for", "canonical_source": "https://runtimewire.com/article/confident-ai-jeffrey-ip-kritin-vongthongsri-ai-evals", "published_at": "2026-10-26 16:13:20+00:00", "updated_at": "2026-10-26 19:15:52.735307+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-startups", "mlops", "ai-safety"], "entities": ["Confident AI", "Jeffrey Ip", "DeepEval", "Finom", "Y Combinator"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/startup-spotlight-confident-ai-turns-production-agent-failures-into-tests-for", "markdown": "https://wpnews.pro/news/startup-spotlight-confident-ai-turns-production-agent-failures-into-tests-for.md", "text": "https://wpnews.pro/news/startup-spotlight-confident-ai-turns-production-agent-failures-into-tests-for.txt", "jsonld": "https://wpnews.pro/news/startup-spotlight-confident-ai-turns-production-agent-failures-into-tests-for.jsonld"}}