{"slug": "show-hn-an-ai-office-work-benchmark-and-4-bugs-we-found-in-our-own-judge", "title": "Show HN: An AI office-work benchmark, and 4 bugs we found in our own judge", "summary": "A developer released an AI office-work benchmark and found four bugs in its own evaluation judge, highlighting challenges in measuring AI performance on real-world tasks. The benchmark, posted on Hacker News, aims to assess AI agents on office-related activities, but the self-identified flaws raise questions about the reliability of such evaluations.", "body_md": "Skip to content\nNavigation Menu\nSign in\nAppearance settings\nPlatform\nAI CODE CREATION\nGitHub Copilot\nWrite better code with AI\nGitHub Copilot app\nDirect agents from issue to merge\nMCP Registry\nIntegrate external tools\nDEVELOPER WORKFLOWS\nActions\nAutomate any workflow\nCodespaces\nInstant dev environments\nIssues\nPlan and track work\nCode Review\nManage code changes\nCode Quality\nEnforce quality at merge\nAPPLICATION SECURITY\nGitHub Advanced Security\nFind and fix vulnerabilities\nCode security\nSecure your code as you build\nSecret protection\nStop leaks before they start\nEXPLORE\nWhy GitHub\nDocumentation\nBlog\nChangelog\nMarketplace\nView all features\nSolutions\nBY COMPANY SIZE\nEnterprises\nSmall and medium teams\nStartups\nNonprofits\nBY USE CASE\nApp Modernization\nDevSecOps\nDevOps\nCI/CD\nView all use cases\nBY INDUSTRY\nHealthcare\nFinancial services\nManufacturing\nGovernment\nView all industries\nView all solutions\nResources\nEXPLORE BY TOPIC\nAI\nSoftware Development\nDevOps\nSecurity\nView all topics\nEXPLORE BY TYPE\nCustomer stories\nEvents & webinars\nEbooks & reports\nBusiness insights\nGitHub Skills\nSUPPORT & SERVICES\nDocumentation\nCustomer support\nCommunity forum\nTrust center\nPartners\nView all resources\nOpen Source\nCOMMUNITY\nGitHub Sponsors\nFund open source developers\nPROGRAMS\nSecurity Lab\nMaintainer Community\nAccelerator\nGitHub Stars\nArchive Program\nREPOSITORIES\nTopics\nTrending\nCollections\nEnterprise\nENTERPRISE SOLUTIONS\nEnterprise platform\nAI-powered developer platform\nAVAILABLE ADD-ONS\nGitHub Advanced Security\nEnterprise-grade security features\nCopilot for Business\nEnterprise-grade AI features\nPremium Support\nEnterprise-grade 24/7 support\nPricing\nType\n/\nto search\nSign in\nSign up\nAppearance settings\nYou signed in with another tab or window.\nReload\nto refresh your session.\nYou signed out in another tab or window.\nReload\nto refresh your session.\nYou switched accounts on another tab or window.\nReload\nto refresh your session.\nDismiss alert\n{{ message }}\nFind code, projects, and people on GitHub:\nSearch\nContact Support\n—\nGitHub Status\n—\n@githubstatus\nYou can’t perform that action at this time.", "url": "https://wpnews.pro/news/show-hn-an-ai-office-work-benchmark-and-4-bugs-we-found-in-our-own-judge", "canonical_source": "https://github.com/dongsheng123132/ai2work-bench", "published_at": "2026-08-05 07:00:56+00:00", "updated_at": "2026-08-05 07:23:00.294808+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-tools"], "entities": ["Hacker News"], "alternates": {"html": "https://wpnews.pro/news/show-hn-an-ai-office-work-benchmark-and-4-bugs-we-found-in-our-own-judge", "markdown": "https://wpnews.pro/news/show-hn-an-ai-office-work-benchmark-and-4-bugs-we-found-in-our-own-judge.md", "text": "https://wpnews.pro/news/show-hn-an-ai-office-work-benchmark-and-4-bugs-we-found-in-our-own-judge.txt", "jsonld": "https://wpnews.pro/news/show-hn-an-ai-office-work-benchmark-and-4-bugs-we-found-in-our-own-judge.jsonld"}}