{"slug": "can-you-beat-an-llm-building-humans-vs-humanity-s-last-exam", "title": "Can You Beat an LLM? Building Humans vs. Humanity's Last Exam", "summary": "A developer built Humans vs. HLE, a web-based quiz that pits humans against frontier AI models on Humanity's Last Exam, a benchmark of expert-level academic questions. The app runs on Cloudflare Workers with encrypted answer tokens and a persistent leaderboard using Durable Objects to prevent cheating. The developer noted, 'The interesting engineering challenge in a quiz app is: how do you stop people from just reading the answers in the network tab?'", "body_md": "Frontier models are crushing benchmark after benchmark...so I built a quiz to ask this simple yet humbling question: can a human still beat them?\n\n**Humans vs. HLE** is a web-based quiz that pits you against frontier AI models on [Humanity's Last Exam](https://agi.safe.ai/.) HLE is a benchmark of expert-level academic questions designed to be insanely challenging. You answer a series of multiple-choice questions (if you miss 4, you're out and the quiz ends!) and your score is stacked up both against how the best models perform on the same material as well as against other humans who have taken this quiz via a persistent human leaderboard.\n\nSpoiler: it's harder than it looks.\n\nEntire tracks prompts, outputs, and tool calls. You can see some of my prompts here.\n\nYou can see I used Claude code Opus 4.8, edited 27 files, and added 912 lines of code (and removed 239) for that checkpoint/feature.\n\nThe entire (see what I did there) app runs on Cloudflare Workers. Here are a few decisions I want to call out.\n\nRather than fetch questions at runtime, they're curated offline from the gated `cais/hle`\n\nHugging Face dataset into `src/questions.json`\n\nand then imported directly by `src/questions.ts`\n\nso the bank ships inside the Worker bundle. This means no database or external fetch, and that the deployed Worker never calls Hugging Face or agi.safe.ai at request time. This means the app is fast to serve and that nothing sensitive travels over the wire.\n\nThe interesting engineering challenge in a quiz app is: how do you stop people from just reading the answers in the network tab?\n\nThe approach keeps correct answers server-side the entire quiz lifecycle. `GET /api/quiz/new`\n\nshuffles each question's choices, strips the answers, and hands back an AES-GCM–encrypted answer-key token that's opaque to the browser. `POST /api/quiz/answer`\n\ndecrypts it server-side to grade one question at a time. `POST /api/quiz/submit`\n\nreturns an HMAC-signed result token — so a score forged in the client simply won't verify. The client gets what it needs to render a question, but never the answer key, and scoring happens where it can't be tampered with.\n\nThe leaderboard and result signing use Cloudflare Durable Objects, which are my go-to dev tool for consistent, stateful coordination (ie. a persistent leaderboard).\n\nThe entire development process (again, see what I did there) was tracked with [Entire](https://entire.io), so every architectural decision (why encrypted tokens, why Durable Objects) has its reasoning captured alongside the code, not lost to a closed chat window.\n\nThe pieces that matter, and what each one owns:\n\n```\nscripts/\n  build-questions.py   # curate cais/hle → questions.json (offline)\nsrc/\n  questions.json       # 140 text-only MCQs, baked in\n  questions.ts         # imports the JSON into the bundle\n  index.ts             # Worker + the /api/quiz/* routes\n  models.ts            # published HLE accuracy + AS_OF date\n```\n\nThe API surface: `/api/quiz/new`\n\n, `/api/quiz/answer`\n\n, `/api/quiz/submit`\n\n, and `/api/questions`\n\nfor the browse page.\n\nStandard Workers setup — install, drop in two secrets, and go.\n\n```\nnpm install\ncp .dev.vars.example .dev.vars\n# then set both values in .dev.vars, e.g.:\n#   printf \"QUIZ_TOKEN_KEY=%s\\n\" \"$(openssl rand -base64 32)\"\n#   printf \"RESULT_TOKEN_SECRET=%s\\n\" \"$(openssl rand -base64 32)\"\nnpm run dev\n```\n\nTests:\n\n```\nnpm run test\n```\n\nDeploy:\n\n```\nwrangler secret put QUIZ_TOKEN_KEY        # an openssl rand -base64 32 value\nwrangler secret put RESULT_TOKEN_SECRET   # a different one\nnpm run deploy\n```\n\nThe AI numbers aren't the models sitting your exact ten questions live. They're each model's published overall accuracy on the full HLE benchmark, sourced from [Scale AI's leaderboard](https://labs.scale.com/leaderboard/humanitys_last_exam) (with an `AS_OF`\n\ndate in `models.ts`\n\n). So it's you-on-a-sample versus them-on-the-whole-thing — close enough to sting, honest enough to admit.\n\nThink you can beat the models? Give it a shot, and let me know your score in the comments (as well as the leaderboard.)", "url": "https://wpnews.pro/news/can-you-beat-an-llm-building-humans-vs-humanity-s-last-exam", "canonical_source": "https://dev.to/entire/can-you-beat-an-llm-building-humans-vs-humanitys-last-exam-1673", "published_at": "2026-07-20 23:05:57+00:00", "updated_at": "2026-07-20 23:30:08.160745+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "developer-tools", "ai-infrastructure"], "entities": ["Cloudflare Workers", "Durable Objects", "Hugging Face", "Scale AI", "Claude Code Opus", "Entire"], "alternates": {"html": "https://wpnews.pro/news/can-you-beat-an-llm-building-humans-vs-humanity-s-last-exam", "markdown": "https://wpnews.pro/news/can-you-beat-an-llm-building-humans-vs-humanity-s-last-exam.md", "text": "https://wpnews.pro/news/can-you-beat-an-llm-building-humans-vs-humanity-s-last-exam.txt", "jsonld": "https://wpnews.pro/news/can-you-beat-an-llm-building-humans-vs-humanity-s-last-exam.jsonld"}}