Can You Beat an LLM? Building Humans vs. Humanity's Last Exam

A developer built Humans vs. HLE, a web-based quiz that pits humans against frontier AI models on Humanity's Last Exam, a benchmark of expert-level academic questions. The app runs on Cloudflare Workers with encrypted answer tokens and a persistent leaderboard using Durable Objects to prevent cheating. The developer noted, 'The interesting engineering challenge in a quiz app is: how do you stop people from just reading the answers in the network tab?'

Frontier models are crushing benchmark after benchmark...so I built a quiz to ask this simple yet humbling question: can a human still beat them? Humans vs. HLE is a web-based quiz that pits you against frontier AI models on Humanity's Last Exam https://agi.safe.ai/. HLE is a benchmark of expert-level academic questions designed to be insanely challenging. You answer a series of multiple-choice questions if you miss 4, you're out and the quiz ends and your score is stacked up both against how the best models perform on the same material as well as against other humans who have taken this quiz via a persistent human leaderboard. Spoiler: it's harder than it looks. Entire tracks prompts, outputs, and tool calls. You can see some of my prompts here. You can see I used Claude code Opus 4.8, edited 27 files, and added 912 lines of code and removed 239 for that checkpoint/feature. The entire see what I did there app runs on Cloudflare Workers. Here are a few decisions I want to call out. Rather than fetch questions at runtime, they're curated offline from the gated cais/hle Hugging Face dataset into src/questions.json and then imported directly by src/questions.ts so the bank ships inside the Worker bundle. This means no database or external fetch, and that the deployed Worker never calls Hugging Face or agi.safe.ai at request time. This means the app is fast to serve and that nothing sensitive travels over the wire. The interesting engineering challenge in a quiz app is: how do you stop people from just reading the answers in the network tab? The approach keeps correct answers server-side the entire quiz lifecycle. GET /api/quiz/new shuffles each question's choices, strips the answers, and hands back an AES-GCM–encrypted answer-key token that's opaque to the browser. POST /api/quiz/answer decrypts it server-side to grade one question at a time. POST /api/quiz/submit returns an HMAC-signed result token — so a score forged in the client simply won't verify. The client gets what it needs to render a question, but never the answer key, and scoring happens where it can't be tampered with. The leaderboard and result signing use Cloudflare Durable Objects, which are my go-to dev tool for consistent, stateful coordination ie. a persistent leaderboard . The entire development process again, see what I did there was tracked with Entire https://entire.io , so every architectural decision why encrypted tokens, why Durable Objects has its reasoning captured alongside the code, not lost to a closed chat window. The pieces that matter, and what each one owns: scripts/ build-questions.py curate cais/hle → questions.json offline src/ questions.json 140 text-only MCQs, baked in questions.ts imports the JSON into the bundle index.ts Worker + the /api/quiz/ routes models.ts published HLE accuracy + AS OF date The API surface: /api/quiz/new , /api/quiz/answer , /api/quiz/submit , and /api/questions for the browse page. Standard Workers setup — install, drop in two secrets, and go. npm install cp .dev.vars.example .dev.vars then set both values in .dev.vars, e.g.: printf "QUIZ TOKEN KEY=%s\n" "$ openssl rand -base64 32 " printf "RESULT TOKEN SECRET=%s\n" "$ openssl rand -base64 32 " npm run dev Tests: npm run test Deploy: wrangler secret put QUIZ TOKEN KEY an openssl rand -base64 32 value wrangler secret put RESULT TOKEN SECRET a different one npm run deploy The AI numbers aren't the models sitting your exact ten questions live. They're each model's published overall accuracy on the full HLE benchmark, sourced from Scale AI's leaderboard https://labs.scale.com/leaderboard/humanitys last exam with an AS OF date in models.ts . So it's you-on-a-sample versus them-on-the-whole-thing — close enough to sting, honest enough to admit. Think you can beat the models? Give it a shot, and let me know your score in the comments as well as the leaderboard.