cd /news/ai-agents/item-review-desk-exam-questions-that… · home › topics › ai-agents › article
[ARTICLE · art-143513] src=dev.to ↗ pub= topic=ai-agents verified=true sentiment=· neutral

Item Review Desk: exam questions that can't go live until a second person signs them off

A developer built Item Review Desk, a Sanity-backed exam item review system that enforces a separation-of-duties workflow where AI agents can draft and submit exam questions but only a human other than the author can approve them. The project, built end to end by a Claude agent from a domain brief, encodes item-writing lint rules and a shared workflow rules file so the drafting agent has no code path to the approved state, with a Next.js practice bank serving only approved items.

by read4 min views1 publishedOct 1, 2026

This is a submission for the Sanity Challenge, Path Two: Vibe-Code Something Strange I run a training school in Lagos and have spent years writing exam questions and certification prep for working professionals, a lot of it for customer service and contact centre roles. The part of that job nobody sees is item review. A question gets drafted, a subject expert checks it, it goes back for changes, it gets approved, and years later it gets retired because the process it tests has changed. In most teams that trail lives in email threads and a spreadsheet column called "status".

Item Review Desk puts that process into Sanity as data. Each exam item carries its own state and a log of every move: who moved it, from where to where, when, and why. An AI agent can draft items and submit them for review. Only a person can approve, and the person who wrote an item cannot approve it. A Next.js site builds a practice bank from approved items only, and a board shows how far each exam blueprint is from its target.

The sample content is a real-shaped exam: Contact Centre Associate, Level 1, with four learning objectives and twelve items sitting in every state (approved, in review, changes requested, draft).

https://github.com/drainocode/item-review-desk Honest version first: I didn't type this code. I use an AI agent (Claude) that looks for challenges I can enter and builds the entries, based on my background. This one it built end to end in one session from a brief rooted in how I run item reviews. My part was the domain, reviewing the result, setting up the Sanity project and publishing. So this writeup covers what the agent did and where it went wrong, taken from the session.

Stack. Next.js 16, Sanity 6 with the Studio embedded at /studio through next-sanity, groq-js for an offline mode, and plain Node test runner for the rules.

The main design choice: one rules file for people and agents. lib/workflow.ts defines the states and transitions and who may fire each one:

Move From Allowed Extra rule
Submit for review draft, changes requested human, agent no item-writing errors
Request changes in review human note required
Approve in review human no errors, approver is not the author
Retire approved human note required
Reopen retired, approved human note required

The Studio document actions and the drafting agent script both call the same check() and buildPatch() functions, so an agent literally has no code path to "approved". The plain Publish button is removed for items; the only way an item reaches the practice site is through a logged transition.

Item-writing checks as a custom input. lib/lint.ts encodes the rules I'd give a new item writer: one clear problem in the stem, negatives like NOT and EXCEPT in capitals, three to five options, exactly one key, no "all of the above", a rationale, a linked objective. It also warns when the correct option is much longer than the distractors (candidates learn to pick the longest answer) and when only the key repeats words from the stem. The checks show live inside the item form and errors disable Submit and Approve.

Where it went wrong and how it was fixed:

next build failed while prerendering the home page (HTTP 403, host not in allowlist). Rather than skip verification, it added an OFFLINE_SEED=1 mode that runs the exact same GROQ queries against seed.ndjson using groq-js. That let it build, run and screenshot the site, and it doubles as a way for anyone to preview the project without an account.publish in the same instant, and at that instant the Studio still thought there was nothing to publish, so it skipped it. The fix writes the transition straight to the published document in one transaction and removes the draft. I discarded the stuck drafts, ran both actions again, and the approved item went live with the move logged on the board. What I'd do next: move the agent from a script to Sanity Functions so it drafts items when an objective falls below its blueprint target, and add a second reviewer step for high-stakes exams.

x97jsgh6 production exam (with blueprint rows), objective, item (options with "why a candidate might pick this", rationale, cognitive level, state, transition log)

── more in #ai-agents 4 stories · sorted by recency
── more on @sanity 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/item-review-desk-exa…] indexed:0 read:4min 2026-10-01 · —