Decision model application of the day

A blogger used Anthropic's Claude (identified in the post as Opus 5.5) to build a quiz game that calls a decision-model API to grade typed answers, and reports the model was "nearly perfect" at sorting responses into correct, incorrect and other cases without prompt tuning. Claude produced a usable version in about 5 minutes, and the author argues LLM-friendly APIs let agents assemble features that would be impractical to build into services themselves. The post recommends keeping a decision-model API key in an .env file the LLM can access, noting secret management can consume most of the human time on a small project.

I've been thinking about https://www.natemeyvis.com/blog/?q=API%20design API design in the AI era and about dividing responsibilities between a service and an AI agent calling into that service. I have no firm conclusions here yet, but am still here to report that LLM-friendly APIs enable all sorts of useful things that would be impractical to build into the services themselves. So, I am a trivia fan who: So, I had Claude sort my overdue questions into categories and spin up a quiz for me in the style of a beloved quiz show. It has me type in the answers and calls into a decision model https://www.natemeyvis.com/blog/?q=decision%20models to tell me if I'm right or wrong. I can override the model's judgments, but there's rarely a reason to. Even without tuning the prompt, the decision model has been nearly perfect in sorting my responses into correct, incorrect, and other cases.<sup 1 https://www.natemeyvis.com/decision-model-application-of-the-day/ fn-1 </sup Here, for example, is Jev recognizing an unlisted, not-completely-obvious alternate answer: The point isn't that this is possible; it's that it's easy . I did it on a whim. Claude<sup 2 https://www.natemeyvis.com/decision-model-application-of-the-day/ fn-2 </sup one-shotted a usable version in about 5 minutes, and the few modifications I wanted were similarly easy. Just making this for myself took less effort and time than I would have spent writing a hypothetical feature request to a human. 3 https://www.natemeyvis.com/decision-model-application-of-the-day/ fn-3 Consider keeping a decision model API key available in an .env file your LLM knows about, if you're into this sort of thing; managing secrets can be the majority of the human time of a small project these days. I expect to modify and permute this in any number of ways; I love drilling flashcards more than almost anyone, but even I find myself wanting a bit more variety. As LLMs get better, plugging into an API and asking for something new gets safer, faster, and easier. I expect more of my software usage to follow this pattern. The other cases are "prompt me" e.g., if I write "goose" when asked for the state bird of Hawai'i <https://en.wikipedia.org/wiki/Nene bird , "throw this question out" if the model detects that the question is somehow ill-formed , and "did not attempt" which still counts as wrong on the Zippyflash back-end but does not deduct points in the game . ↩ https://www.natemeyvis.com/decision-model-application-of-the-day/ fnref-1 Opus 5.5. ↩ https://www.natemeyvis.com/decision-model-application-of-the-day/ fnref-2 ...if I were not also the human running Zippyflash. ↩ https://www.natemeyvis.com/decision-model-application-of-the-day/ fnref-3