I've been thinking about API design in the AI era and about dividing responsibilities between a service and an AI agent calling into that service.
I have no firm conclusions here yet, but am still here to report that LLM-friendly APIs enable all sorts of useful things that would be impractical to build into the services themselves.
So, I am a trivia fan who:
So, I had Claude sort my overdue questions into categories and spin up a quiz for me in the style of a beloved quiz show. It has me type in the answers and calls into a decision model to tell me if I'm right or wrong. I can override the model's judgments, but there's rarely a reason to. Even without tuning the prompt, the decision model has been nearly perfect in sorting my responses into correct, incorrect, and other cases.<sup>1</sup> Here, for example, is Jev recognizing an unlisted, not-completely-obvious alternate answer:
The point isn't that this is possible; it's that it's easy. I did it on a whim. Claude<sup>2</sup> one-shotted a usable version in about 5 minutes, and the few modifications I wanted were similarly easy. Just making this for myself took less effort and time than I would have spent writing a hypothetical feature request to a human.3
Consider keeping a decision model API key available in an .env file your LLM knows about, if you're into this sort of thing; managing secrets can be the majority of the human time of a small project these days.
I expect to modify and permute this in any number of ways; I love drilling flashcards more than almost anyone, but even I find myself wanting a bit more variety. As LLMs get better, plugging into an API and asking for something new gets safer, faster, and easier. I expect more of my software usage to follow this pattern.
The other cases are "prompt me" (e.g., if I write "goose" when asked for the state bird of Hawai'i, "throw this question out" (if the model detects that the question is somehow ill-formed), and "did not attempt" (which still counts as wrong on the Zippyflash back-end but does not deduct points in the game).↩
Opus 5.5.[↩](https://www.natemeyvis.com/decision-model-application-of-the-day/#fnref-2)
...if I were not also the human running Zippyflash.[↩](https://www.natemeyvis.com/decision-model-application-of-the-day/#fnref-3)