Most "AI tutor" apps are a chat window bolted onto a system prompt. We wanted something that actually holds state — knows what you've mastered, adapts difficulty, remembers you tomorrow — across five very different domains: a general subject tutor, exam prep for India's JEE/NEET/UPSC (each with its own official syllabus and negative-marking rules), a language-learning track that goes CEFR A0 through C1, an AI career coach with mock interviews, and a competitive "arena" mode.
That range turned out to be the actual engineering problem. Not "can an LLM explain photosynthesis" — every model can do that — but: what happens when you have 60+ distinct AI-backed features, each with different cost/quality tradeoffs, and one of them starts failing at 2am?
Early on, model calls were scattered — each feature picked its own model, its own retry logic (often none), its own idea of what "thinking budget" meant. A cost audit found thinking tokens (billed at the output rate) were the single largest leak in the whole system. That's what forced the redesign.
Now every AI-backed feature is one entry in a single registry file:
\
js
export const FEATURES = {
chat_message: { sparks: 2, chain: LITE_FIRST, stream: true },
generate_quiz: { sparks: 1, chain: LITE_FIRST, cacheTtlMs: 30 * 60_000 },
km_build_concepts: { sparks: 8, chain: FLASH_FIRST },
// ...60+ more, one line each
};
``
Each entry declares its price, its model chain (primary + fallbacks), and its generation config. The router walks the chain on failure:
If a cheaper step in the chain ends up answering, the router refunds the price difference automatically. Response caching is an allowlist, not a heuristic — only deterministic-ish generators (quizzes, reviews, curricula) get a TTL; personalized chat is explicitly never cached. And the client-side paywall keeps its own copy of the prices — a test asserts the two stay byte-identical, so drift is a CI failure instead of a support ticket. None of this is exotic. It's the boring, unglamorous plumbing that every AI product eventually needs and almost none ship on day one — because day one is about the prompt, not the failure mode of the prompt.
Here's the one that stung. We had a whole "mastery" system — spaced repetition, progress bars, the works — and users' mastery scores just... never moved. Not broken-looking, just flat.
The cause: the only thing that triggered a quiz (and therefore a mastery write) was the user typing a hidden keyword like "quiz" or "check" into chat. Practice mode and Review mode — the two surfaces actually designed for this — had no write-back path at all. The UI was doing real work and throwing the result away.
The fix wasn't a clever algorithm, it was making the invisible contract visible: a shared masteryRules\
pathProgress\
engine, and an actual "Check my understanding" button instead of a magic word nobody was told about. The lesson generalizes past this one bug — if a core feedback loop depends on the user knowing an undocumented trigger, it doesn't exist.
The other pattern worth mentioning: for a while, every AI call that errored was caught and swallowed — console.error(e)\
and render null\
. From the user's side, that's just a blank screen with no explanation, indistinguishable from the app being broken. We replaced every one of those call sites with a shared error-state component that actually tells the user what happened and gives them a retry path. Small change, but it's the difference between "this app is broken" and "this app hit a snag" in the user's head — and those get very different reactions.
It's live at vararuchi.com if you want to poke at I'd genuinely rather have three people tell me what's broken than a hundred silent signups, so — try to break something, and tell me what you find.