Klyro's diagnostic core runs on exactly two LLM calls per run, not one, and the split isn't cosmetic. An Analyst looks at the load-test results and figures out what's actually wrong. An Investigator takes that diagnosis and writes the code fix. Two agents, two prompts, two different jobs, because asking a single call to both spot a performance regression and safely edit source code tends to blur the line between noticing a problem and being trusted to touch a codebase. The Analyst runs on each provider's smaller, faster model since diagnosis is closer to pattern matching; the Investigator runs on the larger one, because proposing an actual patch deserves the model with more room to reason.
The provider story behind this pipeline isn't a straight line. Groq was the original plan. Then a Mistral key showed up with limits that looked better on paper, and the pipeline moved over. Then Mistral's account came back rate-limited to zero requests per minute, the kind of thing no dashboard warns you about in advance and no amount of retrying fixes on its own. The response wasn't to go looking for a third single provider to bet on. It was to stop betting on one provider at all: the pool now holds two Mistral keys and two Groq keys at once, spanning both providers, so a 429 against one provider's entire account still has somewhere else to go.
The mechanics live in LLMProvider, a small class that treats the pool as an ordered list of { apiKey, baseUrl, model } entries. Hit a 429 on whichever entry is active, and it rotates to the next one and retries the exact same prompt immediately, cycling through the whole pool before it ever falls back to a plain backoff-and-retry. That's deliberately narrow: the pool and its order are configured up front in SSM Parameter Store, not improvised mid-run. The one thing rotation never touches is schema repair. If a model returns JSON that fails validation, the retry goes back to the same pool entry with the error appended to the prompt, because that's the model getting something wrong, not the provider going down, and those two failure modes call for different fixes.
Letting an LLM propose code changes only works if the blast radius is small and provable. The Investigator can write to exactly three files: demo-app/src/orders.js, demo-app/src/logger.js, and demo-app/config/logger.json. Any other path gets rejected before a single build step runs, no exceptions. Every patch also has to carry the original_sha256 of the file it targets, checked against the live file at apply time. If the hash doesn't match, the patch doesn't land. That single check rules out an entire class of problems: a stale patch written against a version of the file that's already moved on, or a patch quietly overwriting a change no one asked for.
The pool rotation and the allowlist look like they're pulling in opposite directions, one bending to keep the pipeline running, the other refusing to bend at all. They're actually the same design instinct applied twice. Decide in advance exactly where flexibility is safe, wire it in deliberately, and then hold the line everywhere else without exception. A rate limit shouldn't stall a demo run. A model hallucinating a change to a file it was never handed shouldn't reach production, even in a sandbox. Neither rule got relaxed to solve the other's problem.