Resilient LLM Calls in TypeScript: Retries, Breakers, Fallbacks

Priya Nair published a tutorial showing how to wrap Anthropic's Claude API calls in TypeScript with Cockatiel retries and a circuit breaker that fails over from Claude Opus 5 to Claude Sonnet 5, using about 70 lines of code. The implementation uses Cockatiel 4.0.0 and @anthropic-ai/sdk 0.122.0, with retry policy set to 3 attempts after the first call, exponential backoff starting at 500ms capped at 8s, and a circuit breaker that opens after 5 consecutive transient failures and allows a probe after 30 seconds.

Resilient LLM Calls in TypeScript: Retries, Breakers, Fallbacks Wrap Claude API calls with Cockatiel retries and a circuit breaker that fails over to a second model. Priya Nair https://sourcefeed.dev/u/priya nair 1. What you'll build An ask prompt function that calls Claude Opus 5 with exponential-backoff retries, trips a circuit breaker after repeated failures, and answers from Claude Sonnet 5 whenever the primary model is overloaded, rate-limited, unreachable, or gone. About 70 lines, built on Cockatiel https://github.com/connor4312/cockatiel and the Anthropic TypeScript SDK https://platform.claude.com/docs/en/cli-sdks-libraries/sdks/typescript . 2. Prerequisites Node.js https://nodejs.org/ 24 LTS verified on 24.20.0 . Node runs .ts files directly since 22.18, so there's no build step. Cockatiel 4 requires Node 22 or newer. @anthropic-ai/sdk 0.122.0 and cockatiel 4.0.0, the current releases as of this writing. typescript 7.0.2 and @types/node , used only for tsc --noEmit ; Node strips the types at runtime.- An Anthropic API key exported as ANTHROPIC API KEY , with access to claude-opus-5 and claude-sonnet-5 . - Commands are for macOS/Linux. On Windows, set the environment variables with $env:NAME = "value" in PowerShell. 3. Set up the project mkdir resilient-llm && cd resilient-llm npm init -y npm pkg set type=module npm install @anthropic-ai/sdk cockatiel npm install -D typescript @types/node type: module matters: Cockatiel 4 ships ESM only. Add a minimal tsconfig.json so your editor and tsc understand .ts imports: { "compilerOptions": { "target": "es2022", "module": "nodenext", "strict": true, "noEmit": true, "allowImportingTsExtensions": true, "skipLibCheck": true }, "include": "src" } 4. Define the retry and circuit-breaker policies Create src/resilient.ts . The first job is deciding which errors count as transient. The SDK throws typed subclasses of Anthropic.APIError with a status field, which makes this a one-liner: python import Anthropic from "@anthropic-ai/sdk"; import { BrokenCircuitError, ConsecutiveBreaker, ExponentialBackoff, circuitBreaker, fallback, handleWhen, retry, wrap, } from "cockatiel"; export const PRIMARY MODEL = process.env.PRIMARY MODEL ?? "claude-opus-5"; export const FALLBACK MODEL = process.env.FALLBACK MODEL ?? "claude-sonnet-5"; // maxRetries: 0 hands retries to Cockatiel. The SDK default of 2 would // multiply every Cockatiel attempt by three. const client = new Anthropic { maxRetries: 0, timeout: 60 000 } ; // Worth retrying: network errors, 408/409/429, and any 5xx 529 = overloaded . // 400/401/403/404 are our mistake or a dead model; retrying just burns time. const isTransient = err: unknown : boolean = err instanceof Anthropic.APIConnectionError || err instanceof Anthropic.APIError && err.status === 408 || err.status === 409 || err.status === 429 || err.status ?? 0 = 500 ; // 3 retries after the first call, starting around 500ms and capped at 8s, with jitter. export const retryPolicy = retry handleWhen isTransient , { maxAttempts: 3, backoff: new ExponentialBackoff { initialDelay: 500, maxDelay: 8 000 } , } ; // Open after 5 consecutive transient failures; let one probe through after 30s. export const breaker = circuitBreaker handleWhen isTransient , { halfOpenAfter: 30 000, breaker: new ConsecutiveBreaker 5 , } ; // Retry outside, breaker inside. An open circuit throws BrokenCircuitError, // which isTransient doesn't match, so the retry loop stops immediately. const primaryPolicy = wrap retryPolicy, breaker ; retryPolicy.onRetry { attempt, delay } = console.warn retry ${PRIMARY MODEL} failed; retry ${attempt}/3 in ${Math.round delay }ms , ; breaker.onBreak = console.warn breaker open: fast-failing ${PRIMARY MODEL} for 30s ; breaker.onHalfOpen = console.warn breaker half-open: probing ${PRIMARY MODEL} ; breaker.onReset = console.warn breaker closed: ${PRIMARY MODEL} healthy again ; Two details: maxAttempts counts retries after the initial call, so this policy makes up to four requests. And because the breaker sits inside the retry, every failed attempt feeds ConsecutiveBreaker ; one fully failed request is four of the five strikes. 5. Add the model fallback Append this to src/resilient.ts . The fallback policy is built per call because its factory can't see the wrapped function's arguments and needs the prompt. It holds no state, so that's free: // Fall back when the primary is unhealthy or gone, not when the request itself is bad. const shouldFallBack = err: unknown : boolean = isTransient err || err instanceof BrokenCircuitError || err instanceof Anthropic.NotFoundError; export interface Answer { model: string; text: string; } async function callModel model: string, prompt: string : Promise<Answer { const message = await client.messages.create { model, max tokens: 1024, messages: { role: "user", content: prompt } , } ; const text = message.content .filter block = block.type === "text" .map block = block.text .join "" ; return { model: message.model, text }; } export async function ask prompt: string : Promise<Answer { const withFallback = fallback handleWhen shouldFallBack , = { console.warn fallback ${PRIMARY MODEL} unavailable; answering with ${FALLBACK MODEL} ; return retryPolicy.execute = callModel FALLBACK MODEL, prompt ; } ; return await withFallback.execute = primaryPolicy.execute = callModel PRIMARY MODEL, prompt , ; } NotFoundError is on the fallback list on purpose: a retired or mistyped model ID returns 404, and that's exactly when traffic should move. BadRequestError 400 is not on the list; a malformed request fails on the secondary too, so let it throw. The fallback call reuses retryPolicy but skips the breaker. The breaker tracks the primary's health, and sharing it would let a sick primary block a healthy secondary. Now src/index.ts . Node's type stripping requires the .ts extension on relative imports: js import { ask } from "./resilient.ts"; const { model, text } = await ask "In one sentence, what does a circuit breaker do in a distributed system?", ; console.log ${model} ${text} ; 6. Verify it works Type-check, then run: npx tsc node src/index.ts tsc prints nothing on success. On a healthy day the primary answers and no policy logs fire: claude-opus-5 A circuit breaker stops sending requests to a failing service so the failure can't cascade, then periodically tests whether it has recovered. Now force the fallback path by pointing the primary at a model that doesn't exist. The API returns a 404, which skips retries and goes straight to Sonnet: bash $ PRIMARY MODEL=claude-nope-1 node src/index.ts fallback claude-nope-1 unavailable; answering with claude-sonnet-5 claude-sonnet-5 A circuit breaker detects repeated failures from a dependency and temporarily blocks calls to it, giving it time to recover instead of cascading the failure. During a real 529 storm you'll see retry ... retry 1/3 in 612ms lines before either the primary recovers or fallback fires. If several requests fail back to back, breaker open appears and later requests skip the primary for 30 seconds. Node 24.1 through 24.19 print ExperimentalWarning: Type Stripping is an experimental feature on startup. It's harmless; 24.20 dropped it. 7. Troubleshooting TypeError ERR UNKNOWN FILE EXTENSION : Unknown file extension ".ts" You're on Node 20 or an early 22.x. Type stripping shipped by default in 22.18 and 23.6. Upgrade to 24 LTS, or run the file with npx tsx src/index.ts on older versions. Error ERR MODULE NOT FOUND : Cannot find module '.../src/resilient' imported from .../src/index.ts The import is missing its extension. Node doesn't resolve ./resilient to ./resilient.ts ; write the .ts explicitly. The allowImportingTsExtensions flag in tsconfig.json keeps tsc from objecting. Error: Could not resolve authentication method. Expected one of apiKey, authToken, credentials, config, or profile to be set. ANTHROPIC API KEY isn't visible to the process. Run export ANTHROPIC API KEY=sk-ant-... in the same shell, or prefix the command with it. The client reads the variable at construction time, so a key set after import won't be picked up. BadRequestError: 400 {"type":"error","error":{"type":"invalid request error", ...}} The request itself is invalid, so neither the retry nor the fallback policy handles it, by design. Read the message field; it names the offending parameter for example a max tokens above the model's limit . Fix the request rather than widening shouldFallBack . 8. Next steps Honor retry-after on 429s. The SDK's built-in retries do this and Cockatiel's don't; swap ExponentialBackoff for a DelegateBackoff that reads the header off the failed error in the retry context. For production, replace ConsecutiveBreaker with SamplingBreaker so a 20% failure rate over 30 seconds opens the circuit instead of five unlucky requests in a row, and wire onBreak / onReset into your metrics. Cross-provider fallback is the same shape: point the fallback factory at a different SDK and keep the policies unchanged. The Claude API also has a server-side fallback for a different failure: a request the model declines rather than an HTTP error. See Refusals and fallback https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback for the fallbacks: "default" beta. Sources & further reading - Cockatiel README retry, circuitBreaker, fallback, wrap APIs https://github.com/connor4312/cockatiel — github.com - Anthropic TypeScript SDK: errors, retries, timeouts https://platform.claude.com/docs/en/cli-sdks-libraries/sdks/typescript — platform.claude.com - Claude API errors https://platform.claude.com/docs/en/api/errors — platform.claude.com - Claude models overview model IDs https://platform.claude.com/docs/en/models/overview — platform.claude.com - Node.js: Modules: TypeScript type stripping https://nodejs.org/api/typescript.html — nodejs.org - Refusals and fallback https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback — platform.claude.com Priya Nair https://sourcefeed.dev/u/priya nair · AI & Developer Experience Writer Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to. Discussion 0 No comments yet Be the first to weigh in.