Wrap Claude API calls with Cockatiel retries and a circuit breaker that fails over to a second model.
1. What you'll build #
An ask(prompt)
function that calls Claude Opus 5 with exponential-backoff retries, trips a circuit breaker after repeated failures, and answers from Claude Sonnet 5 whenever the primary model is overloaded, rate-limited, unreachable, or gone. About 70 lines, built on Cockatiel and the Anthropic TypeScript SDK.
2. Prerequisites #
Node.js24 LTS (verified on 24.20.0). Node runs.ts
files directly since 22.18, so there's no build step. Cockatiel 4 requires Node 22 or newer.@anthropic-ai/sdk
0.122.0 andcockatiel
4.0.0, the current releases as of this writing.typescript
7.0.2 and@types/node
, used only fortsc --noEmit
; Node strips the types at runtime.- An Anthropic API key exported as
ANTHROPIC_API_KEY
, with access toclaude-opus-5
andclaude-sonnet-5
. - Commands are for macOS/Linux. On Windows, set the environment variables with
$env:NAME = "value"
in PowerShell.
3. Set up the project #
mkdir resilient-llm && cd resilient-llm
npm init -y
npm pkg set type=module
npm install @anthropic-ai/sdk cockatiel
npm install -D typescript @types/node
type: module
matters: Cockatiel 4 ships ESM only. Add a minimal tsconfig.json
so your editor and tsc
understand .ts
imports:
{
"compilerOptions": {
"target": "es2022",
"module": "nodenext",
"strict": true,
"noEmit": true,
"allowImportingTsExtensions": true,
"skipLibCheck": true
},
"include": ["src"]
}
4. Define the retry and circuit-breaker policies #
Create src/resilient.ts
. The first job is deciding which errors count as transient. The SDK throws typed subclasses of Anthropic.APIError
with a status
field, which makes this a one-liner:
import Anthropic from "@anthropic-ai/sdk";
import {
BrokenCircuitError,
ConsecutiveBreaker,
ExponentialBackoff,
circuitBreaker,
fallback,
handleWhen,
retry,
wrap,
} from "cockatiel";
export const PRIMARY_MODEL = process.env.PRIMARY_MODEL ?? "claude-opus-5";
export const FALLBACK_MODEL = process.env.FALLBACK_MODEL ?? "claude-sonnet-5";
// maxRetries: 0 hands retries to Cockatiel. The SDK default of 2 would
// multiply every Cockatiel attempt by three.
const client = new Anthropic({ maxRetries: 0, timeout: 60_000 });
// Worth retrying: network errors, 408/409/429, and any 5xx (529 = overloaded).
// 400/401/403/404 are our mistake or a dead model; retrying just burns time.
const isTransient = (err: unknown): boolean =>
err instanceof Anthropic.APIConnectionError ||
(err instanceof Anthropic.APIError &&
(err.status === 408 ||
err.status === 409 ||
err.status === 429 ||
(err.status ?? 0) >= 500));
// 3 retries after the first call, starting around 500ms and capped at 8s, with jitter.
export const retryPolicy = retry(handleWhen(isTransient), {
maxAttempts: 3,
backoff: new ExponentialBackoff({ initialDelay: 500, maxDelay: 8_000 }),
});
// Open after 5 consecutive transient failures; let one probe through after 30s.
export const breaker = circuitBreaker(handleWhen(isTransient), {
halfOpenAfter: 30_000,
breaker: new ConsecutiveBreaker(5),
});
// Retry outside, breaker inside. An open circuit throws BrokenCircuitError,
// which isTransient doesn't match, so the retry loop stops immediately.
const primaryPolicy = wrap(retryPolicy, breaker);
retryPolicy.onRetry(({ attempt, delay }) =>
console.warn(`[retry] ${PRIMARY_MODEL} failed; retry ${attempt}/3 in ${Math.round(delay)}ms`),
);
breaker.onBreak(() => console.warn(`[breaker] open: fast-failing ${PRIMARY_MODEL} for 30s`));
breaker.onHalfOpen(() => console.warn(`[breaker] half-open: probing ${PRIMARY_MODEL}`));
breaker.onReset(() => console.warn(`[breaker] closed: ${PRIMARY_MODEL} healthy again`));
Two details: maxAttempts
counts retries after the initial call, so this policy makes up to four requests. And because the breaker sits inside the retry, every failed attempt feeds ConsecutiveBreaker
; one fully failed request is four of the five strikes.
5. Add the model fallback #
Append this to src/resilient.ts
. The fallback policy is built per call because its factory can't see the wrapped function's arguments and needs the prompt. It holds no state, so that's free:
// Fall back when the primary is unhealthy or gone, not when the request itself is bad.
const shouldFallBack = (err: unknown): boolean =>
isTransient(err) ||
err instanceof BrokenCircuitError ||
err instanceof Anthropic.NotFoundError;
export interface Answer {
model: string;
text: string;
}
async function callModel(model: string, prompt: string): Promise<Answer> {
const message = await client.messages.create({
model,
max_tokens: 1024,
messages: [{ role: "user", content: prompt }],
});
const text = message.content
.filter((block) => block.type === "text")
.map((block) => block.text)
.join("");
return { model: message.model, text };
}
export async function ask(prompt: string): Promise<Answer> {
const withFallback = fallback(handleWhen(shouldFallBack), () => {
console.warn(`[fallback] ${PRIMARY_MODEL} unavailable; answering with ${FALLBACK_MODEL}`);
return retryPolicy.execute(() => callModel(FALLBACK_MODEL, prompt));
});
return await withFallback.execute(() =>
primaryPolicy.execute(() => callModel(PRIMARY_MODEL, prompt)),
);
}
NotFoundError
is on the fallback list on purpose: a retired or mistyped model ID returns 404, and that's exactly when traffic should move. BadRequestError
(400) is not on the list; a malformed request fails on the secondary too, so let it throw.
The fallback call reuses retryPolicy
but skips the breaker. The breaker tracks the primary's health, and sharing it would let a sick primary block a healthy secondary.
Now src/index.ts
. Node's type stripping requires the .ts
extension on relative imports:
import { ask } from "./resilient.ts";
const { model, text } = await ask(
"In one sentence, what does a circuit breaker do in a distributed system?",
);
console.log(`[${model}] ${text}`);
6. Verify it works #
Type-check, then run:
npx tsc
node src/index.ts
tsc
prints nothing on success. On a healthy day the primary answers and no policy logs fire:
[claude-opus-5] A circuit breaker stops sending requests to a failing service so the failure can't cascade, then periodically tests whether it has recovered.
Now force the fallback path by pointing the primary at a model that doesn't exist. The API returns a 404, which skips retries and goes straight to Sonnet:
$ PRIMARY_MODEL=claude-nope-1 node src/index.ts
[fallback] claude-nope-1 unavailable; answering with claude-sonnet-5
[claude-sonnet-5] A circuit breaker detects repeated failures from a dependency and temporarily blocks calls to it, giving it time to recover instead of cascading the failure.
During a real 529 storm you'll see [retry] ... retry 1/3 in 612ms
lines before either the primary recovers or [fallback]
fires. If several requests fail back to back, [breaker] open
appears and later requests skip the primary for 30 seconds.
Node 24.1 through 24.19 print ExperimentalWarning: Type Stripping is an experimental feature
on startup. It's harmless; 24.20 dropped it.
7. Troubleshooting #
** TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts"** You're on Node 20 or an early 22.x. Type stripping shipped by default in 22.18 and 23.6. Upgrade to 24 LTS, or run the file with
npx tsx src/index.ts
on older versions.** Error [ERR_MODULE_NOT_FOUND]: Cannot find module '.../src/resilient' imported from .../src/index.ts** The import is missing its extension. Node doesn't resolve
./resilient
to ./resilient.ts
; write the .ts
explicitly. The allowImportingTsExtensions
flag in tsconfig.json
keeps tsc
from objecting.Error: Could not resolve authentication method. Expected one of apiKey, authToken, credentials, config, or profile to be set.
ANTHROPIC_API_KEY
isn't visible to the process. Run export ANTHROPIC_API_KEY=sk-ant-...
in the same shell, or prefix the command with it. The client reads the variable at construction time, so a key set after import won't be picked up.
** BadRequestError: 400 {"type":"error","error":{"type":"invalid_request_error", ...}}** The request itself is invalid, so neither the retry nor the fallback policy handles it, by design. Read the
message
field; it names the offending parameter (for example a max_tokens
above the model's limit). Fix the request rather than widening shouldFallBack
.## 8. Next steps
Honor retry-after
on 429s. The SDK's built-in retries do this and Cockatiel's don't; swap ExponentialBackoff
for a DelegateBackoff
that reads the header off the failed error in the retry context. For production, replace ConsecutiveBreaker
with SamplingBreaker
so a 20% failure rate over 30 seconds opens the circuit instead of five unlucky requests in a row, and wire onBreak
/onReset
into your metrics. Cross-provider fallback is the same shape: point the fallback factory at a different SDK and keep the policies unchanged.
The Claude API also has a server-side fallback for a different failure: a request the model declines rather than an HTTP error. See Refusals and fallback for the fallbacks: "default"
beta.
Sources & further reading #
Cockatiel README (retry, circuitBreaker, fallback, wrap APIs)— github.com - Anthropic TypeScript SDK: errors, retries, timeouts— platform.claude.com - Claude API errors— platform.claude.com - Claude models overview (model IDs)— platform.claude.com - Node.js: Modules: TypeScript (type stripping)— nodejs.org - Refusals and fallback— platform.claude.com
Priya Nair· AI & Developer Experience Writer
Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.
Discussion 0 #
No comments yet
Be the first to weigh in.