# Resilient LLM Calls in TypeScript: Retries, Breakers, Fallbacks

> Source: <https://sourcefeed.dev/a/resilient-llm-calls-in-typescript-retries-breakers-fallbacks>
> Published: 2026-08-28 17:43:21+00:00

# Resilient LLM Calls in TypeScript: Retries, Breakers, Fallbacks

Wrap Claude API calls with Cockatiel retries and a circuit breaker that fails over to a second model.

[Priya Nair](https://sourcefeed.dev/u/priya_nair)

## 1. What you'll build

An `ask(prompt)`

function that calls Claude Opus 5 with exponential-backoff retries, trips a circuit breaker after repeated failures, and answers from Claude Sonnet 5 whenever the primary model is overloaded, rate-limited, unreachable, or gone. About 70 lines, built on [Cockatiel](https://github.com/connor4312/cockatiel) and the [Anthropic TypeScript SDK](https://platform.claude.com/docs/en/cli-sdks-libraries/sdks/typescript).

## 2. Prerequisites

[Node.js](https://nodejs.org/)24 LTS (verified on 24.20.0). Node runs`.ts`

files directly since 22.18, so there's no build step. Cockatiel 4 requires Node 22 or newer.`@anthropic-ai/sdk`

0.122.0 and`cockatiel`

4.0.0, the current releases as of this writing.`typescript`

7.0.2 and`@types/node`

, used only for`tsc --noEmit`

; Node strips the types at runtime.- An Anthropic API key exported as
`ANTHROPIC_API_KEY`

, with access to`claude-opus-5`

and`claude-sonnet-5`

. - Commands are for macOS/Linux. On Windows, set the environment variables with
`$env:NAME = "value"`

in PowerShell.

## 3. Set up the project

```
mkdir resilient-llm && cd resilient-llm
npm init -y
npm pkg set type=module
npm install @anthropic-ai/sdk cockatiel
npm install -D typescript @types/node
```

`type: module`

matters: Cockatiel 4 ships ESM only. Add a minimal `tsconfig.json`

so your editor and `tsc`

understand `.ts`

imports:

```
{
  "compilerOptions": {
    "target": "es2022",
    "module": "nodenext",
    "strict": true,
    "noEmit": true,
    "allowImportingTsExtensions": true,
    "skipLibCheck": true
  },
  "include": ["src"]
}
```

## 4. Define the retry and circuit-breaker policies

Create `src/resilient.ts`

. The first job is deciding which errors count as transient. The SDK throws typed subclasses of `Anthropic.APIError`

with a `status`

field, which makes this a one-liner:

``` python
import Anthropic from "@anthropic-ai/sdk";
import {
  BrokenCircuitError,
  ConsecutiveBreaker,
  ExponentialBackoff,
  circuitBreaker,
  fallback,
  handleWhen,
  retry,
  wrap,
} from "cockatiel";

export const PRIMARY_MODEL = process.env.PRIMARY_MODEL ?? "claude-opus-5";
export const FALLBACK_MODEL = process.env.FALLBACK_MODEL ?? "claude-sonnet-5";

// maxRetries: 0 hands retries to Cockatiel. The SDK default of 2 would
// multiply every Cockatiel attempt by three.
const client = new Anthropic({ maxRetries: 0, timeout: 60_000 });

// Worth retrying: network errors, 408/409/429, and any 5xx (529 = overloaded).
// 400/401/403/404 are our mistake or a dead model; retrying just burns time.
const isTransient = (err: unknown): boolean =>
  err instanceof Anthropic.APIConnectionError ||
  (err instanceof Anthropic.APIError &&
    (err.status === 408 ||
      err.status === 409 ||
      err.status === 429 ||
      (err.status ?? 0) >= 500));

// 3 retries after the first call, starting around 500ms and capped at 8s, with jitter.
export const retryPolicy = retry(handleWhen(isTransient), {
  maxAttempts: 3,
  backoff: new ExponentialBackoff({ initialDelay: 500, maxDelay: 8_000 }),
});

// Open after 5 consecutive transient failures; let one probe through after 30s.
export const breaker = circuitBreaker(handleWhen(isTransient), {
  halfOpenAfter: 30_000,
  breaker: new ConsecutiveBreaker(5),
});

// Retry outside, breaker inside. An open circuit throws BrokenCircuitError,
// which isTransient doesn't match, so the retry loop stops immediately.
const primaryPolicy = wrap(retryPolicy, breaker);

retryPolicy.onRetry(({ attempt, delay }) =>
  console.warn(`[retry] ${PRIMARY_MODEL} failed; retry ${attempt}/3 in ${Math.round(delay)}ms`),
);
breaker.onBreak(() => console.warn(`[breaker] open: fast-failing ${PRIMARY_MODEL} for 30s`));
breaker.onHalfOpen(() => console.warn(`[breaker] half-open: probing ${PRIMARY_MODEL}`));
breaker.onReset(() => console.warn(`[breaker] closed: ${PRIMARY_MODEL} healthy again`));
```

Two details: `maxAttempts`

counts retries after the initial call, so this policy makes up to four requests. And because the breaker sits inside the retry, every failed attempt feeds `ConsecutiveBreaker`

; one fully failed request is four of the five strikes.

## 5. Add the model fallback

Append this to `src/resilient.ts`

. The fallback policy is built per call because its factory can't see the wrapped function's arguments and needs the prompt. It holds no state, so that's free:

```
// Fall back when the primary is unhealthy or gone, not when the request itself is bad.
const shouldFallBack = (err: unknown): boolean =>
  isTransient(err) ||
  err instanceof BrokenCircuitError ||
  err instanceof Anthropic.NotFoundError;

export interface Answer {
  model: string;
  text: string;
}

async function callModel(model: string, prompt: string): Promise<Answer> {
  const message = await client.messages.create({
    model,
    max_tokens: 1024,
    messages: [{ role: "user", content: prompt }],
  });
  const text = message.content
    .filter((block) => block.type === "text")
    .map((block) => block.text)
    .join("");
  return { model: message.model, text };
}

export async function ask(prompt: string): Promise<Answer> {
  const withFallback = fallback(handleWhen(shouldFallBack), () => {
    console.warn(`[fallback] ${PRIMARY_MODEL} unavailable; answering with ${FALLBACK_MODEL}`);
    return retryPolicy.execute(() => callModel(FALLBACK_MODEL, prompt));
  });

  return await withFallback.execute(() =>
    primaryPolicy.execute(() => callModel(PRIMARY_MODEL, prompt)),
  );
}
```

`NotFoundError`

is on the fallback list on purpose: a retired or mistyped model ID returns 404, and that's exactly when traffic should move. `BadRequestError`

(400) is not on the list; a malformed request fails on the secondary too, so let it throw.

The fallback call reuses `retryPolicy`

but skips the breaker. The breaker tracks the primary's health, and sharing it would let a sick primary block a healthy secondary.

Now `src/index.ts`

. Node's type stripping requires the `.ts`

extension on relative imports:

``` js
import { ask } from "./resilient.ts";

const { model, text } = await ask(
  "In one sentence, what does a circuit breaker do in a distributed system?",
);
console.log(`[${model}] ${text}`);
```

## 6. Verify it works

Type-check, then run:

```
npx tsc
node src/index.ts
```

`tsc`

prints nothing on success. On a healthy day the primary answers and no policy logs fire:

```
[claude-opus-5] A circuit breaker stops sending requests to a failing service so the failure can't cascade, then periodically tests whether it has recovered.
```

Now force the fallback path by pointing the primary at a model that doesn't exist. The API returns a 404, which skips retries and goes straight to Sonnet:

``` bash
$ PRIMARY_MODEL=claude-nope-1 node src/index.ts
[fallback] claude-nope-1 unavailable; answering with claude-sonnet-5
[claude-sonnet-5] A circuit breaker detects repeated failures from a dependency and temporarily blocks calls to it, giving it time to recover instead of cascading the failure.
```

During a real 529 storm you'll see `[retry] ... retry 1/3 in 612ms`

lines before either the primary recovers or `[fallback]`

fires. If several requests fail back to back, `[breaker] open`

appears and later requests skip the primary for 30 seconds.

Node 24.1 through 24.19 print `ExperimentalWarning: Type Stripping is an experimental feature`

on startup. It's harmless; 24.20 dropped it.

## 7. Troubleshooting

** TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts"**
You're on Node 20 or an early 22.x. Type stripping shipped by default in 22.18 and 23.6. Upgrade to 24 LTS, or run the file with

`npx tsx src/index.ts`

on older versions.** Error [ERR_MODULE_NOT_FOUND]: Cannot find module '.../src/resilient' imported from .../src/index.ts**
The import is missing its extension. Node doesn't resolve

`./resilient`

to `./resilient.ts`

; write the `.ts`

explicitly. The `allowImportingTsExtensions`

flag in `tsconfig.json`

keeps `tsc`

from objecting.`Error: Could not resolve authentication method. Expected one of apiKey, authToken, credentials, config, or profile to be set.`

`ANTHROPIC_API_KEY`

isn't visible to the process. Run `export ANTHROPIC_API_KEY=sk-ant-...`

in the same shell, or prefix the command with it. The client reads the variable at construction time, so a key set after import won't be picked up.

** BadRequestError: 400 {"type":"error","error":{"type":"invalid_request_error", ...}}**
The request itself is invalid, so neither the retry nor the fallback policy handles it, by design. Read the

`message`

field; it names the offending parameter (for example a `max_tokens`

above the model's limit). Fix the request rather than widening `shouldFallBack`

.## 8. Next steps

Honor `retry-after`

on 429s. The SDK's built-in retries do this and Cockatiel's don't; swap `ExponentialBackoff`

for a `DelegateBackoff`

that reads the header off the failed error in the retry context. For production, replace `ConsecutiveBreaker`

with `SamplingBreaker`

so a 20% failure rate over 30 seconds opens the circuit instead of five unlucky requests in a row, and wire `onBreak`

/`onReset`

into your metrics. Cross-provider fallback is the same shape: point the fallback factory at a different SDK and keep the policies unchanged.

The Claude API also has a server-side fallback for a different failure: a request the model declines rather than an HTTP error. See [Refusals and fallback](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback) for the `fallbacks: "default"`

beta.

## Sources & further reading

-
[Cockatiel README (retry, circuitBreaker, fallback, wrap APIs)](https://github.com/connor4312/cockatiel)— github.com -
[Anthropic TypeScript SDK: errors, retries, timeouts](https://platform.claude.com/docs/en/cli-sdks-libraries/sdks/typescript)— platform.claude.com -
[Claude API errors](https://platform.claude.com/docs/en/api/errors)— platform.claude.com -
[Claude models overview (model IDs)](https://platform.claude.com/docs/en/models/overview)— platform.claude.com -
[Node.js: Modules: TypeScript (type stripping)](https://nodejs.org/api/typescript.html)— nodejs.org -
[Refusals and fallback](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback)— platform.claude.com

[Priya Nair](https://sourcefeed.dev/u/priya_nair)· AI & Developer Experience Writer

Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.

## Discussion 0

No comments yet

Be the first to weigh in.