# Using Mastra Agents With a Custom OpenAI-Compatible API: the url Model Config, the Silent 5-Step Limit and Two Retry Layers That Multiply

> Source: <https://dev.to/amankumar_apiclaw/using-mastra-agents-with-a-custom-openai-compatible-api-the-url-model-config-the-silent-5-step-34oc>
> Published: 2026-10-18 09:38:03+00:00

I'm Aman Kumar. Mastra is a TypeScript framework for agents and workflows, and most of its examples use model strings like `"openai/gpt-4o"` that resolve through a built-in provider registry. That's convenient until you want the same agent to talk to something that isn't in the registry: a self-hosted vLLM or LiteLLM server, Ollama's `/v1` endpoint, or a hosted OpenAI-compatible gateway.

Mastra handles this with a small config object instead of a separate provider package. It works, but a few defaults surprised me, so I tested them with `@mastra/core` 1.78.0 on Node 20.19.2 against a local mock server that logs every request body and can return tool calls, 401, 429 with `Retry-After`, and 503 on demand. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is `https://apiclaw.biz/v1`; I use `YOUR_MODEL_ID` below, so take real IDs from `/v1/models` or your provider's model list.

```
npm i @mastra/core zod
js
import { Agent } from "@mastra/core/agent";

const agent = new Agent({
  id: "support",
  name: "support",
  instructions: "You answer in two sentences.",
  model: {
    id: "apiclaw/YOUR_MODEL_ID",
    url: "https://apiclaw.biz/v1",
    apiKey: process.env.APICLAW_API_KEY,
  },
});

const res = await agent.generate("What does HTTP 429 mean?");
console.log(res.text, res.usage);
```

The `model` field accepts a registry string, an AI SDK model instance, or this object. There is also a `{ providerId, modelId, url, apiKey }` form. In my tests it produced exactly the same request.

Here is what the mock received:

```
POST /v1/chat/completions
{"model": "YOUR_MODEL_ID",
 "messages": [{"role": "system", "content": "You answer in two sentences."},
              {"role": "user", "content": "What does HTTP 429 mean?"}]}
```

The user agent was `ai-sdk/openai-compatible`, so under the hood this is the Vercel AI SDK's OpenAI-compatible provider. There was no `GET /v1/models` call and no extra parameters.

`id` and `url`
**The part before the first slash is a label, not a lookup.** `apiclaw/` gets stripped and only the rest is sent as `model`. Only the first slash counts, so `id: "apiclaw/deepseek/deepseek-chat"` sends `"model": "deepseek/deepseek-chat"`. That's what you want for gateways whose model IDs contain slashes, but it means you can't drop the prefix. Without `url`, Mastra tries the prefix against its registry, and an unknown one fails with `No API URL found for apiclaw/YOUR_MODEL_ID`.

**`url` must include `/v1`.** Mastra appends `/chat/completions` to whatever you pass, nothing more. With `url: "http://127.0.0.1:8766"` the request went to `/chat/completions`, which most servers answer with a 404. A trailing slash (`.../v1/`) was harmless: the request still went to `/v1/chat/completions`.

**`apiKey` doesn't fall back to an environment variable.** When I left it out and set `APICLAW_API_KEY` in the environment, the request went out with no `Authorization` header at all and the server returned 401. For a custom provider, read the variable yourself, as in the example above.

**`api: "responses"` switches endpoints.** With that flag the agent posted to `/v1/responses` with an `input` array instead of `messages`. Many OpenAI-compatible servers only implement Chat Completions, so leave it off unless your server documents the Responses API. My mock returned 404 and the agent threw `AI_APICallError: Unknown path /v1/responses`.

Custom headers work as you'd expect. `headers: { "X-Team": "docs" }` showed up on every request.

Tools are where I lost the most time. A normal tool call looked fine: the first request carried `tools` and `"tool_choice": "auto"`, the model returned a tool call, Mastra ran my `execute` function and sent a second request with four messages (system, user, assistant tool call, tool result). That's two steps.

Then I had the mock keep requesting the tool forever, which is what a weaker model does when it can't decide it's finished:

``` js
import { createTool } from "@mastra/core/tools";
import { z } from "zod";

const weather = createTool({
  id: "get_weather",
  description: "Current weather for a city",
  inputSchema: z.object({ city: z.string() }),
  execute: async ({ city }) => ({ city, tempC: 31 }),
});

const agent = new Agent({ /* ...same model... */ tools: { weather } });
const res = await agent.generate("Weather in Pune?");
console.log(JSON.stringify(res.text), res.steps.length, res.finishReason);
// "" 5 tool-calls
```

The agent made five requests, ran the tool five times, and then returned normally. `text` was an empty string and `finishReason` was `"tool-calls"`. It didn't throw an error or log a warning. If your code only reads `res.text`, the user just gets a blank answer.

Two fixes:

``` js
// raise or lower the budget per call
const res = await agent.generate(prompt, { maxSteps: 3 });

// and treat a run that ended on a tool call as unfinished
if (res.finishReason === "tool-calls" || !res.text) {
  // retry with a stronger model, or return a clear error
}
```

With `maxSteps: 3` I got exactly three requests and three tool executions. Keep in mind that each step resends the whole conversation, so the fifth request carried ten messages. On per-token pricing that's where the cost goes. On a request-capped plan, one stuck question uses five requests.

This was the most surprising result. I pointed an agent at a model that always returned 503:

| Configuration | Requests sent | Time to fail | 
|---|---|---|
| Defaults | 3 | 6 s | 
| Agent `maxRetries: 0` | 3 | 6 s | 
| `modelSettings: { maxRetries: 0 }` | 3 | 6 s | 
| `modelSettings: { maxRetries: 5 }` | 18 | 99 s | 
| `errorProcessorDefaults: false` on the agent | 1 | 0.1 s | 
| `errorProcessors: []` on the`generate` call | 1 | 0 s | 

The first layer is the model-call retry from the AI SDK, set by `modelSettings.maxRetries` (or `maxRetries` on an entry in a model array). It's 0 by default in Mastra, it backs off exponentially (1, 2, 4, 8 and 16 seconds in my log), and it honoured `Retry-After: 2` on a 429.

The second layer is Mastra's default error processors. They retry a failed generation twice, about three seconds apart, whatever `maxRetries` says. With both layers on, each of the three outer attempts ran its own six inner attempts, so one bad upstream turned into 18 requests and a 99-second wait. A 401 failed at once, with no retry.

What I settled on is one layer, set explicitly:

``` js
const agent = new Agent({
  // ...
  errorProcessorDefaults: false, // turn off the outer retry
});

await agent.generate(prompt, { modelSettings: { maxRetries: 2 } }); // 3 attempts, honours Retry-After
```

You can also keep the defaults and leave `maxRetries` at 0. Just don't turn up both, and if you have your own retry around `generate()`, that makes a third layer.

`model` also accepts an array, and Mastra moves to the next entry when one fails:

```
model: [
  { model: { id: "apiclaw/YOUR_PRIMARY_MODEL", url, apiKey }, maxRetries: 1 },
  { model: { id: "apiclaw/YOUR_BACKUP_MODEL", url, apiKey }, maxRetries: 0 },
],
```

That worked, but with the default error processors still on, the primary got six requests (two inner attempts times three outer) before the backup got its first one, and the whole call took 9 seconds. If you rely on fallbacks, turn off the outer retry so the switch happens after one or two failures.

`agent.stream()` streamed text fine, but the request had no `stream_options`, so a server that follows OpenAI's behaviour sends no usage chunk, and `await stream.usage` came back with empty token counts. If you log tokens, build the model with the AI SDK provider directly and pass that instance to Mastra:

``` js
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";

const apiclaw = createOpenAICompatible({
  name: "apiclaw",
  baseURL: "https://apiclaw.biz/v1",
  apiKey: process.env.APICLAW_API_KEY,
  includeUsage: true,
});

const agent = new Agent({ id: "a", name: "a", instructions: "...", model: apiclaw("YOUR_MODEL_ID") });
```

With that, the request included `"stream_options": {"include_usage": true}` and `usage` reported input, output and total tokens. `generate()` reported usage either way.

`{ id: "label/YOUR_MODEL_ID", url, apiKey }`. Everything after the first slash is sent as the model name.`/v1` in `apiKey` explicitly.`api` unset unless your server implements `/v1/responses`.` finishReason` after tool runs, and set `maxSteps` on purpose.`errorProcessorDefaults: false` plus `modelSettings.maxRetries`, or the defaults with `maxRetries` at 0.`createOpenAICompatible` with `includeUsage: true`.
*Disclosure: I build [APIClaw](https://apiclaw.biz), the OpenAI-compatible gateway in these examples (flat monthly plans; crypto on the site, other payment methods through support). Everything above applies to any compatible endpoint.*