Using Mastra Agents With a Custom OpenAI-Compatible API: the url Model Config, the Silent 5-Step Limit and Two Retry Layers That Multiply

Developer Aman Kumar tested Mastra's custom OpenAI-compatible model configuration against a local mock server and documented several non-obvious defaults: the model id prefix before the first slash is stripped and sent as the model name, the url field must include /v1 since Mastra only appends /chat/completions, and apiKey does not fall back to environment variables, causing unauthenticated 401 requests. He also found that agents silently cap tool-calling loops at five steps and that Mastra's retry layer can compound with the provider's own retries.

I'm Aman Kumar. Mastra is a TypeScript framework for agents and workflows, and most of its examples use model strings like "openai/gpt-4o" that resolve through a built-in provider registry. That's convenient until you want the same agent to talk to something that isn't in the registry: a self-hosted vLLM or LiteLLM server, Ollama's /v1 endpoint, or a hosted OpenAI-compatible gateway. Mastra handles this with a small config object instead of a separate provider package. It works, but a few defaults surprised me, so I tested them with @mastra/core 1.78.0 on Node 20.19.2 against a local mock server that logs every request body and can return tool calls, 401, 429 with Retry-After , and 503 on demand. The examples point at APIClaw, an OpenAI-compatible gateway I build, so weigh my opinions accordingly. Its base URL is https://apiclaw.biz/v1 ; I use YOUR MODEL ID below, so take real IDs from /v1/models or your provider's model list. npm i @mastra/core zod js import { Agent } from "@mastra/core/agent"; const agent = new Agent { id: "support", name: "support", instructions: "You answer in two sentences.", model: { id: "apiclaw/YOUR MODEL ID", url: "https://apiclaw.biz/v1", apiKey: process.env.APICLAW API KEY, }, } ; const res = await agent.generate "What does HTTP 429 mean?" ; console.log res.text, res.usage ; The model field accepts a registry string, an AI SDK model instance, or this object. There is also a { providerId, modelId, url, apiKey } form. In my tests it produced exactly the same request. Here is what the mock received: POST /v1/chat/completions {"model": "YOUR MODEL ID", "messages": {"role": "system", "content": "You answer in two sentences."}, {"role": "user", "content": "What does HTTP 429 mean?"} } The user agent was ai-sdk/openai-compatible , so under the hood this is the Vercel AI SDK's OpenAI-compatible provider. There was no GET /v1/models call and no extra parameters. id and url The part before the first slash is a label, not a lookup. apiclaw/ gets stripped and only the rest is sent as model . Only the first slash counts, so id: "apiclaw/deepseek/deepseek-chat" sends "model": "deepseek/deepseek-chat" . That's what you want for gateways whose model IDs contain slashes, but it means you can't drop the prefix. Without url , Mastra tries the prefix against its registry, and an unknown one fails with No API URL found for apiclaw/YOUR MODEL ID . url must include /v1 . Mastra appends /chat/completions to whatever you pass, nothing more. With url: "http://127.0.0.1:8766" the request went to /chat/completions , which most servers answer with a 404. A trailing slash .../v1/ was harmless: the request still went to /v1/chat/completions . apiKey doesn't fall back to an environment variable. When I left it out and set APICLAW API KEY in the environment, the request went out with no Authorization header at all and the server returned 401. For a custom provider, read the variable yourself, as in the example above. api: "responses" switches endpoints. With that flag the agent posted to /v1/responses with an input array instead of messages . Many OpenAI-compatible servers only implement Chat Completions, so leave it off unless your server documents the Responses API. My mock returned 404 and the agent threw AI APICallError: Unknown path /v1/responses . Custom headers work as you'd expect. headers: { "X-Team": "docs" } showed up on every request. Tools are where I lost the most time. A normal tool call looked fine: the first request carried tools and "tool choice": "auto" , the model returned a tool call, Mastra ran my execute function and sent a second request with four messages system, user, assistant tool call, tool result . That's two steps. Then I had the mock keep requesting the tool forever, which is what a weaker model does when it can't decide it's finished: js import { createTool } from "@mastra/core/tools"; import { z } from "zod"; const weather = createTool { id: "get weather", description: "Current weather for a city", inputSchema: z.object { city: z.string } , execute: async { city } = { city, tempC: 31 } , } ; const agent = new Agent { / ...same model... / tools: { weather } } ; const res = await agent.generate "Weather in Pune?" ; console.log JSON.stringify res.text , res.steps.length, res.finishReason ; // "" 5 tool-calls The agent made five requests, ran the tool five times, and then returned normally. text was an empty string and finishReason was "tool-calls" . It didn't throw an error or log a warning. If your code only reads res.text , the user just gets a blank answer. Two fixes: js // raise or lower the budget per call const res = await agent.generate prompt, { maxSteps: 3 } ; // and treat a run that ended on a tool call as unfinished if res.finishReason === "tool-calls" || res.text { // retry with a stronger model, or return a clear error } With maxSteps: 3 I got exactly three requests and three tool executions. Keep in mind that each step resends the whole conversation, so the fifth request carried ten messages. On per-token pricing that's where the cost goes. On a request-capped plan, one stuck question uses five requests. This was the most surprising result. I pointed an agent at a model that always returned 503: | Configuration | Requests sent | Time to fail | |---|---|---| | Defaults | 3 | 6 s | | Agent maxRetries: 0 | 3 | 6 s | | modelSettings: { maxRetries: 0 } | 3 | 6 s | | modelSettings: { maxRetries: 5 } | 18 | 99 s | | errorProcessorDefaults: false on the agent | 1 | 0.1 s | | errorProcessors: on the generate call | 1 | 0 s | The first layer is the model-call retry from the AI SDK, set by modelSettings.maxRetries or maxRetries on an entry in a model array . It's 0 by default in Mastra, it backs off exponentially 1, 2, 4, 8 and 16 seconds in my log , and it honoured Retry-After: 2 on a 429. The second layer is Mastra's default error processors. They retry a failed generation twice, about three seconds apart, whatever maxRetries says. With both layers on, each of the three outer attempts ran its own six inner attempts, so one bad upstream turned into 18 requests and a 99-second wait. A 401 failed at once, with no retry. What I settled on is one layer, set explicitly: js const agent = new Agent { // ... errorProcessorDefaults: false, // turn off the outer retry } ; await agent.generate prompt, { modelSettings: { maxRetries: 2 } } ; // 3 attempts, honours Retry-After You can also keep the defaults and leave maxRetries at 0. Just don't turn up both, and if you have your own retry around generate , that makes a third layer. model also accepts an array, and Mastra moves to the next entry when one fails: model: { model: { id: "apiclaw/YOUR PRIMARY MODEL", url, apiKey }, maxRetries: 1 }, { model: { id: "apiclaw/YOUR BACKUP MODEL", url, apiKey }, maxRetries: 0 }, , That worked, but with the default error processors still on, the primary got six requests two inner attempts times three outer before the backup got its first one, and the whole call took 9 seconds. If you rely on fallbacks, turn off the outer retry so the switch happens after one or two failures. agent.stream streamed text fine, but the request had no stream options , so a server that follows OpenAI's behaviour sends no usage chunk, and await stream.usage came back with empty token counts. If you log tokens, build the model with the AI SDK provider directly and pass that instance to Mastra: js import { createOpenAICompatible } from "@ai-sdk/openai-compatible"; const apiclaw = createOpenAICompatible { name: "apiclaw", baseURL: "https://apiclaw.biz/v1", apiKey: process.env.APICLAW API KEY, includeUsage: true, } ; const agent = new Agent { id: "a", name: "a", instructions: "...", model: apiclaw "YOUR MODEL ID" } ; With that, the request included "stream options": {"include usage": true} and usage reported input, output and total tokens. generate reported usage either way. { id: "label/YOUR MODEL ID", url, apiKey } . Everything after the first slash is sent as the model name. /v1 in apiKey explicitly. api unset unless your server implements /v1/responses . finishReason after tool runs, and set maxSteps on purpose. errorProcessorDefaults: false plus modelSettings.maxRetries , or the defaults with maxRetries at 0. createOpenAICompatible with includeUsage: true . Disclosure: I build APIClaw https://apiclaw.biz , the OpenAI-compatible gateway in these examples flat monthly plans; crypto on the site, other payment methods through support . Everything above applies to any compatible endpoint.