Schema Drift Reaches the Tool Definition
Context layer series Everyone who sells a context layer talks about freshness. Fresh rows, streaming…
Avoma ships an MCP server now, like most SaaS vendors do. We wanted to know how the Avoma MCP server holds up against a task-specific server on the same data, so we ran it through the same setup we used for Google Ads and HubSpot. Both servers pointed at one Avoma workspace. One agent model asked both of them the same ten questions, and a judge checked every answer against ground truth.
One caveat before the numbers. The Nexla server in this test was built for call analysis and nothing else, so that is the part of the comparison that is fair, and it is the part this post is about.
We used Nexla’s own production Avoma workspace, which has real dialer calls and real meetings in it. Each of the ten tasks ran once against each server in its own isolated session, with claude-opus-5 acting as the agent and then as the judge. We built ground truth for the call tasks by pulling the raw call records in weekly slices and counting them by hand.
The four call analysis tasks are the kind of thing a sales ops lead asks every week: how many calls happened in a month, how many were outbound versus inbound, and which reps made them. Avoma’s server could not answer any of them.
T7
How many dialer calls were logged in October 2025?T8
How many dialer calls were logged in November 2025?T9
Split November calls into outbound and inbound.T10
Which reps made calls across October and November, and who was most active?The Nexla server connects over streamable HTTP with a bearer token. You paste a URL and a service key into your config and you are done. There is no approval step and no token to refresh later.
The Avoma MCP server runs through mcp-remote with OAuth, which works fine until the token expires. Ours expired partway through the run. Five cells failed with
401 Invalid API key
and we had to re-authorise and run them again. That cost us an hour in a benchmark. If this had been an unattended agent, it would have quietly stopped working and nobody would have noticed until someone asked why the numbers were stale.mcp-remote
stdio proxy with OAuth401
When you ask Nexla’s server how many dialer calls happened in November 2025, it makes one tool call, reads the modeled call table, and answers 398. When you ask for the direction split, it slices the month into weekly windows and comes back with 394 outbound and 4 inbound. Both answers match ground truth exactly.
The Avoma MCP server fails on the same questions no matter which route the agent takes. The analytics endpoint only covers the last 30 days, so anything from 2025 is out of reach. The meetings endpoint should work, but it crashes on the call records themselves:
Tool execution failed: Failed to list meetings:
2 validation errors for CallDetails
frm
Input should be a valid string [type=string_type, input_value=None]
to
Input should be a valid string [type=string_type, input_value=None]
The client declares CallDetails.frm
and .to
as required strings, and dialer calls store both as null. So every page that contains a dialer call throws before the agent sees any data. Avoma scored 0.00 on all three measured call tasks. On the October count 14 of its 23 tool calls errored, on November 6 of 9, and on the direction split 5 of 8.
| Task | Ground truth | Nexla MCP | Avoma MCP |
|---|---|---|---|
| Nov 2025 call count | 398 | 398 ✓ | No answer 6 of 9 calls errored |
| Nov direction split | 394 out / 4 in | 394 / 4 ✓ | No answer 5 of 8 calls errored |
To its credit, the agent on Avoma’s side refused to guess. But that is the whole point. The vendor’s own server cannot read an entire product surface, and a task-specific server that models the call records directly can.
Across all ten tasks Nexla averaged 4.0 tool calls per task, Avoma 5.9. Nexla finished in 95 seconds per task, Avoma in 102.5. Nexla logged one errored tool call in the whole run and Avoma logged 25. Nexla’s tool schema is also a third the size, and that matters more than it sounds because the schema gets resent on every agent turn.
Two smaller Avoma limits turned up along the way. page_size
tops out at 10, which the tool description does not mention, so reading a month with 1,300 meetings takes about 130 calls. And list_team_usage_metrics
only accepts a 30 day window, so you cannot use it for anything historical.
The Avoma MCP server, like any vendor MCP server, is a wrapper around the vendor’s API, and it carries every edge of that API with it. When one of those edges is a schema bug, a whole class of questions goes dark and the agent has no way around it. A task-specific server built on modeled data does not inherit that problem. It answers the question you built it for in one call, using a credential you can revoke in one click.
If you are picking a server for agents that need call data, start by asking which one can read the calls at all. Today only one of these two can.
Want the wider pattern? Read the Google Ads benchmark, the HubSpot benchmark and the BigQuery benchmark, which also walks through how we run these tests. We also wrote about why the future is not one MCP server per application. For the platform side, MCP Studio is where these servers get built. You can watch a short introduction to MCP Studio in the demo center, or read the launch announcement for the bigger picture.
Its API client declares CallDetails.frm
and .to
as required strings, and dialer calls store them as null. Any page containing a dialer call throws a validation error, so the agent cannot reach those records. Its analytics endpoint also only covers the last 30 days.
Yes on the November tasks. It returned 398 calls and a 394 outbound / 4 inbound split, both matching ground truth exactly.
Nexla’s server takes a URL and a service key over streamable HTTP. The Avoma MCP server runs through mcp-remote with OAuth, and the token expired during our run, which forced five re-runs.
It was asked them, and it correctly reported it had no tools for them. This server was built for call analysis. Avoma’s server handles taxonomy and meeting volume well.
claude-opus-5 for both the agent and the judge, streaming, one isolated session per task and server, scored against ground truth derived from the raw call records.
Describe the questions your agents need answered. MCP Studio builds a server on modeled, governed data from 1000+ systems, and it installs with a URL and a key.
Context layer series Everyone who sells a context layer talks about freshness. Fresh rows, streaming…
Context layer series MCP tool schema design is the practice of writing a tool’s name,…
Context layer series Two MCP servers sit in front of the same warehouse. You ask…