Evals for MCP Servers: Slow is Smooth, Smooth is Fast

In a technical blog post, the team behind xplainable, an ML platform, details how they built evaluation harnesses for their MCP server to test multi-step agentic workflows, arguing that early evals accelerate development. The post outlines a nine-step churn model build process and highlights failure modes like silent error compounding and train/serve parity issues, sharing their approach in the xplainable-mcp-server GitHub repository.

Evals often feel like a slowdown, an afterthought in the world of agentic AI development. You’ve shipped the tools, the demo works, the agent does the thing. Why formalise it? There’s a saying engrained in my mind from my rowing days: ”slow is smooth, smooth is fast.” It’s always easier to hold good form than to recover it once you’ve lost it. Evals are the same. The teams that build them early move faster later, because every model release, every prompt tweak, every tool change becomes a measurable event instead of a vibe check. This post walks through how we built evals for our MCP server. Not as a prescription, more as a worked example you can steal from: the harness ships inside our MCP server repo on GitHub https://github.com/xplainable/xplainable-mcp-server . We build xplainable https://www.xplainable.io , an ML platform covering data preparation, training, deployment, prediction and optimisation. Our MCP server exposes that lifecycle to agents. With that in mind, it’s worth delineating two very different kinds of MCP servers. Simple tool calls. “Send an email to Joe Bloggs about the new functionality.” “Send a batch email to the user cohort captured in the CRM.” One or two tool calls, a clear success condition, largely deterministic. If the tool works once, it mostly works. Multi-step workflows. The agent has to orchestrate a process : many tools, in a meaningful order, where the output of one step feeds the next, and where a plausible-looking transcript can still be wrong. We’re firmly in the second category. Our MCP server exposes the full machine learning development lifecycle, and then takes it a step further, providing the tools to create mathematically-driven recommendations from the trained models, abstracting away the need to set up experiments manually. A single “build me a churn model” request steps through: 3. Data preparation 4. Feature engineering 5. Persist the preparation pipeline so serving matches training 6. Train the model 7. Deploy it 8. Predict through the deployment 9. Create a report returns a URL 10. Optimise : generate prescriptions/recommendations optional As you can see, this is substantially more complex than “send an email.” It’s worth pausing on why this class of workflow resists the usual testing instincts. Every step depends on the last, and errors compound silently. This is true even within a single step: a preprocessing pipeline is itself sequential, where each transformation operates on the output of everything before it. Drop a column at step 2 and the encoding at step 5 sees a different frame; reorder two steps and the same spec produces different data. And the same dependency chain runs through the whole workflow: the preprocessing you apply determines the features the model sees; the features determine what the model learns; what the model learns determines whether the optimiser’s recommendations mean anything. A mistake at step 3 doesn’t fail at step 3. It surfaces, if it surfaces at all, as a subtly worse model at step 6, or nonsense prescriptions at step 10. By then the transcript is forty tool calls long and every one of them returned 200 OK . Train/serve parity is the classic trap. The transformations applied at training time must be applied identically at inference time. If the agent trains on engineered features but the deployment serves raw data or vice versa , nothing crashes; predictions are just quietly wrong. This bug has haunted human ML teams for a decade; an agent orchestrating it through tools inherits it wholesale, with less visibility. The state lives server-side, not in the conversation. Datasets, preprocessor versions, model versions, deployments: these are artifacts on a platform, referenced by IDs the agent must create, track, and wire together correctly across a long context. Training and report generation run as async jobs behind polling. The model only ever sees narrow slices of this state through tool responses; it can’t inspect the pipeline the way an engineer would. “Success” is not a status code. A model can train without error and be useless. Probabilities can saturate to 0.99 across the board. An optimiser can return “recommendations” that recommend changing nothing, or that change columns the business declared immutable. A prescription to reduce churn by making the customer younger is mathematically valid and operationally absurd. Every one of these looks like a pass if all you check is that the calls succeeded. And the whole chain runs end-to-end , from raw dataset to mathematical recommendations fed back into the business. The surface area for plausible-but-wrong is enormous, and the failure modes aren’t crashes; they’re silent process failures : training on raw data instead of the engineered features, deploying a model that saturates every probability, generating recommendations that say “do nothing.” Before going further, a caveat I think matters. From the Pydantic Evals docs: Evals are an Emerging Practice Unlike unit tests, evals are an emerging art/science. Anyone who claims to know exactly how your evals should be defined can safely be ignored. I’d echo that. If anyone claims “ this is how evals must be done,” you can quickly discount the rest of the advice. What follows is what worked for us, for this kind of server. Your failure modes will differ. If you’re an AI engineer, you’re probably familiar with the current loop: Notice how many layers a single change travels through, across three repositories, before an agent ever calls it: API, client wrapper, package release, pin bump, server CI, deployment, consumption. The MCP server generates its tool surface at runtime from the client’s registry, so the new tool appears automatically once the pin lands; the only guard rail at that layer is a test pinning the exact tool count. The unit tests at each layer can all be green while the end-to-end behaviour an agent actually completing a job with the tool has regressed. That gap between “every layer passes” and “the workflow works” is exactly where evals live. Then you verify by hand: 4. Check the tool shows up in your consumption method CLI, Claude Desktop 5. Do one pass through the dataset/prompt you use for all your testing This works, right up until it doesn’t. Two situations break it: One pass through one prompt tells you nothing about either. We built ours on pydantic-ai to drive the agent and pydantic-evals to score it . Pydantic Evals follows a code-first approach: datasets, cases, and evaluators are all defined in Python. Note: the original suggestion was mcp-use an open-source way to connect any LLM to any MCP server and build custom MCP agents, worth a look . We went with pydantic-ai to keep everything in one ecosystem: its agent, MCP client, and eval framework share types. The core loop is small. pydantic-ai speaks MCP natively, so the agent gets your server’s actual tool surface. No mocks, no re-declared schemas: python from pydantic ai import Agentagent = Agent model, "anthropic:claude-sonnet-4-6", or anything else toolsets= mcp toolset , your MCP server, in-process or over HTTP result = await agent.run scenario.task prompt Two targets, two auth modes.The harness is built to point at the serverin-process authenticated with an API key from the environment, ideal for fast local iteration and CI or at thehosted production serverover HTTP behind the same OAuth flow real users go through . Same scenarios, same evaluators; only the toolset construction changes. Being able to eval the exact deployment users hit, auth and all, is worth wiring up early. Each scenario is one realistic job with an expected shape: Scenario name="telco churn minimal", task prompt="Build and deploy a churn model on the uploaded dataset...", expected stages= DATA PREP, PERSIST PREP, TRAIN, DEPLOY, PREDICT , And pydantic-evals runs it as a dataset of cases, each scored by evaluators: python from pydantic evals import Case, Datasetdataset = Dataset cases= Case name=f"{scenario.name} {i} ", inputs=scenario for i in range k , evaluators= StageEvaluator scenario.expected stages , SemanticDetectors , Efficiency , report = await dataset.evaluate run scenario The single most useful design decision: evaluators inspect what actually happened on the platform , not which tools the agent called. “Did it call train model?" is a weak assertion: the call can succeed while the process fails. To know what actually happened, each case is bracketed by a session ledger: snapshot the platform before the agent runs, diff after, and tear down whatever the run created. The diff is ground truth for “what did the agent build”, and the teardown keeps the eval team clean between runs this is a real platform with real state and real quotas, not a sandbox : class EvalSession: """Before/after ledger of platform artifacts for one eval case.""" def snapshot self : self. before = self. list platform ids datasets, models, deployments... def diff self - CreatedArtifacts: now = self. list platform ids return CreatedArtifacts {k: now k - self. before k for k in now} def teardown self, created: CreatedArtifacts - list str : """Delete everything the run created; return whatever would not delete.""" With that in place, evaluators can assert against platform state instead of transcript vibes: php def check train outcome - bool: """Model exists AND was trained on the transformed data: the train call must reference a preprocessor the agent created.""" if not outcome.created.models: return False created = set outcome.created.preprocessors return any call.name in TRAIN TOOLS and not call.error and any arg in created for arg in leaf values call.args for call in outcome.tool calls That check exists because of a real regression: the agent was happily “training” models on raw data, skipping every engineered feature, and every transcript looked fine. An eval that only counted tool calls would have passed. Alongside stage checks, we run semantic detectors for known failure modes: degenerate prescriptions recommendations that change nothing , saturated probabilities every prediction ≈ 1.0 , drift in columns that were declared immutable. Each detector is a boolean that fires when the failure is present, and each one encodes a specific silent failure we actually observed. This one exists because a live optimiser run once returned twenty “prescriptions” that all prescribed identical lever values, and everything upstream reported success: php def degenerate prescriptions prescriptions: list dict - bool: """All rows prescribe identical lever values. True = failure.""" if len prescriptions < 2: return False mappings = prescribed changes row for row in prescriptions return bool mappings 0 and all m == mappings 0 for m in mappings 1: These came directly from post-mortems; your list will come from yours. One thing that turned out to matter as much as the evals themselves: structured errors . We propagate typed error envelopes Pydantic models all the way through to the tool response: VALIDATION ERROR Pipeline compilation failed: 'BinOp' object hasno attribute 'type' - Suggestion: check the transformer spec syntax Readable errors are important for humans; they’re just as important for agents. In our runs, models recover from structured errors and retry correctly, where an opaque 500 used to send them into a doom loop. The eval harness measures this too: failed calls that get retried show up as wasted calls, so error-message quality becomes a number you can watch. Each run writes a results JSON: per-case stage assertions, detector flags, step count, wasted calls, duration, token usage and dollar cost OpenRouter reports the cost of every request when you ask for it, so a run's price tag is captured automatically , and the tool-call transcript. A small reporting CLI turns any set of result files into a comparison: Dashes in the cost column are runs recorded before cost capture landed; old result files stay comparable. The headline metric is pass@k , borrowed from code-generation benchmarks. The idea: agent runs are stochastic. The same model, same prompt, and same tools will sometimes succeed and sometimes fail. So instead of asking “did it work once?”, pass@k asks “if I gave the agent k attempts, what is the probability that at least one of them completes the whole workflow?” pass@1 is the single-shot success rate; pass@5 tells you whether retrying rescues the failures or whether the model just cannot do the task. The gap between the two is informative on its own: a model with pass@1 of 0.6 but pass@5 of 1.0 is flaky but capable, while a model stuck at 0.6 for both has a systematic blind spot that no amount of retrying will fix. To compute it we run each scenario n times, count the c successes, and use the standard unbiased estimator rather than naively resampling: php def pass at k n: int, c: int, k: int - float: """n samples, c passes: 1 - C n-c, k / C n, k """ return 1.0 - comb n - c, k / comb n, k …plus the familiar pass@k curves, per-stage pass-rate profiles so you can see where in the pipeline a model falls over, not just that it did , and step-count distributions. The per-stage profile earns its keep quickly. Look at that glm-fullrow again: pass@k of 0.00 sounds like total failure, but the stage columns tell a completely different story. Take a real run of GLM 5.3 over the full ten-stage scenario, four scored attempts, at a captured cost of $8.26. The model went four for four on nine of the ten stages, every single attempt: exploration, data prep, feature engineering, training, deployment, prediction, even optimisation. It failed exactly one stage, every time: surfacing the URL of an asynchronously generated report. The headline metric says “cannot do the workflow”; the profile says “does the entire workflow, except one async-timing interaction it never gets right”. Those demand very different responses. The first sends you model shopping; the second sends you to fix one tool’s job-status ergonomics. This is why a single scalar is not enough for multi-step servers: the shape of the failure is the actionable part. The tool-call log makes the same point even more bluntly. Across those four attempts the model made 240 tool calls, and 103 of them, 43%, were to a single endpoint: reports get job status, polled over and over waiting for the async report, with 57 of those calls returning errors. Every other tool on the surface behaved. One endpoint’s ergonomics consumed nearly half the model’s effort: The call-by-call timeline shows how each attempt actually died. Every one starts the same way: a long, clean run of workflow calls that carries the model through data prep, training, deployment, prediction and optimisation. Then the report job kicks off and the back half of every attempt dissolves into polling and errors: This is where the harness pays for itself. Research keeps showing that a well-built harness can let a smaller model compete with the frontier https://arxiv.org/abs/2607.08938 , and with output-token prices spread 25–40x between tiers https://vinvashishta.substack.com/p/minimum-viable-model-structured-model , the cheap model can burn far more tokens and still come out ahead, right up until the next release makes the comparison obsolete. The only sane response is to make model swaps cheap to measure . Because pydantic-ai handles provider routing, testing a new model is a CLI flag. OpenRouter support required zero harness changes : python -m evals.run \ --model anthropic:claude-sonnet-4-6 \ --model openrouter:z-ai/glm-5.3 \ --scenario telco churn minimal -k 3 A real example from this week, same scenario, same tools, same prompt: Both models completed the entire ML lifecycle. One took ~60% more steps and ~75% more wall-clock. That’s the shape of answer you want on release day: The intent is to wire this into CI: on a new model release or a new server version , GitHub Actions runs the eval matrix, logs results, and we check for regressions before anything ships, the same way you’d treat a dependency bump. The part nobody writes about. Two things kept our harness from rotting on day one: Derive what you can from the server itself. Our tool classifications read vs write, train tools, predict tools come programmatically from the same registry the MCP server uses to build its surface, not hand-maintained lists. When a new endpoint is wrapped and appears on the server, the harness picks it up automatically. The trade-off is real, though: deterministic derivation gives you one surface to maintain, but limited flexibility around edge cases; prompts like to be seen , and a scenario will never exercise a tool its prompt doesn’t ask for. We keep one deliberate tripwire: a test that pins the exact tool count, so the surface can never change silently. Treat scenarios and prompts as a growing library. New use cases become new scenarios; new prompt variants become a --prompt axis in the same cross-product as --model. The harness doesn't care which one you're varying. And one operational lesson: eval runs create real artifacts datasets, models, deployments, reports . Snapshot before, diff after, tear down everything the run created, and run them against a dedicated team/workspace, never one you work in. The obvious next step is to point this at the rest of the development practice: the frontend, external services, API design, DB design. If an agent is going to drive development, the same question applies everywhere: how do you know the multi-step process actually worked? The less obvious step is the feedback loop in reverse: our evals have already reshaped the API. Watching agents fail told us which errors needed structure, which tools needed better affordances, which workflows needed to move server-side. It happened again while writing this post: the harness’s teardown couldn’t delete the reports that eval runs generate, because nothing had ever needed to delete one programmatically. The delete endpoint existed in the API; no client wrapper exposed it. Closing that gap for the evals shipped a new wrapper and a new tool on the agent surface the same afternoon. There’s a whole development lifecycle emerging around refactoring APIs for agents, and evals are the instrument that tells you whether the refactor helped. The node that deserves the highlight is the data model. Every API affordance an agent needs, a pollable job status, a listable artifact lineage, a clear owner for every object, is only expressible if the database models it. When our agents flailed polling reports get job status, the fix wasn't a prompt tweak; it was making job state a first-class, queryable thing. Agent-facing API design is DB design wearing a costume. The evals are what tell you which part of the schema the agents are tripping over. Slow is smooth. Smooth is fast. We build xplainable : explainable ML through a platform, an API, and an MCP server. The server, including the eval harness from this post, is open on GitHub . Evals for MCP Servers: Slow is Smooth, Smooth is Fast https://pub.towardsai.net/evals-for-mcp-servers-slow-is-smooth-smooth-is-fast-11c726250a5d was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.