Vizra Evals and Pest's Evals Plugin: When You Want Which

A developer has released Vizra Evals, a Pest plugin for evaluating AI agents in Laravel, and compared it with pestphp/pest-plugin-evals. Vizra Evals persists every run to a database, enabling baselines, row-level regression detection, and historical trend analysis via a companion dashboard package. The developer recommends Vizra Evals when regression tracking is needed, and the Pest plugin when only current performance matters.

If you are testing AI agents in Laravel, there are now two packages with "evals" in the description, and the obvious question is whether you need both. Short answer: probably not, and which one depends on a single question. Do you need to know whether your agent is good now , or whether it is worse than it was last month ? This is a genuine comparison rather than a sales pitch. I wrote one of them, and there is a section below telling you when to use the other one on its own. Vizra Evals is built on Pest. Not alongside it, not as an alternative to it. Your evals are Pest tests, they live in your test suite, and they run through the Pest binary. Both packages also use the same activation contract: ./vendor/bin/pest evals skipped ./vendor/bin/pest --evals evals run against the real model The --evals flag and the PEST EVALS environment variable mean the same thing to both. That is deliberate. You can have both installed in one suite and neither will fight the other for the flag. So this is not a migration decision. Nothing has to be torn out. pestphp/pest-plugin-evals scores agent output inside an expectation. You point it at an agent, give it a prompt, and assert on the response: php expect CapitalCityAgent::class - prompt 'What is the capital of Japan?' - toBeCorrect expected: 'Tokyo' ; It has a genuinely broad set of expectations. Deterministic ones that cost nothing: toContain , toMatch , toBe , toBeJson , toHaveToolCalls and toFollowTrajectory , which checks tools were called in a given order. Then scored ones that call a model: toBeCorrect , toBeRelevant , toBeSafe , toBeSimilar for semantic similarity via embeddings, toSatisfy for plain-English criteria, and toPassScorer for your own. Scorers return 0.0 to 1.0 against a configurable threshold. toBeCorrect is more nuanced than a boolean: an exact match scores 1.0, approximately equal 0.9, a superset of the expected answer 0.8, a subset 0.6, and a contradiction 0.0. It handles nondeterminism too. repeat runs the same prompt several times and asserts every expectation against all of the samples, so a test only passes when the agent is consistent. That is the right instinct and a lot of eval tooling misses it. prompt also takes a plain closure, not just an agent class, so you can score anything that returns a string. This is a well-built plugin. If you have read this far expecting me to find fault with it, that is not where this is going. One thing, and everything else follows from it: it keeps the runs. Every sample, every score, every judge's reasoning, every tool call, every token cost, written to your own database. That turns a passing test into a data point rather than an event. Once runs are persisted, three things become possible that are not possible from a single assertion. A baseline. Your first passing run becomes the reference. Everything after it is measured against that rather than against a fixed threshold you guessed at. Row-level regression detection. Rows are joined across runs by a hash of their content, so the framework knows that "What is your refund policy?" in today's run is the same row as in last month's. When that specific row drops, the build fails and names it: Eval pest: answers support questions from documented policy score 61.7%, pass rate 33.3% across 6 samples. Gate failed: 2 rows regressed against the reference run allowed: 0 . ↓ regressed: "What is your refund policy?" 96.7% → 51.7% ↓ regressed: "Can I return it?" 93.3% → 55.0% Not "the suite scored 61.7%". Which two inputs got worse, and by how much. That difference matters most in a pull request, where the useful question is never "is this good" but "is this worse than what we had". History you can look at. A second package, vizra/evals-ui , mounts a dashboard route inside your own app: score trends per suite, any sample's assertions and judge reasoning one click deep, two runs side by side. It reads the same database your evals already write to. The eval itself looks like this: it 'answers support questions from documented policy', function { expect SupportBot::class - toPassEval fn $eval = $eval - dataset base path 'evals/support.jsonl' - samples 3 - assert fn $a, $row = $a - notEmpty - gate - contains $row- expected - costBelow 0.02 - judge 'Answers using only documented store policy.', min: 7 - gate minScore: 0.8, maxRegressions: 0 ; } ; Note maxRegressions: 0 . That parameter cannot exist without stored history, and it is the whole argument for this package in one keyword. Plenty of the time. Genuinely. You have a handful of agents and a handful of checks. If what you need is "this classifier returns valid JSON" or "this agent never recommends a competitor", that is an expectation, and adding a database and a dashboard to it buys you nothing. You want zero extra infrastructure. Vizra needs a migration. Pest's plugin needs nothing. If you are evaluating in a package rather than an application, or in something with no database at all, that is close to decisive. You are exploring rather than shipping. Working out whether the prompt is any good in the first place is a different activity from defending it over time. Expectations are the faster loop. Your agent's quality genuinely does not drift. Some don't. A structured extraction task with a pinned model and a stable prompt can sit still for a year. If nothing changes, there is nothing to compare. You don't have a baseline worth defending yet. Baselines are only useful once you have a version you are happy with. On week one you are still finding it. In all of those, the honest recommendation is to use Pest's plugin and stop reading. The switch tends to flip at a specific moment, and it is usually a question someone asks you. "Did the answers get worse after we changed the model?" With expectations alone, the answer is a shrug, or a threshold that either failed or didn't. You cannot compare against a number you never wrote down. Concretely, the baseline earns its keep when: There is no conflict. A reasonable setup uses Pest's expectations for quick assertions during development, and Vizra evals for the suites you want to defend over time. Both respond to --evals , so one command runs everything. The docs cover running them side by side https://docs.vizra.ai/evals/pest if you want the detail. | Pest evals plugin | Vizra Evals | | |---|---|---| | Question it answers | Is this agent good? | Is it worse than last month? | | Sampling | repeat | samples | | Scored assertions | Correct, relevant, safe, similar, custom | Judge with criteria and dimensions | | Tool call assertions | Yes | Yes | | Results persisted | No | Yes, your database | | Baseline comparison | No | Yes, per row | | Dashboard | No | Optional, self-hosted | | Infrastructure | None | A migration | | Activation | --evals | --evals | If you are starting today, start with Pest's plugin. It is less to install and it answers the first question you will have. Come back to this when someone asks whether the answers got worse, and you realise you have no way to know. If you want to see what kept history looks like before installing anything, there is a live demo https://vizra.ai/demo?utm source=devto&utm medium=organic&utm campaign=blog-pest-comparison with six weeks of runs behind it.