Amazon asked an outside group to grade its homework, then published the grade. In a post on Amazon Science dated October 20, 2026, the company said the nonprofit AVERI (AI Verification and Evaluation Research Institute) reviewed its internal safety-evaluation prompts for Amazon's models. The review confirmed strengths, flagged gaps, and Amazon says it made four changes. Aggregators framed it this week as Amazon "updating AI safety testing after an independent review identified weaknesses."
The unusual part is the target. AVERI did not test Amazon's models. It examined the tests themselves, asking whether they actually stress safety boundaries. That is called meta-evaluation, and it is rare. Below: what was reviewed, what was found, what changed, and what the post cannot tell you.
TL;DR: the review at a glance #
| Question | Answer |
|---|---|
| Who reviewed? | AVERI, an independent nonprofit; led by executive director Miles Brundage |
| What was reviewed? | A random sample of prompts from Amazon's internal safety-evaluation suite |
| Was Amazon's eval team involved? | Not in the assessment, per Amazon |
| Confirmed strengths | Topical coverage, diversity of testing approaches, low contamination risk |
| Gaps flagged | Uneven multilingual coverage, too few multistep adversarial prompts, stale references, superficial refusal cues |
| Amazon's fixes | Test-set refresh, difficulty tiers, quarterly reviews, broader multilingual prompts |
| Independent confirmation of the fixes? | None found; the account is Amazon's own |
Agents that act on your behalf need rules. Agentbeam stops the dangerous actions →
What did AVERI actually review? #
Per Amazon's post, the company provided "a random sample of prompts drawn from our internal safety-evaluation suite," spanning categories such as harmful-content generation, cybersecurity risks, privacy protection and copyright compliance. AVERI's team, led by Miles Brundage and including Sean McGregor, A. Feder Cooper and George Balston, examined the prompts for diversity, difficulty calibration and methodological soundness.
Amazon says the assessment was done "without input from our evaluation team," so the findings would reflect an external view. Note what that does not include: the full test suite, grading outputs, or the models' actual results. A random sample can reveal patterns, but it cannot certify a whole suite.
Amazon already publishes frontier safety material for its Nova models; see its papers on Nova Premier under the Frontier Model Safety Framework and Nova 2.0 Lite. This review looks one layer down, at the quality of the prompts feeding such evaluations.
What did the review confirm? #
A droplet seeping between trays, picturing evaluation contamination that the Amazon safety test review rated low
Amazon lists three validated strengths:
- Comprehensive topical coverage , aligned with publicly documented safety categories including self-harm and cybersecurity.
- Diversity of testing approaches : professional versus informal writing styles, single-turn and multiturn conversations, and original content.
- Low contamination risk : prompts appeared to be original, not copied from public datasets, which lowers the chance that a model saw them in pretraining.
Contamination matters because a model that memorized the test passes without being safe. We covered the same worry for capability benchmarks in our post on DeepMind's double-blind evaluation work against benchmark contamination.
What weaknesses did it surface? #
A green probe pressing a crack in a shell, showing red-team style gaps found in AI safety evaluation prompts
Amazon's list of "opportunities for improvement," quoted closely:
- Linguistic and stylistic diversity: extend multilingual coverage more evenly across risk categories and vary prompt structures.
- Attack sophistication: add more multistep adversarial prompts so evaluations "probe deeper model reasoning" and better reflect real misuse.
- Freshness: a small number of prompts had time-limited references, and a few edge cases had unclear grading criteria.
- Signal quality: "In some cases, models may refuse based on superficial cues in a prompt, so passing those tests doesn't necessarily indicate reliable safety behavior in new situations."
The last point is the most important for anyone who reads safety scores. If a model refuses because a prompt contains a scary word, it passes the test without understanding the request. Real attackers rephrase. This is the same reason jailbreak research focuses on multi-turn and indirect attacks, a theme in our coverage of METR's red-team review of OpenAI's internal agent monitoring.
The four changes Amazon says it made #
| Change | What it means in practice |
|---|---|
| Test set refresh | Removed redundant prompts, added harder adversarial scenarios |
| Difficulty calibration framework | Tiered levels so tests include prompts that push limits, not only easy cases |
| Quarterly review cadence | Regular maintenance to catch staleness and repetition and add new attack patterns |
| Broader language coverage | Multilingual prompts spread across all risk categories |
Amazon says it plans to keep engaging independent evaluators as its models evolve and calls third-party meta-evaluation "an emerging best practice."
What this does and does not show #
It shows: a major lab let an outside group inspect the quality of its safety tests and published specific shortcomings. That is more transparency than most labs offer about evaluation quality. For context on how other labs use outside evaluators, see Anthropic's independent evaluator arrangements with METR and the FTC probe into OpenAI and Anthropic safety.
It does not show:
- How Amazon's models score. No model results are given.
- That the fixes work. We found no independent verification of the changes, and the account is Amazon's own.
- How big the gaps were. The post says "a small number" for stale prompts but gives no counts, percentages or sample size.
- What AVERI's own report says. We reviewed Amazon's description; we did not find a separate AVERI publication in our search.
- Whether AVERI was paid or what its funding is. The post does not say, and readers should ask, since independence depends on it.
Why meta-evaluation matters for people building with AI #
Teams that pick models on safety scorecards inherit the quality of the underlying tests. Three practical takeaways:
- Ask what a safety score is measured against. A pass rate on easy, single-turn prompts says little about a determined attacker.
- Test in your languages. Amazon's own gap on even multilingual coverage is common. If your users are not English-speaking, run your own tests in their language.
- Add multistep attacks. Agents act over many steps; single-prompt tests miss the failure modes that matter. If you deploy agents with real access,AgentBeam is the agent security platform from the explainx.ai team, built to stop AI agents before they take dangerous actions.
A simple internal checklist for your own evals: refresh prompts quarterly, track which prompts are time-limited, separate easy and hard tiers, spot-check that refusals are for the right reason, and keep a held-out set you never publish.
How to audit your own eval set, step by step #
Amazon's four fixes translate into a routine any team can run on its own tests, whether they are safety prompts, jailbreak suites or product acceptance checks.
- Sample, then read. Pull 50 to 100 random prompts and read them as an attacker would. You will quickly see duplicates, outdated references ("the latest model", a past election, a retired product) and prompts that are really the same test in different words.
- Tag difficulty. Label each prompt as easy (a direct request), medium (a reworded or role-played request) or hard (multistep, indirect or tool-assisted). If nearly all fall in the first bucket, a high pass rate is cheap. Amazon's tiered difficulty calibration is this exercise, formalized.
- Check why a model passed. For a sample of refusals, vary the prompt: remove the trigger word, translate it, split it across turns. If the refusal disappears, the model was reacting to surface cues, which is the "superficial cues" problem AVERI flagged.
- Measure language and style spread. Count prompts by language and by category. A suite with 90% of its multilingual prompts concentrated in two risk categories gives you false confidence everywhere else.
- Set a calendar. Put a recurring review on the team's schedule. Amazon chose quarterly. Without a date, test sets fossilize.
- Keep a private hold-out. If your prompts are public, future models may train on them. Keep a small set that never leaves the building.
None of this requires a nonprofit, though an outside reader is worth having: people who wrote the tests tend to miss the same blind spots twice.
Context: why independent checks on evaluations are becoming a theme #
Safety claims increasingly rest on evaluations, and evaluations rest on prompts that someone wrote. When a lab reports that a model "passes" a safety suite, the claim is only as strong as the suite. This is why outside review has become a recurring story this year. Recent examples on this site include the FTC's probe into how two leading labs describe their safety work, METR's review of Anthropic's risk report, which found that prompt sensitivity matters, and the broader proposal for embedded assessments of AI agent control.
The Amazon-AVERI work is narrower and more modest than those: a sample of prompts, a short blog post, no model scores. Its value is the precedent. A company publicly saying "an outsider found our tests were too easy in places, and here is what we changed" sets a bar that others can be asked to meet. It also invites the obvious follow-up questions listed above about sample size, funding and publication of AVERI's own findings.
What to watch next #
- Whether AVERI publishes its own account of the review, with sample sizes and methods.
- Whether other labs commission similar meta-evaluations, and whether results are published.
- Whether Amazon publishes updated Nova safety results using the refreshed test set.
- Quarterly refresh evidence: a second dated update would show the cadence is real.
Related reading #
Details are accurate as of October 26, 2026 and rest on Amazon Science's October 20 post.