{"slug": "strengthening-ai-safety-evaluation-through-independent-third-party-review", "title": "Strengthening AI safety evaluation through independent third-party review", "summary": "Amazon had the independent nonprofit AI Verification and Evaluation Research Institute (AVERI) review a random sample of prompts from its internal safety-evaluation suite, and AVERI's team — led by executive director Miles Brundage with researchers Sean McGregor, A. Feder Cooper and George Balston — validated the suite's topical coverage, testing-approach diversity and low contamination risk while recommending broader multilingual coverage, more multistep adversarial prompts, fresher prompts and clearer grading criteria. Amazon said it responded with four concrete changes: a test-set refresh removing redundant prompts and adding harder adversarial scenarios, a tiered difficulty-calibration framework, a quarterly review cadence and expanded multilingual prompts across all risk categories. The review matters because it applies outside scrutiny to the quality of the safety tests themselves, not just model outputs, at a time when frontier-model safety evaluations face growing external pressure.", "body_md": "As frontier AI models grow in capability, so does the importance of ensuring that safety evaluations are rigorous and comprehensive. At Amazon, we invest heavily in internal safety testing for our models, spanning categories like harmful-content generation, cybersecurity risks, privacy protection, and copyright compliance.\n\nA core principle of robust quality assurance is that even rigorous internal processes are strengthened by external scrutiny. That's why we partnered with [AVERI](https://www.averi.org/) (AI Verification and Evaluation Research Institute), an independent nonprofit dedicated to third-party evaluation of AI systems, to conduct an independent review of our internal safety evaluation datasets.\n\nAVERI’s mandate wasn’t just to test model outputs but to examine the *quality and rigor of our tests themselves* — asking whether our evaluations stress-test safety boundaries.\n\nHow the collaboration worked\n\nWe provided AVERI with a random sample of prompts drawn from our internal safety-evaluation suite. AVERI's team — led by executive director Miles Brundage and including researchers Sean McGregor, A. Feder Cooper, and George Balston — examined the evaluation prompts for diversity, difficulty calibration, and methodological soundness.\n\nAVERI assessed the sample independently without input from our evaluation team, ensuring their findings reflected a genuine external perspective.\n\nWhat the review confirmed\n\nAVERI’s review validated several strengths of our evaluation process:\n\n- **Comprehensive topical coverage** — aligned with publicly documented safety categories including self-harm, cybersecurity, and other risk areas;\n- **Diversity of testing approaches** — variation in writing style (professional vs. informal), single-turn and multiturn conversations, and original content;\n- **Low contamination risk** — content that appeared to be original rather than sourced from publicly available datasets, reducing risk of evaluation leakage through pretraining.\n\nOpportunities for improvement\n\nWith any fresh review comes the opportunity to identify areas that internal teams — deeply familiar with their own work — may overlook. AVERI's external perspective surfaced several opportunities to further strengthen an already comprehensive suite:\n\n- **Expanding linguistic and stylistic diversity***—* While our evaluations include multilingual prompts, AVERI recommended extending multilingual coverage more evenly across risk categories and further varying prompt structures to broaden the range of scenarios tested;\n- **Calibrating attack sophistication** *—* AVERI recommended adding more multistep, adversarial prompts to our existing set, so evaluations probe deeper model reasoning and better reflect sophisticated real-world misuse attempts;\n- **Maintaining evaluation freshness** *—* AVERI identified a small number of prompts with time-limited references and a few edge cases where grading criteria could be made clearer;\n- **Ensuring evaluation signal quality** *—* In some cases, models may refuse based on superficial cues in a prompt, so passing those tests doesn’t necessarily indicate reliable safety behavior in new situations.\n\nHow we incorporated the feedback\n\nConcrete improvements made:\n\n1. **Test set refresh** : Updated evaluation datasets to address specific issues identified, removing redundant prompts and adding more-challenging adversarial scenarios\n2. **Difficulty calibration framework** : Introduced tiered difficulty levels to ensure that evaluations include prompts that genuinely push model safety limits, not just straightforward cases\n3. **Quarterly review cadence** : Established a regular evaluation maintenance process to catch staleness, reduce repetition, and incorporate emerging attack patterns\n4. **Broader language coverage** : Expanded multilingual prompts across all risk categories rather than concentrating them in specific areas\n\nA model for the field\n\nThis collaboration represents what we believe is an emerging best practice: engaging independent third parties to evaluate not just *model outputs*, but the *quality of the evaluation process itself*. As the AI safety field matures, we expect this kind of meta-evaluation to become an important complement to traditional red-teaming and benchmarking.\n\nThe benefits are mutual: Amazon gains an independent check on our evaluation quality, while AVERI gains practical experience with the real-world challenges of assessing internal evaluation processes — insights that can inform best practices for the field as a whole.\n\nWe're grateful to AVERI for their thorough and constructive review, and we plan to continue engaging independent evaluators as our models evolve. Transparency about our safety processes — including where we can improve — is fundamental to building justified confidence in AI safety.", "url": "https://wpnews.pro/news/strengthening-ai-safety-evaluation-through-independent-third-party-review", "canonical_source": "https://www.amazon.science/blog/strengthening-ai-safety-evaluation-through-independent-third-party-review", "published_at": "2026-10-20 17:57:55+00:00", "updated_at": "2026-10-20 18:15:32.109144+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "large-language-models", "artificial-intelligence"], "entities": ["Amazon", "AVERI", "Miles Brundage", "Sean McGregor", "A. Feder Cooper", "George Balston"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/strengthening-ai-safety-evaluation-through-independent-third-party-review", "markdown": "https://wpnews.pro/news/strengthening-ai-safety-evaluation-through-independent-third-party-review.md", "text": "https://wpnews.pro/news/strengthening-ai-safety-evaluation-through-independent-third-party-review.txt", "jsonld": "https://wpnews.pro/news/strengthening-ai-safety-evaluation-through-independent-third-party-review.jsonld"}}