{"slug": "automatic-hard-example-synthesis-with-multi-level-agentic-data-curation", "title": "Automatic Hard Example Synthesis with Multi-Level Agentic Data Curation", "summary": "Researchers propose an automated red-teaming framework that uses a multi-agent architecture to synthesize hard examples for multimodal LLMs, reducing false negative rates on safety benchmarks from 41.2% to 24.5% without human labeling.", "body_md": "arXiv:2607.14256v1 Announce Type: new\nAbstract: Multimodal Large Language Models (MLLMs) are increasingly deployed for nuanced content safety and moderation tasks, yet they remain vulnerable to adversarial attacks and out-of-distribution edge cases. Traditional active learning and manual annotation fail to scale against the complexity and volume of novel multimodal threats. In this paper, we propose an automated, agentic red-teaming framework that systematically synthesizes difficult examples using an iterative strategy that proposes novel hypotheses as well as mutating on past attempts. Leveraging a multi-agent architecture that consists of a high-reasoning Architect agent, an advanced image generator, and a multi-level verification committee of LLM raters, our system autonomously uncovers boundary-pushing violations and ambiguous policy edge cases without any human intervention. By employing these carefully synthesized adversarial examples as in-context demonstrations via test-time Retrieval, we substantially improve the target model's robustness, reducing the False Negative Rate (FNR) from 41.2% to 24.5% in a public image safety benchmark without relying on any human labeling.", "url": "https://wpnews.pro/news/automatic-hard-example-synthesis-with-multi-level-agentic-data-curation", "canonical_source": "https://arxiv.org/abs/2607.14256", "published_at": "2026-07-17 04:00:00+00:00", "updated_at": "2026-07-17 04:28:00.816521+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-safety", "ai-agents", "computer-vision"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/automatic-hard-example-synthesis-with-multi-level-agentic-data-curation", "markdown": "https://wpnews.pro/news/automatic-hard-example-synthesis-with-multi-level-agentic-data-curation.md", "text": "https://wpnews.pro/news/automatic-hard-example-synthesis-with-multi-level-agentic-data-curation.txt", "jsonld": "https://wpnews.pro/news/automatic-hard-example-synthesis-with-multi-level-agentic-data-curation.jsonld"}}