{"slug": "regdivergence-101-an-llm-benchmark-for-cross-jurisdiction-regulatory-detection", "title": "RegDivergence-101: An LLM Benchmark for Cross-Jurisdiction Regulatory Contradiction Detection in Life Sciences", "summary": "Researchers released RegDivergence-101, a 101-pair benchmark for detecting regulatory contradictions between FDA and EMA guidance in life sciences, with an LLM judge (Claude Haiku) achieving 0.830 macro-F1, outperforming lexical (0.511), NLI (0.233), and Graph-RAG (0.663) baselines. The benchmark classifies pairs as AGREE, DIVERGE, or SILENT, with inter-annotator agreement kappa = 0.85, and highlights corpus-level graph construction as the target for large-scale silent detection.", "body_md": "arXiv:2608.28607v1 Announce Type: new\nAbstract: Pharmaceutical sponsors developing a drug for both the United States and the European Union must reconcile guidance issued independently by the FDA and the EMA. Where the two agencies require substantively the same thing, a sponsor can file once; where they diverge, a single trial design risks rejection in one region; where one agency is silent on a point the other regulates, the sponsor must infer obligations. Today this reconciliation is performed manually by regulatory-affairs experts. We introduce cross-jurisdiction regulatory divergence detection: given an FDA requirement and an EMA requirement on the same topic, classify their relationship as AGREE, DIVERGE, or SILENT. SILENT is inherently directional (SILENT_FDA vs. SILENT_EMA); we record direction per pair and report per-direction F1 alongside the collapsed label. We release RegDivergence-101, a 101-pair expert-grounded pilot evaluation benchmark (labels grounded in three peer-reviewed FDA/EMA comparison studies and primary FDA/EMA/ICH guidance text; dual-annotation inter-annotator kappa = 0.85), and systematically characterise a four-method baseline hierarchy: lexical heuristic (0.511 macro-F1, 95% CI [0.411-0.605]), NLI cross-encoder (0.233), obligation-level Graph-RAG (0.663 [0.570-0.747]), and flat LLM judge / Claude Haiku (0.830 [0.747-0.908]). Three directional observations emerge at pilot scale (n = 101): SILENT is semantically detectable but invisible to entailment-only formulations; pair-level obligation graphs improve over lexical methods but trail flat-LLM context (CIs partially overlapping); and corpus-level graph construction is the indicated architectural target for large-scale silent-detection. RegDivergence-101 is a pilot release establishing the task formulation and baseline hierarchy; four unrepresented regulatory domains and an expansion roadmap are described in Section 7.", "url": "https://wpnews.pro/news/regdivergence-101-an-llm-benchmark-for-cross-jurisdiction-regulatory-detection", "canonical_source": "https://arxiv.org/abs/2608.28607", "published_at": "2026-09-01 04:00:00+00:00", "updated_at": "2026-09-01 04:26:54.403878+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "natural-language-processing", "ai-research"], "entities": ["FDA", "EMA", "Claude Haiku", "RegDivergence-101", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/regdivergence-101-an-llm-benchmark-for-cross-jurisdiction-regulatory-detection", "markdown": "https://wpnews.pro/news/regdivergence-101-an-llm-benchmark-for-cross-jurisdiction-regulatory-detection.md", "text": "https://wpnews.pro/news/regdivergence-101-an-llm-benchmark-for-cross-jurisdiction-regulatory-detection.txt", "jsonld": "https://wpnews.pro/news/regdivergence-101-an-llm-benchmark-for-cross-jurisdiction-regulatory-detection.jsonld"}}