{"slug": "firstpass-a-multi-domain-multi-round-peer-review-dataset-grounded-in-real", "title": "FIRSTPASS: A Multi-Domain, Multi-Round Peer Review Dataset Grounded in Real Editorial Outcomes", "summary": "Researchers released FIRSTPASS, the first large-scale peer review dataset built on complete multi-round editorial dialogues from Nature Communications' mandatory transparent peer review, comprising 3,668 records across biology, chemistry, neuroscience, physics, and earth science. Each record includes referee reports, author responses, and updated assessments, with outcome labels derived from editorial decisions, providing ground truth for training AI systems to critique scientific work beyond computer science. Expert reviews average 2,155 words, and all data and pipelines are released for reproducible benchmarking.", "body_md": "arXiv:2608.26129v1 Announce Type: new\nAbstract: Scientific peer review datasets have trained AI systems exclusively on Computer Science and Machine Learning venues, producing models that critique ablation studies yet have never seen a biology reviewer demand contamination controls or a chemist question Nuclear Magnetic Resonance (NMR) spectral assignments. We introduce FIRSTPASS, the first large-scale peer review dataset built on complete multi-round editorial dialogues from a multidisciplinary high-impact journal. Curated from Nature Communications mandatory transparent peer review (instituted November 2022), FIRSTPASS comprises 3,668 records spanning five scientific domains (biology, chemistry, neuroscience, physics, and earth science), capturing the full iterative structure of scientific validation: initial referee reports, author point-by-point responses, and updated reviewer assessments. Each record carries an outcome label derived directly from editorial decisions (STANDARD for two-round review; EXTENDED for three or more rounds), providing ground truth absent in all prior corpora. An automated audit confirms 100% content integrity. Expert reviews average 2,155 words, substantially denser than conference venue reviews. All data, parsing pipelines, and evaluation scripts are released to enable reproducible benchmarking of AI scientific judgment across disciplines.", "url": "https://wpnews.pro/news/firstpass-a-multi-domain-multi-round-peer-review-dataset-grounded-in-real", "canonical_source": "https://arxiv.org/abs/2608.26129", "published_at": "2026-08-28 04:00:00+00:00", "updated_at": "2026-08-28 04:20:18.338868+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-tools"], "entities": ["FIRSTPASS", "Nature Communications", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/firstpass-a-multi-domain-multi-round-peer-review-dataset-grounded-in-real", "markdown": "https://wpnews.pro/news/firstpass-a-multi-domain-multi-round-peer-review-dataset-grounded-in-real.md", "text": "https://wpnews.pro/news/firstpass-a-multi-domain-multi-round-peer-review-dataset-grounded-in-real.txt", "jsonld": "https://wpnews.pro/news/firstpass-a-multi-domain-multi-round-peer-review-dataset-grounded-in-real.jsonld"}}