{"slug": "an-investigation-of-the-neurips-and-icml-2025-position-tracks", "title": "An Investigation of the NeurIPS and ICML 2025 Position Tracks", "summary": "An audit of the NeurIPS 2025 and ICML 2025 Position Paper Tracks finds that three-quarters of accessible submissions critique existing benchmarks, evaluations, or methodologies, while agenda-shifting papers like AlexNet and the Transformer typically introduced new artifacts. The authors argue the tracks should explicitly solicit direction-setting work alongside reformist critiques and propose four CFP-level interventions.", "body_md": "arXiv:2608.16894v1 Announce Type: cross\nAbstract: ML venues shape what kinds of research claims become legible to reviewers and what forms of evidence count as rigorous. The NeurIPS and ICML Position Paper Tracks were created for agenda-setting work, making their early composition worth auditing. \\textbf{This paper argues that the publicly accessible 2025 reviewed pool is dominated by reformist critique, and that the track should explicitly solicit direction-setting work alongside, not in place of, the reformist critiques it already hosts well.} We audit every accessible submission to the NeurIPS 2025 and ICML 2025 Position Tracks under a pre-specified rubric, and compare the resulting pattern with a reference class of widely recognized agenda-shifting ML papers. Three-quarters of audited submissions critique an existing benchmark, evaluation, or methodology; these papers score highly on our artifact-coupling rubric, but evidentiary depth does not predict reviewer rating. The reference class (AlexNet, the Transformer, Concrete Problems in AI Safety, and others) differs from the accessible reviewed pool in \\emph{artifact kind}: agenda-shifting papers typically gave the field something new to build on, test against, or contest, such as a measurement protocol, benchmark proposal, toy implementation, dataset card, audit template, or falsifiable experimental program. We close with four CFP-level interventions aimed at broadening the submission mix without displacing the critiques the track already hosts well.", "url": "https://wpnews.pro/news/an-investigation-of-the-neurips-and-icml-2025-position-tracks", "canonical_source": "https://www.machinebrief.com/news/an-investigation-of-the-neurips-and-icml-2025-position-track-eqb2", "published_at": "2026-08-19 04:00:00+00:00", "updated_at": "2026-08-19 05:11:08.762861+00:00", "lang": "en", "topics": ["ai-policy", "ai-research"], "entities": ["NeurIPS", "ICML", "AlexNet", "Transformer", "Concrete Problems in AI Safety"], "alternates": {"html": "https://wpnews.pro/news/an-investigation-of-the-neurips-and-icml-2025-position-tracks", "markdown": "https://wpnews.pro/news/an-investigation-of-the-neurips-and-icml-2025-position-tracks.md", "text": "https://wpnews.pro/news/an-investigation-of-the-neurips-and-icml-2025-position-tracks.txt", "jsonld": "https://wpnews.pro/news/an-investigation-of-the-neurips-and-icml-2025-position-tracks.jsonld"}}