# How well does AI peer review work? > Source: > Published: 2026-08-11 06:57:16+00:00 # How well does AI peer review work? Claude and I planted 100 known errors into 10 open-access psychology papers and then ran them through frontier models and two commercial AI review tools. In brief: - The best single system caught 71 of 100 errors, while the worst caught 30. - Pooling every system’s output caught 93 of 100. Models are only partly correlated in the errors they find, making ensembling a big lever for finding issues in papers. Check your papers against multiple models! - Seven errors could not be caught by any system. All were omissions — information deleted from a paper rather than mistakes inserted into it. - Refine.ink contributes more unique catches than any other single system, though it’s expensive. - I didn’t measure false positives and I don’t know how this error distribution compares to the distribution of errors in real papers. - I’ve made the papers, errors, model outputs, and the full experiment log [public](https://github.com/Dawes-Institute/ai-peer-review-benchmark). I hope people can build on this work to create a comprehensive eval benchmark across disciplines. That is from Paul Litvak, here is [more](https://www.paullitvak.com/p/how-well-does-ai-peer-review-work?utm_source=cross-post&publication_id=2247864&post_id=203641239&utm_campaign=1010915&isFreemail=true&r=3o9&triedRedirect=true&utm_medium=email). Note that is not even using the very latest generation of models.