# Ai2 releases a benchmark showing research novelty judges swing with prompts > Source: > Published: 2026-10-27 17:48:53+00:00 # Ai2 releases a benchmark showing research novelty judges swing with prompts **Researcher Noy Sternlicht and co-authors found one prompt change pushed GPT-5.4 from 93.5% accuracy to 40.9% on AI-generated idea comparisons.** By [Ryan Merket](https://runtimewire.com/author/ryan-merket) · Published Primary source: [Hugging Face Newsroom](https://huggingface.co/blog/allenai/novelty-bench) ## Why it matters AI-for-science systems may use language models to rank ideas as original. Ai2's results show that small prompt changes can reverse those judgments, so novelty scores need testing across setups before they guide research. A small prompt change sent [GPT-5.4](https://runtimewire.com/models/openai/gpt-5.4)'s accuracy at choosing the more novel research idea from 93.5% to 40.9%, according to a [new study](https://huggingface.co/blog/allenai/novelty-bench?ref=runtimewire) by [Ai2](https://allenai.org/?ref=runtimewire) researcher [Noy Sternlicht](https://noy-sternlicht.github.io/?ref=runtimewire) and five co-authors. The result puts a weak link in AI-assisted science under scrutiny: systems asked to generate research ideas may also rely on another language model to certify that those ideas are original.… When you purchase through links in our articles, [we may earn a small commission](https://runtimewire.com/affiliate-monetization-standards). This doesn’t affect our editorial independence.