Fast on the heels of my last post about a SETI@home for AI Assisted Research, I decided to get a prototype concept in place for at least the portion I could reasonably address: curation of research-focused AI prompts. I decided to call it Prompts for Progress.
To kick this off, I set some agents loose on the web to collect published prompts from AI-assisted math and science research. I’ve put them in one repo, alongside the problems and outcomes. Perhaps there’s something to learn from how people are prompting across these domains.
There is a static site for casual browsing, but the underlying repository can also be cloned for local analysis. The archive currently includes:
Full or partial prompts when publicly available and permitted for inclusion
Context about the research problem and the people and systems involved
Domain categories such as mathematics, physics, biology, and computer science
Outcome categories including complete results, partial progress, disputed claims, and documented attempts that produced no result
A problem index connecting multiple attempts aimed at the same question
A timeline showing the growing number of documented attempts
A machine-readable prompt corpus for aggregate analysis
At this early stage, the archive contains only 23 records. Coverage is necessarily incomplete, and both the record structure and taxonomy are a preliminary best guess. Some entries concern well-known results, others document attempts that did not succeed. That second category may ultimately be just as useful, since unsuccessful and intermediate research interactions are rarely visible.
The broader question is whether putting prompts, problems, and outcomes in one place makes it easier to study how AI-assisted research is actually being conducted. It may reveal recurring prompting strategies, differences between disciplines, or simply better ways to approach these systems.
Credit goes to @OpenAI for publishing their prompt used in the Cycle Double Cover Conjecture. It was far from the first published research prompt, but something about the significance of the result, paired with the craft of the prompt itself, seemed to resonate with people, including me.
The little hype wave that washed over the AI/math/tech world caused by the CDC news left me wondering how many people immediately modified the prompt to a domain of their own interest and gave that structure a shot? We don’t know, but I think we could learn a lot from sharing that kind of information. You can see my own completely speculative attempt to apply it to an open problem in general relativity here. It produced no result, but that is part of the point.
The repository is public, and corrections, missing cases, and new documented attempts are welcome.