# OpenAI Researchers Rate AI Tools Highly for Routine Tasks But Flag Code Scaffolding Gaps

> Source: <https://epyhia.com/openai-researchers-rate-ai-tools-highly-for-routine-tasks-but-flag>
> Published: 2026-11-20 07:06:45+00:00

**Recommendation:** Assign agents routine monitoring and technical Q&A where OpenAI measured 79 percent and 72 percent success, but keep humans on inference, serving and eval-environment code, which succeeded only 25 to 29 percent of the time.
More in the playbook "Designing agent systems": <https://wpnews.pro/playbooks/agent-architecture>

# OpenAI Researchers Rate AI Tools Highly for Routine Tasks But Flag Code Scaffolding Gaps

OpenAI's own researchers say they would rather give up 40 percent of their team's headcount or 30 percent of their compute than lose access to frontier coding agents for three months. At the same time, they report that those agents fail at the complex engineering work that underpins the lab's most critical systems — writing inference and serving code succeeds just 25 percent of the time, and building evaluation environments 29 percent.

The findings come from OpenAI's second "research acceleration" report, published November 18, which attempts to measure how deeply AI agents have embedded themselves in the development of new models [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier). The company frames the exercise as tracking indicators of recursive self-improvement (RSI), where AI systems speed up the creation of their successors.

## Usage surge: token costs now exceed base salaries

By OpenAI's accounting, internal agent usage has grown exponentially. At public API list prices, the median researcher consumed $2,400 worth of tokens per day in October, up 62 percent from September. The top 10 percent of users hit $23,000 per day, a 105 percent increase [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier). Converted to eight-hour workdays, agents now run the equivalent of 5.7 workdays per researcher per calendar day — roughly eight for pre- and post-training teams. More than two-thirds of tracked researchers ran at least four agents in parallel at peak usage [\[1\]](https://the-decoder.com/openais-own-researchers-now-say-they-value-their-ai-tools-more-than-nearly-half-their-colleagues).

The theoretical API-equivalent spend was twice as high as researchers' non-equity compensation in September, though equity makes up 70–80 percent of total pay at OpenAI [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier). Internal agent use still accounts for only 2 percent of the company's total research compute [\[1\]](https://the-decoder.com/openais-own-researchers-now-say-they-value-their-ai-tools-more-than-nearly-half-their-colleagues).

## The jagged frontier: routine tasks succeed, scaffolding fails

Success rates on autonomous completion have risen sharply: from 23 to 46 percent for 2–4 hour tasks, and 13 to 35 percent for 4–8 hour tasks, between March and October [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier). But the aggregate masks wide variation. Agents monitor training runs successfully 79 percent of the time and answer technical questions 72 percent of the time. They struggle with the engineering backbone of model deployment: writing inference and serving stacks (25 percent), building evaluation and RL environments (29 percent), and launching training runs (35 percent) [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier).

OpenAI attributes the gap to a "jagged frontier" — capability gains are uneven, and humans remain central to the research loop [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier).

## Researchers value the tools, but flag "AI slop" and missing taste

In a non-representative survey of 101 research staff, the median respondent said their team would need three months to produce the same output without coding agents [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier). In a separate quick poll of 28 employees, 27 preferred the latest internal model over a mid-career human engineer assisted by a 2024-era model [\[1\]](https://the-decoder.com/openais-own-researchers-now-say-they-value-their-ai-tools-more-than-nearly-half-their-colleagues).

Qualitative interviews with ten researchers painted a less rosy picture. Interviewees said models lack any "taste" for good research ideas, code quality, or data quality. Agents get lost in unnecessary steps, make wrong assumptions in analyses, and submit subpar "AI slop" code changes alongside unusable data artifacts [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier). Close human oversight is described as essential to avoid wasted compute cycles.

One post-training researcher said: "I've tried saying, 'Just make this task like this other task,' and been burned too many times when it's not quite right and I need to make edits. Now I say, 'Go make one. I'll review it and make edits. Then go do the rest.'" [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier)

## Key ideas still come from humans

When asked to identify the most important conceptual ideas behind GPT-6 Astra and GPT-6.2 Astra, post-training and safety researchers said all named ideas originated with humans; models assisted only with implementation. Pretraining researchers reported the same for GPT-6 Astra. Only for GPT-6.2 Astra pretraining did individual researchers say they gained important insights through dialogue with Codex [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier).

## OpenAI's caveats and industry push

OpenAI cautions that historical trends from human–AI collaboration may not extrapolate linearly, noting recent abrupt breakthroughs in mathematics and the possibility of similar jumps in deep-learning research as models gain autonomy [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier). The company reiterates its call for labs to be required to track and publicly report RSI metrics, while stressing the report does not signal an intention to pursue full RSI [\[6\]](https://openai.com/index/research-acceleration-jagged-frontier).

The data offers a rare internal reality check from a frontier lab: agents are now indispensable for routine work, but the scaffolding that turns research into reliable systems remains a human job.

## Sources

1. [OpenAI's own researchers now say they value their AI tools more than ...](https://the-decoder.com/openais-own-researchers-now-say-they-value-their-ai-tools-more-than-nearly-half-their-colleagues) (the-decoder.com)
2. [Science at the Frontier: How scientists use ChatGPT](https://openai.com/index/science-at-the-frontier) (openai.com)
3. [OpenAI Researchers Consume $2400 of AI Per Day as Agents ...](https://finance.biggo.com/news/c7f9d612-6ec1-47d6-814e-b547a6634c52) (finance.biggo.com)
4. [OpenAI's top researchers now burn $23000 in AI tokens a day](https://cryptobriefing.com/openai-researchers-23000-daily-token-usage) (cryptobriefing.com)
5. [Researcher, Frontier Robustness](https://openai.com/careers/researcher-frontier-robustness-san-francisco) (openai.com)
6. [Research acceleration: The jagged frontier | OpenAI](https://openai.com/index/research-acceleration-jagged-frontier) (openai.com)
7. [https://cdn.openai.com/pdf/0e4b2466-0be0-4326-8bf7-117d809e14d2/Science%20at%20the%20Frontier.pdf](https://cdn.openai.com/pdf/0e4b2466-0be0-4326-8bf7-117d809e14d2/Science%20at%20the%20Frontier.pdf) (cdn.openai.com)
