A developer used Codex to iteratively improve an AGENTS.md file across eight versions, measuring each against real pull requests on a five-task training set. The best candidate showed improvements in …
A regression check comparing GPT-5.5 high against a prior run on 21 tasks from the GraphQL-go-tools repository found the model resolved one fewer test, produced two fewer equivalent patches, and recei…
An evaluation of Anthropic's Opus 4.7 model across five reasoning effort settings on 29 real-world tasks from an open source GraphQL repository found that medium reasoning effort produced the best res…
OpenAI's GPT-5.5 Codex model was tested at four reasoning effort levels — low, medium, high, and xhigh — on 26 real-world coding tasks from the open-source GraphQL-go-tools repository. Low and medium …
OpenAI's GPT-5.5 outperformed Anthropic's Opus 4.7 and OpenAI's GPT-5.4 on 56 real coding tasks from two open-source repositories, passing more tests and producing patches that more often survived cod…