The case to BYOB: build your own (coding) benchmarks
A team led by @oh_an_opinion built an internal coding benchmark from its own merged pull requests and found that, depending on task complexity, cheaper and faster agent setups matched frontier Anthrop…