# The models have gone from unreliable, and thus professionally useless, to real-world-ready in 12 months

> Source: <https://markcarrigan.net/2026/11/19/the-models-have-gone-from-unreliable-and-thus-professionally-useless-to-real-world-ready-in-12-months/>
> Published: 2026-11-19 12:58:29+00:00

**Recommendation:** Benchmark frontier models against human professionals on end-to-end real-world tasks, not isolated steps, and re-run those evals every few months since pass rates moved from roughly 50 to 100 percent in 12 months.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

This from the FT’s AI Shift newsletter captures concisely something I was rambling about in a workshop earlier this week:

Last month, researchers at AI training and data company Mercor published [the results of an experiment \[email.newsletters.ft.com\]](https://urldefense.com/v3/__https://email.newsletters.ft.com/c/eJyMkT2O3DoQhE8jZi3wTxQ7YLDJABu_5wM0yeaOvBI1JikPfHtjDNhOjQqr8KGAL9Hgj7P9CJWffecxuIkcSrRYtOCgVlwkKqe84IO2_T2HnJg8lgLa6ATWGwsRpYbonDTJG07I4h6KMkzeoZLozJrMkuSK5hVaMvkotqCldkopVNoY6WfyETNizotJbJSbrPx7qs9lzOk8xB7uYzz6ZN4mfZv07fl8zge3dLZXPelb3M-PSd_u10EVInXet8odytkgck33g9pnB9qgnk84r_HgVs52dPh61e1sQCmdVx1UR5_0TexbH-85OJ_JRkvFolFKRqWUXzMW8WhnvtII_UHtU7Twws-JWts-qE5WHlTTnfvgNlOar0_R-NvFv5gSS14XXsGsWoH16ACzteCMKj5lH9e8_J5z_o9r_n87_jhZvBMjXNeWJ_P2T1JGo9opje2s7zkkIowkE2DxCmyMFmIpDlbvDBrlUbMXV-f25dpyiOtibfQatFQEFs0KEUlCcmVZF2Pcop34HvTPAAAA___5Qrcu__;!!PDiH4ENfjr2_Jw!AP52Ip2ccrQuLMSPbSRzyuWOPLUONxKhDNnRiVNwmgpbVcN-NzwCFzXpSpUJ-u-dMlhrfclTU_B7iDjCx6psYa6-bg$) where they pitted junior accountants against the latest AI models in tackling real-world accounting tasks. The outcome was a decisive victory for AI. Every one of the 20 tasks attempted by Claude’s Opus 5 model was completed flawlessly end to end, while the average score by the human juniors was 37.5 per cent and the highest 90 per cent.

As notable as the result itself is how quickly it has changed. That 100 per cent passing rate for the leading frontier model in mid-2026 was closer to 50 per cent in mid-2025. The models have gone from unreliable, and thus professionally useless, to real-world-ready in 12 months.

The agentive competence of Opus 5.5 is terrifying. I’ve been consciously trying it on every ethically acceptable work task I can think of – thus far it has not failed on a single one. Where the result has been weak, it’s been rapidly corrected through a little bit of engagement. In many cases these tasks have been one-shotted which I honestly believed was pretty much impossible.

I increasingly suspect the shift which took place in the last 12 months will be seen as more significant than the first few years of model development combined. Something has changed which has for the first time left me seriously existentially worried in spite of the intuitive scepticism which pretty much every sociologist naturally has to the doomer case.
