Member-only story
I almost did not write this piece. #
Here,s the Friends link . . . . Every week there is a new model claiming to be the best coding assistant on the planet, and every week the benchmark charts look identical: a bar graph, a green arrow, a headline that says “state of the art.” I stopped trusting those charts a long time ago. So instead of reading another leaderboard, I opened three of my own Python projects and handed the same broken code, the same messy repo, and the same vague ticket to three very different models.
The contenders were Gemma 4, Google DeepMind’s open weight model that you can run on your own machine, Claude Opus 5, Anthropic’s new everyday workhorse model released this July, and GPT 5.5, OpenAI’s flagship released back in April. One is free and local. Two are paid and cloud hosted. I wanted to know if the price gap actually buys you anything when the work is real instead of synthetic.
A quick note before anyone asks: yes, GPT 5.6 shipped publicly in July. I stuck with GPT 5.5 here because it is still the version most teams have actually rolled out in production, and I wanted a fair fight between three models that developers are using today, not the newest…