The manifestation of my imposter syndrome, for me and today, is what does it mean to be a software engineer. There’s a lot more noise than signal on the Internet about agentic engineering, what can be accomplished, and its implications for the future. The title I chose rather gives it away; it’s about choosing — carefully — all the things you need to choose when you’re solving the puzzles of software and systems development.
Beyond the hype and junkie-like marketing fervor of “major model providers”, I found a really interesting power tool with the combination of harness and models. I’ve been following how friends have been using these tools, and learning a ton. As usual, the folks doing some of the most amazing things aren’t the ones crowing about it, or posting narrative blurbs in social media about the end of this profession. They found a “big damn stick”, they’re exploring the fulcrum points, and they’re representing good ole Archimedes to lean into that lever, moving the world.
In the past year, agent harnesses crossed the “can it be done” rubicon. (yep, jumping forward to Roman references). I would not have wished for the world’s knowledge to taken without permission and regard, or the lunatics to delve into economic self-dealing that’s peanut buttering over the otherwise tanking US economy. The economic models for the large models aren’t viable from any report that I’ve seen, but the capability isn’t going away. Instead it’s shrinking (fast!). Open weight models are making (beefy) personal computers quite capable of doing the same. They’re not quite as effective, but the delta in time and capability isn’t large.
“Can it be done” is only the start, not even close to the majority a software or system engineer’s profession. It’s like when I learned to weld in my 20’s – I quickly created things that I couldn’t lift or even get out the door of the shop. (thank goodness for acetylene torches). What I learned then is I think the same lesson, different medium: How something goes together is what makes all the difference.
If you use agentic harnesses to develop with a bit of foresight, you can get not only “it works”, but also “it’s testable” (I heavily lean into the prompt “develop with red/green TDD”). But it’s not very solid much above that. The seams — how your code works, it’s “API”, and how it fits with other software — are as much art as science. It is made up of subjective measures that rely on your viewpoint (and experience, as well as your guesses) for both what you’re solving now, and how to live with that software over a long period of time. Making software debuggable, maintainable, layered, and composable – that’s still quite a trick. Quite a lot of that work requires extensive, thoughtful reasoning. And that’s where the LLM’s today, even the leading edge of the “capability” from frontier models, fall short.
It helps to know that LLMs don’t “reason”. They predict, and the models themselves are effectively written human knowledge compressed. So if it’s in human knowledge that was encoded, it can echo out the human reasoning. For agents focused on software development, those reasoning traces are the precious data for the models. There’s a very approachable research paper on just how bad LLMS are at reasoning called The Illusion of Thinking. There is some research I’m following that includes prediction of results of actions, but that’s not what we have today with coding agents. It’s a pretty different – and fascinating – area of research. If you want to explore, go digging on how “JEPA models” work, LeWorld Model, and recent talks by Yann LeCun.
While you’re working with LLMs though, there’s still a ton of ways to make them more effective. I think there’s a lot of advances that we haven’t even really begun to eek out. Most of the wins I’m seeing today involve providing it good, concise data to work from, at the right time, and providing deterministic validation tooling with natural language feedback that the LLM can use to correct itself. The amazing thing to me isn’t that it can predict what to write, but that it is effective at tool calling and following instructions. Another downside of this instruction following is what
[Simon Willison coined as the lethal trifecta]. Basically – LLM models can’t distinguish between good advice and bad. They’re foundationally incapable of always and consistently preventing prompt injection attacks. “Alignment work”, safety harnesses, and sandboxes all help to add barriers against the worst, but there are fundamental gaps. And frankly, something that tirelessly follows instructions without having good reasoning is nightmare fuel to me.
I hope there will be near-term nadvances in how models are trained to include the equivalent of reasoning traces for post-training (RLHF). In my ideal future, these include more of what it means to build software with clean interfaces, that’s debuggable, and and that’s maintainable as a key part of the reinforced evaluations. Carefully reviewing, planning, and fixing the seams of software (and systems) is one of the critical skills we both can, and need to, employ when developing software – with or without agentic assistants. And as I see the wave of “Oh, that’s easy to implement…” and people reaching for clankers to get it done, I think it’s more important than ever.
It’s a great time to be following folks who write, talk, and share about the craft of software, and how we can be better artisans. Hopefully it’s obvious, but there’s never a single answer — a panacea. It’s always about tradeoffs, choosing what makes sense for the problem at hand. With the help of a lot of great minds sharing their thoughts — both now and going back decades — we have a great tool chest for this work. It’s about picking, or reworking to move to a better choice, the right abstractions. It’s core is managing the cognitive load, learning which pieces we need to be stable, and where we want our work to flex and bend (and how).
And yes, I wrote the damn em-dashes myself. I’m too in love with a recursive parenthetical in my writing, and I like a break from commas and parentheses.