[ Introduction
](#introduction) In this piece, I present several reasons for being an AI sceptic before making an argument in favour of the use of LLMs in software development. I intend to show that while AI scepticism is valid, it should not crowd out the genuinely powerful capabilities that generative AI brings to the software development domain; capabilities that I, too, am now adopting, albeit with a number of significant guardrails that I will explore in more detail below.
[ First, some home truths
](#first-some-home-truths) When ChatGPT first launched , back in late 2022, I was not its greatest fan. Perhaps because of my proximity to the world of machine learning as a data engineer of some 20 years standing, I felt that a lot of the gush that accompanied its release was due, in large part, to the general public finally getting a glimpse of what generative pre-trained transformers can do and believing it was some kind of miracle. It was not, and I remain steadfast in my opinion that this remains true. You don’t have to be Arthur C. Clarke to know that scientific advancement, especially at speed and scale, is still science, even if some of its consumers fail to distinguish it from magic.
Large language models (LLMs) are impressive. But like any technology they have both limits on what they can do and flaws that only become more apparent the further they are pressed to operate outside of those limits. For all that LLMs have advanced over the last several years, I do not believe any of that has changed, even if their abilities, along with the number of domains in which they are able to operate, has significantly increased. When used with sufficient care, attention to detail and safeguarding, they are able to carry out vastly complex operations with only natural language as an input. Anyone who claims that this is not true is, in my opinion, choosing to look the other way. As it happens, I have a lot of sympathy with that position. It isn’t difficult, after all, to find reasons for disliking generative AI.
[ Why people hate generative AI
](#why-people-hate-generative-ai) First, the techbro billionaires behind the generative AI boom of the last four years are, to a man (and they are all men), very much the kind of people we’d like to strap to one of their beloved rockets and fire off into space. From Altman to Zuckerberg and all the others in between, they are living proof that money, even in vast quantities, cannot buy basic humanity. It is something of an irony that these same techbros like nothing more than to launch a product with the words “imagine if you could…” only to then follow it up with something monumentally tone deaf or bone-headed . I cannot remember a previous technology whose leadership was so united in its moral bankruptcy, even when judged by the admittedly flawed Silicon Valley standards that gave us NFTs, the metaverse and Theranos. That the same people making wild claims about GenAI on LinkedIn were, only a few years prior, running their own crypto-scams is no coincidence. And wanting to have nothing to do with these people is not only understandable, it is righteous.
Second, the environmental concern that accompanies the build out of data centres and every single generative AI query sent to them for processing neither can nor should be ignored. The energy and potable water consumption is staggering, and carries an even greater cost for the communities upon which these data centres are inflicted. The grassroots protest groups that have sprung up to push back on local authorities for authorising the build-out of such climate-change-maxxing projects should be commended. There really is no satisfactory answer to the impact GenAI is having on the planet, and the fact the big tech firms are being allowed to get away with this is a criminal failure of government.
Third, there is the hype. The remorseless, 24/7 hype that is blowing generative AI out of all proportion. It is very easy to see why some people hate it when, day after day, they’re told it’s going to take their jobs . Not the jobs we’d be happy to wave goodbye to, but practically every kind of human creative endeavour we’ve got: music, books, the arts, cinema, practically anything we do for our own entertainment is apparently under threat from AI, if the relentless social media hucksters are to be believed. And knowing that this is only made possible because the frontier AI labs were given carte blanche to hoover up every piece of creative output we’ve ever produced as a species only serves to rub salt into an already weeping wound. Technology was supposed to set us free to pursue our own interests; but the hype train is pushing a message that these, too, are going to be taken away by machines. What makes the hype more infuriating is just how much of it is premised upon what AI “might” do in the future. Have you ever looked at one of the many AI-generated “franchise fighty man takes on other franchise fighty man” videos? There’s a reason people call it slop , and it’s not simply because it looks like a bad 90s era video game cutscene. It’s derivative , lowest common denominator , soulless garbage . And yes, humans are just as capable of producing any number of things that might fit that definition, but only AI can produce it at scale. And we’re drowning in it .
Fourth, and perhaps most significantly of all, there’s the economic argument . Generative AI simply does not make any sense, economically speaking, because none of the AI labs are making any profit from it. Meanwhile, eye-watering amounts of private capital are subsidising them (and us, as users), even as money furnaces like OpenAI scramble to raise ever more billions at each new financing round. Anyone imagining all this money is somehow going to bring about the advent of AGI has little understanding of how LLMs work. The singularity has not arrived , is not about to arrive and may never arrive. What may very well be about to arrive is a financial reckoning when Wall Street grows weary, private investment dries up and the AI labs have to choose whether to face the music in the form of a public IPO. That could very well lead to the kind of scenes we saw when the bottom fell out of the subprime market in 2008 . While I do not believe that this will doom generative AI as a technology, it will nevertheless have a huge impact on how it is deployed and the domains within which it will persist.
There are, of course, other arguments about what the long term effects of all this might be, including on our own cognitive abilities , our mental health and how we solve the problem of educating the next generation of “AI natives” for whom the solution to every assignment is just a prompt away. We have not come anywhere close to facing up to this, even if the same can be said of other technological paradigm shifts, including the progenitor of generative AI, the internet itself.
[ This is not 2022, it’s not even 2025
](#this-is-not-2022-its-not-even-2025) All of this can be true, however, without concluding that the underlying technology is useless. You can agree with some or all of the above, but it does not change what LLMs can do, even if it might (and, I would argue, should) colour your judgement of how they are used and applied. I do not believe LLMs are a universal panacea that can be injected, without consent, into every government, industry and organisation across the planet to good purpose.
But I do think they show some significant promise in the domain closest to me, namely software engineering.
That may come as a shock to anyone who has asked for my opinion on the subject in the last few years. I experimented with early AI coding assistants when they first came on the scene and quickly determined they were hopeless. Even with clear, specific prompting, the results could vary wildly from “OK in principle” - even if the code itself might quickly become unmanageable spaghetti - through to “just plain wrong”, including calls to APIs with spurious arguments or, sometimes, APIs themselves that did not exist. If this was the future, I thought, then it was not one I needed to worry about.
The weird thing is that not all of this has changed, but that what has changed is too big even for an erstwhile AI sceptic like me to overlook. AI coding assistants do still get things wrong and they absolutely can write code that is a nightmare to support - and not always because of inherent faults in the prompts they have been given. But they are equally capable of producing fully fledged, production-grade applications that work, and work well. The difference lies in the combination of the advances made in the latest frontier models - especially those produced by Anthropic - and the ways in which experienced software engineers have learned to use them to best effect.
[ Vibe coding for me, but not for thee
](#vibe-coding-for-me-but-not-for-thee) In a way, it’s not that surprising that LLMs should be good at writing code. Programming languages have always, since their inception, been designed to produce predictable results. Unlike natural language, which is imbued with all kinds of nuance and context, programming languages are prescriptive and the meanings of keywords and functions are precise and consistent. LLMs, whose underlying fundamentals rely upon training data and probabilistic mathematics, can and should be good at writing code when given sufficiently clear instructions. That they do still hallucinate, or produce code that doesn’t do what was asked for, is nothing more than a known known, something we must price in to any work we do with them. But unlike using an LLM to do research for you on a topic you know little to nothing about, and thus will struggle to fact check, using one to write code for you is different. Code can be tested, deterministically. This means that in theory you don’t have to know how it works, but rather whether or not it does work.
Now, before the elders of software engineering brand me a heretic for seeming to suggest it is not important to know how code works, I should stress that if you don’t already know what good code looks like and how it should perform, LLMs aren’t ever going to produce anything beyond toy applications for you that fail the rigours and demands of any self-respecting production environment. The rather unfortunate epithet of “vibe coding” has come to mean a lot of different things, but if we treat it as the term was originally intended, then I am far from being one of its proponent. Simply opening up an AI coding assistant and throwing some vague prompts at it will not deliver you a coherent, robust piece of software, even if the first impressions looks passably good. It is for this reason that I am firm in my view that using LLMs for proper software development is something that can only realistically be done to any reasonable quality standard by those well versed in the discipline of software engineering and architecture. This is not some self-perserving attempt on my part to gatekeep, but rather a statement of the facts. You need only spend a few minutes reading through Reddit to find any number of vibe coding horror stories .
[ Why testing has never been more important
](#why-testing-has-never-been-more-important) One of the winners in the advancing LLM coding revolution happens to be a software methodology that I have always supported since first learning about it many years ago, namely Test Driven Development (TDD). There are plenty of books and articles on this topic, and I don’t propose to recapitulate all of them here; but in a nutshell, TDD states that you should not write code to do something before you have written the tests that will prove that it can do it. Not only does the mental exercise of thinking through all of the things that need to be tested aid in the overall design of what you are going to build, thereby making each feature that much more targeted and specific, but it also means that once you have built it, you have the tests ready to go in order to prove it works. And it doesn’t stop there, of course; because nothing in software is ever static. Changes to existing functionality are a fact of life, and they’re also - not coincidentally - a big factor in why vibe coding projects tend to fail over time. Trying to bolt on additional functionality after the fact when you haven’t understood how the original feature works leads to layers upon layers of gradually conflicting and incoherent code; sooner or later the entire edifice collapses. By this stage, your only real option is to start again.
With TDD, however, you innoculate yourself against the problem of features in flux because you already have the tests to check that those changes don’t then break the original functionality that came before them. Furthermore, each change necessitates new tests - or potentially amendments to existing ones - before any new code gets written. You effectively put yourself ahead of the labour to come, and by doing so give yourself a fighting chance of it all working out, in the end.
Applying TDD to the AI-assisted software development lifecycle brings all of these advantages, but it also comes with perhaps the single greatest benefit of all: you can keep the LLM honest. I don’t mean to anthropomorphise these things, they are machines and I don’t have any patience with anyone claiming they are sentient or have personalities. The honesty I am referring to here relates to checking their work. If you start from the premise that you need tests to validate what is built, well, each time you then ask the LLM to write more code, you can run those tests and confirm to your own satisfaction that the code really does do what you wanted. If one or more tests fail, then you know that the LLM has made a mistake - and you can either drop into the code and fix that mistake, or you can use the results of the failed test as a prompt to get the LLM to fix it. It comes down to what you’re most comfortable with, of course, but the tests themselves do not lie. If you have them, then you have a safety net. If you don’t, then you’re always going to have to take the machine’s word for it, or else read every last line of code it writes and your future quickly becomes one of eternal pull request reviews. I cannot speak for every software professional, but reviewing code forever without writing any sounds pretty soul-destroying to me.
[ Plan, revise, document
](#plan-revise-document) Thinking of AI coding assistants purely as automatons that help write code for you is to unduly pigeonhole their capabilities. While writing code is the thing that gets them all the attention , their capacity to work with you as a collaborator and reviewer is something I have personally found to be very rewarding. Building software of even moderate complexity needs a watertight design, and even the best of us rarely manage to produce such designs in isolation. If you have the luxury of a team, you can of course brainstorm with them, but if you’re working solo, it’s essentially you against the world. With a decent frontier model, however, you can easily get a review of your first draft, and request for it to be as merciless as possible. What you will get back is a mixture of useful feedback, one or two areas it probably didn’t understand properly (so you might need to tighten up the prose), and a selection of other items that prompt you, the human architect, to think more deeply. This, in turn, leads to a better design; and that review cycle can run and run. I’ve recently been working on a project that went for ten rounds of LLM-led review bfeore I got to a point where I was happy. Not only that, but since the LLM will be helping you to build it, you can then ask if there are any outstanding areas where it might have to take an independent decision, owing to a lack of precision in the design. Plugging gaps like this helps go a long way towards ensuring the LLM doesn’t take any flights of fancy or drag in a dependency you did not want to rely upon. You can be as prescriptive as you like, not only in terms of the tech stack and languages it is permitted to use but also which libraries and packages it can use or should avoid. No more unthinking invocations of Node.js or other multi-gigabyte runtime environments to execute a simple form validation, for example.
The output of all this back and forth should be a suitable markdown file - or even a selection of them, depending on how complex your project is going to be. These act not only as documentation for the project, but also as invaluable inputs for any fresh context window you open up with your LLM of choice in the future. Relying on chat history and the coding assistant’s own compaction is a recipe for context poisoning , since you lose all control over what it deems relevant and irrelevant to future sessions. And if it decides that a hallucination is relevant, it can then get baked into the foundations of your project, causing serious, ongoing harm. Keeping everything in markdown files that you control, by contrast, means you can be the arbiter of relevancy rather than the machine.
The markdown doesn’t end here, of couse. As you work through the project, you can ask the coding assistant to document what it builds and be as explicit as you like in terms of what style of document it produces. If it’s an area you already know well, you might only require a concise, high level overview; conversely, if you’re stepping into new terrain, you may want to get it to document what it’s doing in more detail. I treat this material not simply as documentation but as inputs for my own continuous learning.
[ Dream big, act small
](#dream-big-act-small) Plans, designs, scoping documents: all of these are familiar inputs to any software project, regardless of whether LLMs are involved or not. But just as you wouldn’t hand over that stack of documentation to a team of engineers and walk away, believing you’ll be able to come back a few days later to find exactly what you asked for , neither too should you expect to behave this way with an LLM. You need to break down the design into the smallest possible building blocks, with each brick being a separate item for the LLM to work on. Define your tests for that brick, then get it to build the brick and run the tests to confirm it really is the brick you wanted. Sometimes these bricks are going to be of little utility on their own, and will need further bricks in combination before they can really do anything meaningful. At other times, you may find that what you build along the way has the makings of its own tool, or a reusable module that could work in other, future projects. By sticking to the principles of TDD, these utilities you create on the journey can be isolated and managed in their own repository going forwards, avoiding the need to re-roll the same component again later on. Instead of thinking of your project as a monolith, think of it as comprising many small stones, with each stone being capable of standing on its own.
Nothing I am saying here departs from sound software engineering principles, of course, but it is my observation that they are all too readily forgotten in the rush to embrace the one-shot pipedream of an applcation produced from a single prompt. Even if that prompt might, in practice, be several pages of markdown, don’t try to one-shot it. You will fail to understand how it works from the ground up if you take that approach, and as a consequence you will not only struggle to make sense of the final product, you’ll also have a much harder time expanding and enhancing it with new features.
The single greatest risk an LLM poses to any software project lies not in writing bad code, but in eroding the ability of its human architects to think through and reason about the thing they are creating. LLMs might save you time, but that should never be done at the expense of your own understanding. If you trade that away for the sake of speed, you can no longer in good conscience call yourself a software engineer.
[ So, where now?
](#so-where-now) This piece started out as a desire to explain how and why my thinking about LLMs in the domain of software engineering has changed, over time. What I hope it makes clear is that you can remain sceptical about GenAI without committing yourself to a position of wilful disregard for what it can do. Conceding some ground to their capabilities does not oblige you to board the LLM hype train and start posting vacuous nonsense on LinkedIn exhorting everyone to “let that sink in”. At the same time, what I have learned is that adopting a refusenik stance is no longer a viable option, at least not for me. AI coding assistants have already changed how modern software is being built, and I either accept that and shape how we use them most effectively, or else opt out of that world altogether, and I’m not about to do that. Perhaps the frontier labs will be brought to ground by the forces of capital markets, but if they are, I cannot foresee the technology itself vanishing from the face of the earth. As a species we’ve never yet found a way to successfully uninvent things, short of total civilisational collapse.
I enjoy programming, and always have done. I started when I was 4 or 5 years old and it is as much a hobby for me as it is a job. The extent to which I use AI coding assistants at work is largely out of my hands, but using them at home is very much a conscious choice. When it comes to retro computing, in particular, I have no appetite to use LLMs because they would only get in the way of my own personal enjoyment. There is a beautiful kind of cognitive satisfaction that comes from writing and debugging your own code, and I am not about to give that up.
In closing, I’d like to state how much of a debt I owe to Robert Smallshire , whose recent talk at the National Museum of Computing earlier this year turned my own thinking on its head. His talk and the things he has shared since have all greatly contributed to how I now think about generative AI in software development and, as a consequence, what I have written here.
But I did not write this piece to persuade anyone of my position, I wrote it simply to state what that position has become.