You cannot sell AI written software

Software generated entirely by AI lacks the intentional design and domain understanding necessary for a viable commercial product. The author contends that developers who use AI to write code without making their own informed decisions cannot meaningfully solve problems or accept feedback, as they have not earned the knowledge required to understand the solutions they present.

You may have seen it too. This trend of “I wrote some software to solve a problem. I think it’s pretty great. Does anyone have any feedback?”. Maybe it’s a budget app. Or some company management thingy, tracking sales. Or invoicing. Maybe you take a look. It looks pretty slick. But then you get a feeling of uncanny valley. It’s just not right. Maybe you can’t even put your finger on it. I’m not an accountant, so when I see some accounting software do something in a different way, it’s interesting. Why is it that way? What can I learn from the fact that a professional thinks it should be this way? You already know what’s weird about it, if nothing else because of the title of this post. The software works this way because the LLM wrote it that way. There’s no reason. It’s not even wrong. How do you give “feedback” on that? My feedback would be that you don’t understand the problem you’re trying to solve, and have shown no sign you intend to understand it, so how could you possibly think you can solve it? You’re not asking for feedback. You’re asking for someone else to do everything. Analogies sometimes work, sometimes not. I’m not sure if this one will work. You have a house with stairs that go nowhere, because “houses have stairs, right?”. You didn’t put the stairs there, and you have no sense for why there should be stairs in any particular place, so sure, why not there? Contrast this with an actual architect. They know why they put the stairs there. They know why they didn’t put the stairs on the other side of the room. It’s not just that they have actual knowledge, it’s also that they actually had to think about “how do you get to upstairs, and what’s the experience of walking up those stairs?”. They had to make many small decisions, each of which required thinking, and required that the choices add up to a coherent whole. The AI put the stairs somewhere, and you said “sure, stairs”. They’re in the kitchen, but since you only had to OK the stairs, not actively choose which room they go in, it didn’t even occur to you that room selection was a thing. You don’t need architect credentials to do this right. You could have selected a room. It probably would have been a fine room. Maybe a professional could give you feedback on that choice. But if you don’t make choices then you’re not actually contributing to the thing you supposedly built. You made no choices, so you don’t understand the problem you supposedly solved. “If I may… Um, I’ll tell you the problem with the scientific power that you’re using here, it didn’t require any discipline to attain it. You read what others had done and you took the next step. You didn’t earn the knowledge for yourselves, so you don’t take any responsibility for it. You stood on the shoulders of geniuses to accomplish something as fast as you could, and before you even knew what you had, you patented it, and packaged it, and slapped it on a plastic lunchbox, and now you’re selling it” Except you didn’t even “read what others had done”. Not even that bare minimum. I use my own email client, written in artisanally hand crafted code, like back in the stone age. It solves a problem for me. I have my own youtube UI. And my own RSS reader. For a while I used my own SSH. I know people who wrote their own shell, and that is their daily driver. I plan to move this blog to my own webserver behind a custom SNI router. Those can get feedback. I’m not saying they’re great, but even if the feedback is “why would you trust OpenSSL more than OpenSSH?”, it’s useful. But honestly, if someone wants an email client kind of like mine, but customized to their preferences, then they may be better off asking an LLM to write their personal email client. That was not true 5 years ago. I don’t want to use email threading. Maybe you do. It’d be annoying to support both. I was having some glitches with my video conference, and wanted to troubleshoot. I couldn’t find a tool that did what I wanted exactly. gping came close, but not quite. I wanted something that pings regularly and frequently, summarizing these many pings about once a second, to illustrate subsecond network outages. And using Linux ICMP sockets, so that it doesn’t require root. I spent about as much time looking for the perfect tool as I then spent telling an LLM to write it. I had it write blipfinder, and it solved my problem. I could have had this tool written for me before my meeting had even ended. Two relevant points about this: Just for fun, I had the LLM write it again. That version was also acceptable. It made some different choices. Which choices were best? Meh, nobody gave it any thought. It can still be the right custom software for you. In fact custom software is a huge win for LLM. But selling that is not. 5 years ago there was no software in this uncanny valley. It was either obvious crap, or it was professional. 5 years ago your potential customers couldn’t replace you and your software with a prompt. But now? If you sell this slop then you don’t have customers, you have “marks”.