An Honest Review of AI Programming A developer's three-month trial of Claude and other AI tools for work found them useful for search and summarization but not for writing code, according to a personal review published on a programming blog. The author, who remains unnamed, warns against asking AI to write code, describing the technology as a large neural network for natural language processing rather than true intelligence. The review highlights AI's effectiveness in answering pointed questions and summarizing texts, while noting limitations such as hallucination and compute requirements. An Honest Review of AI Programming on C++ /tag/cpp/ , Game development /tag/gamedev/ It’s getting hard to avoid LLMs these days. Even if you avoid social media or at least curate a follow list that avoids the bulk of the slop factory and shrug off grandiose marketing statements that end up taken at face value in the news, it will likely come and find you at your place of work. Unlike the silver bullets of the past like microservices or NoSQL , AI adoption seems to have been mandated in many places from the top layer of management, regardless of how many of them ever worked or studied for an engineering job. I do admit that this approach immediately triggered my contrarian side and made me very defiant of any AI tool. I don’t believe someone who has never written a line of code in their life should be telling me what to use for my engineering job. This sounds to me like the most terminal case of micro-management, and that’s never a good thing on top of being personally insulting . Either way, over the past 3 months I got to use Claude and friends for work and I have to admit I found it somewhat useful. As long as you don’t ask it to write code. Please don’t ask it to write code. But I’m getting ahead of myself. Artificial “Intelligence” You probably heard this a million time by now, but artificial intelligence really isn’t that intelligent. It’s all marketing and buzzwords. All we really we have here is a very large neural network specialized in natural language processing. As it turns out a lot of what we humans do on the computer is use text to communicate both ways, that model can be used to parse queries, generate textual 1 responses based on a probabilistic heuristic or output command lines that can be then executed the old fashioned way and their results fed back into the model to make a loop until we reach some exit condition. That’s not to say this is inherently bad. But it’s not magical either. “Agentic workflow” or whatever they’re calling it at the time you’re reading this article is just the realization that every software problem can be solved by adding another layer of indirection, and LLMs are no exception. If the neural network output can be improved by providing more input, then attach more input. And if the best way to figure out which input that would be is to query the model to generate a command and then pipe it through system , so be it. But let’s focus on the user point of view for the rest of this article. We have Google at home… There’s tons of valuable stuff out there on the internet. And assuming we do a decent job at keeping existing records intact, the net total sum of public knowledge can only improve but, spoiler warning, there’s a caveat . The problem is finding it. This isn’t a new thing. I’m old enough to remember the time when you’d first get suggested to try this new “google” thing. But sadly it’s gotten quite worse since the 2000s golden age. They have been fighting an uphill battle against SEO for a while now, and it doesn’t look like they’re winning 2. Enter AI, as both a help and a hindrance. As a search tool, I have found it usually good at answering pointed questions expressed in natural language, especially given the conversational ability that allows refining the answer or bring up follow-up questions with the current context in mind. In practice it is the equivalent of running a bunch of searches, skimming through the top N links, and repeating until we think we’ve got the picture. Writing a short summary of a longer texts seems to be what LLMs are best at, and so it makes sense to use it to automate the process. Plus crawling and summarizing multiple searches is an inherently parallel job so it isn’t hard to see the efficiency that can be brought up by automating the process, assuming we have enough compute available and it doesn’t hallucinate the summaries, we’ll get back to both those points later because they are quite important. The hindrance counterpart is of course that LLMs have lowered the cost of flooding the internet with word salad that dilutes an already precarious sea of information. Freya Holmér published a very good video https://www.youtube.com/watch?v=-opBifFfsMY on the topic and impact of littering the web with AI generated content in the never ending SEO arms race. Searching for information by hand is still an option today, especially if you already know reputable sources on a particular topic. But if you’re not, then a lot of care must be taken to sort out the slop from the real data, and automating it with an LLM loop may not help if you can’t instruct it which sources to keep and which to discard. Even before we talk about hallucinations, an LLM generated summary can only be as good as the sources it ingested. …and now we also have Google at work A particular area where I found an LLM loop useful for searching, that I haven’t found much discussion about so far, is internal company knowledge bases. Wherever you work, chances is there is a combination of wikis, slack conversation, Google Drives, Confluence pages and whatnot where a lot of good information is sitting but no one ever seems able to find it. I’ve experienced some of those “enterprise solutions” to be so bad that sometimes I couldn’t find a page I browsed a week before even when I typed the title or what I remembered the title to be in the search bar. I have written my share of internal technical article that I’m fairly sure no one ever found since, unless they kept a bookmark from the time I first linked it in the tech group chat. In the past few months, and despite being new to the company, I have been able to find a bunch of answers that were written before my time. Because the same way “AI Google Search” can run multiple queries in parallel and summarize the answer, Claude and friends can turn my natural language question into a bunch of search queries for likely synonyms until they hit something. It proved really handy when I ran into a particular edge case with the engine and I could quickly search if someone ever reported it, offered a workaround or discussed why it had to behave this way. Every time you run into an issue in tech, chances are it’s been discussed before and you’re off a much better start if you can find that conversation. This is technically not new tech. One of the things that has made Google and friends efficient for so long is that they automatically generate synonyms when building keyword metadata for a page. Your company wiki or chat search functions likely do not 3. I suspect there is a lot of value sitting in year old slack conversation logs that are absurdly hard to access without an LLM to search for them, or a more veteran coworker being able to remember it and point you towards it. Now is it efficient from a tech perspective to run an expensive LLM to search through a company wiki when they could instead implement basic search engine techniques that are decades old at this point? No, I’m fairly sure it is not. Indexing the content the way Google did 20 years ago with would definitely be a much more efficient solution, compute wise. But from a user perspective it is much more desirable than trying to search for “UI” and getting no results because the plaintext reads “User Interface” in the page they are looking for. Hallucinations and false positives Once an LLM has found an answer, it’s usually a good practice to go look at the primary source. Reading that article, document or chat log will help ensure the answer wasn’t hallucinated. Hallucinations are an inherent property of how LLMs work. Token generation isn’t based on logic or truth, it’s based on statistics and from my understanding there is no way to entirely avoid them. I have experienced them on all models, from the cheap and pretty bad free version of Copilot that comes with Bing to the fanciest paid models of Claude. I have found them most common when asking a very specific question that hasn’t be answered before, like asking for a particular niche feature in CMake or Vulkan or Xcode. Instead of answering “no it can’t be done”, I got the most probabilistic answer which suggested I try to click a button that wasn’t there or enable a feature flag that didn’t exist. The whole thing is akin to asking someone who’s generally knowledgeable about a problem domain, but not the specific software or library you are using. They would reply “yeah that sounds like a thing you should be able to do” because it feels like a reasonable expectation to have and maybe is a feature in similar products. I suspect it’s the same reason why when writing code an LLM would try to call a non-existing API, because based on other languages it sounds like C++ containers should have a .sort function. Because it’s the kind of thing you can do in Python and C . That’s not the case because C++ makes a strong distinction between containers, iterators and algorithms, but LLMs do not reason despite their marketing calling it “reasoning” from first principles. While this could be partially remedied by always asking for a primary source or citation, I dislike the idea that one has to add magical incantations to their queries to get the right results. It’s a good laugh to make fun of “make no mistake” memes, until you start having to consider similar things seriously. Plus models seem to get a new release every year or less, which would probably require the user to revise all their rituals or watch them become pointless rituals that engineers do without remembering why . Another issue I noted is that with connectors it’s somewhat easy to get the LLM to start feeding itself. For example when asked to find prior mentions of a recommendation I was writing for a client, Claude was adamant that this was supported by past reports… until it turned out one of those was the one I was actually writing. It was easy enough to catch because I had the breakdown with sources, but had it just gave me numbers I could easily have created a self-reinforcing loop. Likewise if coworkers were querying the same database and found my work-in-progress report, their summaries may have taken it as gospel. Again, those robots are anything but intelligent, and often you need to lay out some very basic things to avoid really stupid assumptions. To finish on that topic, I also noticed a risk of telephone game happening with connectors that plug into another LLM. Once I got Claude telling me there was concrete evidence that a certain technical design was the result of a deliberate choice by the team, while it turned out it had taken the summary of another AI at face value, and the real primary source was two users speculating on why the module worked this way on the public forums. Again this is a tool that is good at summarizing data, but not always good at selecting which data to trust. Coding? Up until this paragraph my use cases have focused on research. But what about writing code? After all, it is the next big thing™️ and it’s coming any day now, isn’t it? Simply put: it’s not very good. While I have found LLMs useful for researching and planning code changes, my attempts at actually making them write code have been quite lackluster. I found them to be slow and expensive to generate, for a mediocre result. In one use case, after a long discussion, I asked it to make an optimization refactor where it would remove the Update method of a MonoBehaviour derived object, put those objects inside a List