{"slug": "llms-in-professional-software-engineering", "title": "LLMs in Professional Software Engineering", "summary": "A single engineer at Solvares Field Service rebuilt a legacy distance-matrix service for large Vehicle Routing Problem instances in 3 weeks of full-time work using roughly €1200 in LLM tokens, according to a talk at the Waterkant Festival 2026. The service computes a full distance and travel-time matrix for 1,000 locations in under 100 milliseconds — one million routes at roughly 100 nanoseconds each — a task the engineer said earlier LLM attempts could not handle because the models produced unusable changes. The engineer's structured approach was to pick a hard problem, secure unlimited token budget from the CTO, and systematically test LLM usage patterns in March 2026.", "body_md": "This post is based on my talk at the Waterkant Festival 2026.\n\nThe AI hype is as ubiquitous as it is annoying. Some say that AI will beat software engineers at every task in a few months. Some say that human programmers will always be better than AI.\n\nObviously, both extremes are wildly wrong. But what does it help to say \"the truth is more complicated\" without actually figuring out what this complicated bit is?\n\nLike most software engineers, I care about solving real problems in the real world.\nUnsolved problems.\nHard problems.\nThe stuff that tickles your mind, and that requires some novel solutions and the right trade-offs.\nAnd I think it's fun to do excellent work and build something *outstanding*.\n\nCan LLMs even help me at all?\n\nI have tried to use LLMs numerous times throughout the years. Every time I found that they produce slop and waste my time. They either forced me to accept bad changes, or they made me fix up their changes manually, which was slower than just doing everything by hand.\n\nIn early 2026, this changed. Time to collect some evidence.\n\nInstead of just using the latest hype tool, I wanted to understand this properly. Let's take a more structured approach then:\n\n1. Take a properly hard engineering problem.\n2. Get your CTO to pay for infinite tokens.\n3. Systematically try out various ways to use LLMs, and write down how it goes.\n\nThis is what I did in March 2026, and here is what I found out.\n\n## \n\nAt [my current employer](https://solvares-fieldservice.com/), we solve large instances of the Vehicle Routing Problem ([wiki](https://en.wikipedia.org/wiki/Vehicle_routing_problem)).\nOne reasonably hard problem from this domain is that we have to compute a distance matrix as input to our smart algorithms.\nWhat's more, our distance matrix implementation had long suffered from heaps of legacy code and stability issues, making it a good candidate for replacement.\n\nNow, what is a distance matrix, you may ask?\n\nLet me explain.\n\nEssentially, we want to have a web server with a single endpoint.\nAs **input** (request body), it accepts an array of locations as lat/lon coordinates.\n\n```\n{\n  \"coordinates\": [\n    { \"lat\": 54.0, \"lon\": 10.0 },\n    { \"lat\": 54.1, \"lon\": 10.1 },\n    { \"lat\": 54.2, \"lon\": 10.4 }\n  ]\n}\n```\n\nThe server then looks at the world's road network and computes the best routes from each point to each other point, and returns only the distances and travel times as a large matrix. We do not return any information about the routes themselves, except for the length and duration.\n\nConceptually, the **output** (response body) looks like this:\n\n```\n{\n  \"distances\": [\n    [0, 18201, 55879],\n    [18204, 0, 32444],\n    [61390, 38199, 0]\n  ],\n  \"times\": [\n    [0, 1319, 3121],\n    [1343, 0, 2279],\n    [3414, 2670, 0]\n  ]\n}\n```\n\nNote that from each point to itself, the distance is zero metres (and the time is zero seconds).\nDue to one-way streets, turn restrictions, etc, going from A to B is only *approximately* as far as from B to A.\n\nWhat makes this problem so hard is the extreme performance that we require.\nFor example, **for 1,000 locations we have no more than 100 milliseconds**.\n\nThat's right: *one million routes* must be computed and measured, and the results must be encoded and transmitted over the network and parsed by the client, and all of this must happen in *less than a tenth of a second*.\nEven if we ignore all the networking, we only have 100 nanoseconds to compute each route.\n\nThat seems impossible. I guess we can agree that this problem is reasonably hard. Vibe-coding this cannot work.\n\n## \n\nI don't want to go into the details of the sophisticated algorithms and insane optimisations that were needed to pull this off. After all, this post is about how LLMs helped me, not about how the system works. But here is the data on what it took to build this service:\n\n- 3 weeks of regular full-time work\n- by a single person (me)\n- with around €1200 in tokens\n\nAround half of the time (and the tokens) was spent on the actual design and the rust implementation. The other half was spent building testing and benchmarking tooling in order to evaluate the solution, as well as to integrate it into our existing infrastructure.\n\nHere is the p90 performance data for our `c5a.4xlarge` instance on AWS (8 physical AMD Zen 2 cores):\n\n| N locations | N*N matrix cells | time to last byte | \n|---|---|---|\n| 3 | 9 | 0.8 ms | \n| 500 | 250,000 | 31 ms | \n| 1,000 | 1,000,000 | 75 ms | \n| 5,000 | 25,000,000 | 602 ms | \n| 10,000 | 100,000,000 | 4,837 ms | \n\nThat's pretty solid! Computing 100M distances and travel times in under five seconds on a regular 8-core machine can be counted as a success.\n\nAlmost everything is LLM-generated. Out of approximately 15,000 lines of code in total, I think I wrote 4 manually and the other 14,996 or so with an LLM.\n\nAlong the way, I wrote a detailed log of what I tried, what worked, and what didn't.\nI noted down *how* I tried to work with LLMs, rather than saying anything about which algorithms I tried.\n\nThis diary now lets us answer the key question of this entire blog post:\n\n## \n\nThe answer is … that it's the wrong question. At least, there is a better question to ask:\n\n## \n\nKnowing what an LLM is helps you characterize the kind of tasks they can handle.\n\nI mean this in an intuitive sense, not in a technical one.\nSome people say that LLMs are **next-token prediction machines**.\nThat's perfectly accurate but not what I mean.\nThis intuition is not very enlightening in day-to-day work, and it does not tell me how to change my workflow.\n\nInstead, I believe we should understand LLMs as **semantic translation machines**.\nThey translate an idea or a concept from one representation to another.\n\nThe term *semantic translation* needs some clarification.\n\n## \n\nBy semantic translation, I mean that the same underlying concept or idea is translated from one representation to another.\n\nThis goes beyond merely translating between two human languages (something that LLMs are obviously very good at). For example, if you have an English text, it can be translated to a matching English summary.\n\nHypothetically, if you have an English text that describes a program in sufficient detail, such as a line-by-line description of all the operations (corresponding to a given programming language), then LLMs will be extremely good at translating this specification to the actual source code in that language.\n\nSimilarly, if you have a lot of source code, an LLM can summarise it for you.\n\nWhat's common among these examples is that **the idea exists**, and the LLM **rewrites it** and lets you move to a different representation of the same idea.\nIt does not have to come up with anything substantial on its own.[1](#footnote-1)\n\nBut don't we all know that LLMs hallucinate?! Even for simple translations we can't be sure of the output!\n\nCorrect. The process is probabilistic.\n\n## \n\nEssentially, when you shoot your shot at a translation, you don't hit your target exactly.\n\nInstead, the LLM will give you output that is *very close* to what you wanted.\nThe translation has a bit of uncertainty that introduces a slight error.\n\nThere is a great deal to be said about reducing this error. For example, going from a lot of info to very little info works well, and the other way around generally does not. (Trying to restore the long English text from its short summary will leave you with tons of hallucinations, and false and inaccurate statements.)\n\nThat being said, I will leave the discussion of reducing hallucinations to other people. For now, it is enough to acknowledge that semantic translation is probabilistic.\n\nAnother way of looking at this is that every translation incurs a debt to the truth. You not only change the representation of the idea, you also slightly distort the idea itself. This leaves you with a different idea, not quite identical to your original one.\n\nWith every hop to another representation, you add a new layer to your stack of adjacent ideas. The more steps you take, the further you will remove yourself from the concept you started with.\n\n## \n\nIn contrast, here are a few things that are *not* mere translations.\n\nThis may sound obvious. If you want to find out what your customer wants, you should not ask an LLM. You should ask your customer.\n\nIf you ask an LLM about a fact, it will apply semantic translation to that question. The LLM effectively tells you:\n\nGreat question! People who ask these questions also make these statements about the topic …\n\n… and then proceeds to list “facts” that it may or may not reproduce from its training data.\n\nThis is its way of representing the question by an answer that is as similar as possible to the question you asked.\nHowever, the facts needed for that answer were not part of the question, so they cannot be part of the translation and have to be made up.[2](#footnote-2)\n\nFacts, requirements, or novel ideas<sup>[3](#footnote-3)</sup> cannot be LLM-generated.\nThey can only be LLM-translated.<sup>[4](#footnote-4)</sup>\n(This is especially true for things that did not appear often in the training data.)\n\n## \n\nLet's get a little more hands-on. We now have a good intuition for LLMs as semantic translation machines. But how exactly does this help programmers?\n\nThe thing is, semantic translation can happen in several steps, and combine several data sources. For example, a kind of prompt that works very well might contain\n\n- a source file name\n- a problem description\n- a brief sketch of a refactoring plan\n\nand then the LLM can perform the following steps of semantic translation:\n\n1. source file name → a `Read` tool call\n2. the problem description + refactoring plan → a set of refactoring steps\n3. source code from (1) + refactoring steps → list of `Write` tool calls\n\nIf step (2) is non-trivial, or if the refactoring plan in your prompt does not have enough details, an LLM can fix that for you.\nTake your initial prompt and let the LLM translate it to a few `Read` tool calls or web searches that give it enough context to write a better prompt (usually called plan mode).\n\nLLMs automate the grunt work.\n\nYou understand the problem, and you come up with the solution.\nThe LLM helps you get there faster.[5](#footnote-5)\n\nIn my case, doing a lot of performance work requires a ton of tasks that LLMs automate easily.\nInstrumenting code, running benchmarks, generating flamegraphs, sifting through endless amounts of performance metric data, and thereby finding bottlenecks are perfect examples of semantic translation.\nThose are *trivial* tasks for LLMs.\nOnce you know what the exact bottleneck is, it's usually straightforward to fix it and repeat the process.\n\nThat's how LLMs help.\n\n## \n\nA different intuition for LLMs is called *pattern recognition and recombination* machines (thanks to Marc Heimann for telling me about it).\nThe idea is that LLMs do not translate, but that they detect patterns that they can replicate and recombine.\nThat's arguably a more accurate description when you factor in the underlying technology, but I would not say that it is necessarily more intuitive.\nIf somebody interrupted me during programming and gave me a machine to recombine textual patterns, I would not know how to deal with it.\n\nI intentionally chose a very liberal interpretation of the word *translation*.\nIt includes things like translating `2 4 6 8` and `generate 3 more numbers!` to `2 4 6 8 10 12 14`.\nAdmittedly, this is clearly more of a pattern recognition task.\nMy stance, however, is that `2 4 6 8` and `2 4 6 8 10 12 14` both are different representations of the same concept (positive even numbers), and continuing the sequence is just picking a different representation of that concept.\n\n## Footnotes\n\n1. \nThe idea does not necessarily have to be part of the prompt. It can also be the result of a tool call. In both cases, it resides in the context of the session. [↩](#footnote-ref-1)\n2. \nThis is why you can essentially get the LLM to argue any position simply by phrasing the question a bit differently. [↩](#footnote-ref-2)\n3. \nNote that you can very well use LLMs for *brainstorming* ideas. Their ability to put things differently is great for changing your perspective on a problem, and thus getting creative. But either way, the ideas are generated by your brain, not the LLM.[↩](#footnote-ref-3)\n4. \nSometimes, you can take these tasks and turn them into semantic translation problems. For example, if your LLM has access to Google and Wikipedia, it can effectively translate your request for facts to a tool call to search the web, and then rephrase the info it found. This means that it's worth looking for ways to convert your tasks into those that LLMs can do well. [↩](#footnote-ref-4)\n5. \nAnother analogy I came up with is that LLMs are seven-league boots. They are amazing if you know where you want to go. But if you run in the wrong direction half the time, you end up exactly where you started. [↩](#footnote-ref-5)", "url": "https://wpnews.pro/news/llms-in-professional-software-engineering", "canonical_source": "https://knorpelsenf.me/posts/semantic-translation", "published_at": "2026-09-28 09:03:49+00:00", "updated_at": "2026-09-28 09:19:39.659666+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-tools", "developer-tools"], "entities": ["Solvares Field Service", "Waterkant Festival 2026", "Vehicle Routing Problem"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llms-in-professional-software-engineering", "markdown": "https://wpnews.pro/news/llms-in-professional-software-engineering.md", "text": "https://wpnews.pro/news/llms-in-professional-software-engineering.txt", "jsonld": "https://wpnews.pro/news/llms-in-professional-software-engineering.jsonld"}}