Consider this sorting function:
from llm_api_of_your_choice import setup_llm_api
llm = setup_llm_api(API_KEY)
def llm_sort(words):
prompt = f"Given these words: {' '.join(words)}, sort them alphabetically. Reply only with the words separated by spaces."
reply = llm(prompt)
return reply.split()
No sane person would ever use this, right? Even bubble sort would outperform an LLM on this task in speed, accuracy and reliability. It’s quite simple to see why. An LLM burns through a massive amount of compute to do something a first-year algorithms course solved decades ago. Worse: it might even get the answer wrong! Even worse: the API might be down! Not to mention if you are not self-hosting your model your data is now being transmitted to a random third party with a vague clause in their privacy policy.
Now tell me: WHY REACH FOR AN LLM FOR EVERY DATA PROBLEM, YOU BOZO!?!
The sort function above is a joke. Don’t turn your production system into the same joke with a higher AWS bill just because your data is slightly messy.
Okay. Rant over.
The Rot #
In the age of LLMs, a wicked temptation has taken hold. Instead of reaching for proper engineering judgment, the default has become reaching for an LLM. A dataset needs cleaning? Ask the model. Two records need matching? Ask the model. Some text needs extracting? Ask the model. Something needs validating, routing, deduplicating, normalising, clustering or searching?
Ask the model.
Have you considered that maybe, perhaps, perchance, the magical text box is not the right tool for every job?
LLMs are not useless, quite the contrary. They are amazing pieces of technology when applied to the right problems. What happens though is that they are flexible enough to look like the right solution in places where they are the slow, unpredictable, hard-to-debug option. Likewise, they make bad solutions look sophisticated because they are great at making demos look good.
But production systems do not care about your demo in your AI-generated notebook on five cherry-picked examples that only needs to work once in your presentation to the CIO.
They care about cost, latency, throughput, repeatability, inspectability, observability, reliability, edge cases, failure modes, and if the same input gives the same output tomorrow morning.
LLMs do not understand your business logic, edge-cases, your taxonomy, your customers. You do. You might say: “Just fine-tune it, bro.” Sure, “bro”, but if the plan starts and ends at “fine-tune it,” and nobody in the room can explain what data you need, what labels actually mean, what a loss function is, and what failure modes you are willing to tolerate, then you do not have a solution. You have an even more expensive demo.
The What Before the How #
Next time you are faced with a choice, don’t ask: How can we use an LLM for this? Rather, try to understand your problem deeply and ask:
What’s the smallest, most reliable tool that solves this problem?
You will end up with something that outperforms an LLM in speed and accuracy, is more inspectable and reliable, and can be built on a laptop in a week.
Sometimes that tool is a regular expression. Sometimes it’s a lookup table or a join. Sometimes it’s a text-distance algorithm, a sparse retrieval model, a dense embedding model, a search index, classifier, rules engine, parser or better data. Sometimes it is doing it by hand.
And yes, sometimes it is an LLM.
Knowing which of these to reach for is the job. That judgment is what separates engineering from autocomplete. Please just don’t let LLMs become a default.
Sure, nobody is raising money by saying: “we used a join.” Have you considered that boring and correct might be enough? You work for an insurance company. Don’t turn it into Big Data Startup Enterprises Incorporated Limited.
Don’t use a sledgehammer when tweezers will do.