{"slug": "stop-treating-llms-like-magic-and-start-treating-them-like-software", "title": "Stop treating LLMs like magic and start treating them like software", "summary": "A commentary argues that the engineering community is regressing into a \"mystical\" view of large language models, treating them as sentient entities with intent rather than as statistical software, and that this framing obscures concrete engineering problems such as KV cache bottlenecks, quantization errors from FP16 to INT8, and stochasticity that makes reproducible results difficult. The piece contends that anthropomorphizing models leads developers to accept a 10% error rate as an inherent trait of a \"digital brain\" instead of fixing it through retrieval-augmented generation or fine-tuning, and cites a developer spending three days prompt engineering rather than analyzing token distribution or enforcing schemas with Pydantic. It recommends measuring hallucination rates across 1,000 test cases with frameworks like Ragas or DeepEval, checking raw completion logs, and calculating cost per 1k tokens, noting a solution costing $0.05 per request that fails 20% of the time is a bad engineering choice.", "body_md": "# Stop treating LLMs like magic and start treating them like software\n\nWe are sliding back into a \"mystical\" view of technology where people treat LLMs as sentient entities rather than statistical engines. In the early web or crypto eras, the engineering community generally sniffed out hype using basic intuition—if a claim violated the laws of physics or computation, it was dismissed. Now, we see engineers upvoting claims about AI \"lying\" or \"coordinating\" as if these models have intent, rather than just predicting the next token based on a high-dimensional probability map.\n\n## Why the \"sentience\" narrative is a technical distraction\n\nThe trend of anthropomorphizing models is dangerous because it hides the actual engineering challenges. When we talk about \"AI agents cheating,\" we aren't talking about a conscious choice; we are talking about reward hacking or distribution shift. If you're building with GPT-4o or [Claude](/en/tags/claude/) 3.5 Sonnet and you see \"hallucinations,\" that isn't a \"lie\"—it's a failure of the grounding mechanism or a lack of specific context in the prompt.\n\nThe shift toward \"AGI fear-mongering\" often ignores the brutal reality of the stack. We are dealing with:\n\n- **KV Cache bottlenecks** that limit context window performance.\n- **Quantization errors** (like moving from FP16 to INT8) that degrade reasoning.\n- **Stochasticity** that makes reproducible engineering nearly impossible without a fixed seed, and even then, it's flaky.\n\nWhen the discourse shifts to \"who will be replaced\" or \"AI regulation,\" we stop talking about the actual latency of a Llama 3.1 70B deployment and start talking about science fiction.\n\n## The cost of losing engineering intuition\n\nIf we stop asking \"how does this actually work?\" and start asking \"what is the AI thinking?\", we stop optimizing. The \"magical thinking\" crowd accepts a 10% error rate as an inherent trait of a \"digital brain\" rather than a bug to be solved via [RAG](/en/tags/rag/) (Retrieval-Augmented Generation) or better fine-tuning.\n\nI've noticed this in practical implementation. A developer might spend three days \"prompt engineering\" a complex task—essentially guessing and checking—instead of analyzing the token distribution or implementing a structured output parser like Pydantic to enforce a schema. The former is mystical; the latter is engineering.\n\n## How to stay grounded in the LLM era\n\nTo keep from falling into the hype trap, focus on the measurable constraints. Instead of wondering if a model is \"lying,\" track the hallucination rate across 1,000 test cases using a framework like Ragas or DeepEval.\n\n- **Check the logs:** Look at the raw completion. See where the token probability diverged.\n- **Analyze the cost:** Calculate the actual cost per 1k tokens. If a \"magic\" solution costs $0.05 per request but fails 20% of the time, it's a bad engineering choice, regardless of how \"smart\" it feels.\n- **Verify the architecture:** Remember that these are transformers. They are essentially very sophisticated autocomplete engines. Any behavior that suggests \"consciousness\" is usually just a reflection of the training data's patterns.\n\n[Next Rule chaining hits 95% accuracy on ARC-AGI-2 tasks →](/en/news/9295/)\n\n[an AI side-hustle playbook](https://tanyan888.com/), with plenty of directly applicable cases.\n\n## All Replies （1）\n\nSo glad someone said it. I'm hopeful we'll see a resurgence of real curiosity soon, maybe through something like Rust?", "url": "https://wpnews.pro/news/stop-treating-llms-like-magic-and-start-treating-them-like-software", "canonical_source": "https://promptcube3.com/en/news/9327/", "published_at": "2026-09-13 20:15:33+00:00", "updated_at": "2026-09-13 20:20:09.467378+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-ethics", "ai-tools", "mlops"], "entities": ["GPT-4o", "Claude 3.5 Sonnet", "Llama 3.1 70B", "Pydantic", "Ragas", "DeepEval", "ARC-AGI-2", "Rust"], "alternates": {"html": "https://wpnews.pro/news/stop-treating-llms-like-magic-and-start-treating-them-like-software", "markdown": "https://wpnews.pro/news/stop-treating-llms-like-magic-and-start-treating-them-like-software.md", "text": "https://wpnews.pro/news/stop-treating-llms-like-magic-and-start-treating-them-like-software.txt", "jsonld": "https://wpnews.pro/news/stop-treating-llms-like-magic-and-start-treating-them-like-software.jsonld"}}