Why we keep using LLMs for O(1) problems

Developers are increasingly using large language models for simple, deterministic tasks like email validation or password strength checks, incurring unnecessary latency and cost, according to a technical analysis. The post argues that LLMs should be reserved for tasks requiring nuance or judgment, while binary rules and lookups should remain in standard code to avoid what it calls the 'shiny object tax.'

Why we keep using LLMs for O 1 problems The "Shiny Object" Tax We've all seen it: a dev needs to validate an email address, and instead of a regex, they implement a model call. Suddenly, you've got API latency, loading spinners, and a monthly bill for something that should take a microsecond. You've essentially traded a local CPU cycle for a round-trip to a data center. Determinism vs. Probabilism The core difference is simple: an if statement is deterministic. The same input always yields the same output. An LLM is probabilistic—it can give you a different answer based on the temperature setting or a model update. If-statement: Zero cost, millisecond execution, 100% predictable. LLM call: Per-token cost, seconds of latency, variable output. When to actually use each If you're building an AI workflow, you need to be ruthless about where the model actually adds value. Stick to standard code if/else, regex, lookups when: - The rule is binary. e.g., "Is the cart total over $50?" - You need absolute consistency. e.g., Billing logic, permission checks . - Performance is critical and there's no ambiguity. Deploy an LLM agent when: - The task requires nuance or judgment. e.g., Summarizing a transcript . - The input is unpredictable free text. e.g., "What is this customer actually upset about?" . - Variability is a feature, not a bug. e.g., Creative copywriting . Real-world examples Password strength: "8 characters and a symbol" is a rule. Don't use AI for this. Urgency detection: Checking if a ticket contains the word "urgent" is a keyword search. Understanding that a customer is subtly furious based on their tone is a language problem. Use AI here. Discount codes: Checking if a code is expired is a database lookup. Stop asking models to check your DB. Meeting notes: Condensing 200 pages of rambling into three bullet points? That's exactly what LLMs are for. Choosing the wrong tool doesn't just look like over-engineering; it kills your UX with unnecessary latency and burns your budget on tasks that a junior dev could solve with three lines of JavaScript. Next Vibe Coding: Why AI-Assisted Projects Fail → /en/threads/2240/