# Why are we using a sledgehammer to crack a nut?

> Source: <https://promptcube3.com/en/threads/2438/>
> Published: 2026-07-23 17:01:37+00:00

# Why are we using a sledgehammer to crack a nut?

I recently audited my email triage agent. I was running a 7B parameter model (Qwen 2.5) via Ollama just to decide if an email was a "newsletter" or "work." It was absurd. I had a GPU humming 24/7 and agonizingly slow inference just to identify a sender domain. I replaced that bloated mess with a tiny Go classifier that's only 2.4 MB.

Here is the routing logic in Go:

If you're building an AI workflow, stop treating the LLM as the default. Use a practical tutorial approach: map your data, identify the obvious patterns, and use the smallest possible tool for the job. Most of the time, a basic classifier from a decade ago beats a modern LLM on efficiency and speed.

The strategy is simple: Put the LLM last.

## The Triage Hierarchy

Instead of a direct API call, I implemented a three-layer filter. If layer one can solve it, we stop. If not, we move to layer two. The LLM is the absolute last resort for the "uncertain tail."

**Layer 1: Deterministic Rules.** If the sender is

, it's a newsletter. No brain cells required.[[email protected]](/cdn-cgi/l/email-protection)**Layer 2: Small ML Model.** Using TF-IDF and logistic regression on the CPU. Sub-millisecond latency.**Layer 3: The LLM.** Only triggered if the small model's confidence score is below a specific threshold.

Here is the routing logic in Go:

```
func (c *Classifier) decide(m Message) Decision {
	// 1. Deterministic rules: obvious senders, decided at the door.
	if d := preClassify(m); d != nil {
 return *d
	}
	// 2. Small ML model: sub-millisecond, on CPU.
	pred := c.ml.Predict(m)
	if pred.Confidence >= c.threshold {
 return decisionFrom(pred)
	}
	// 3. LLM last: only the uncertain tail reaches the cloud.
	return c.classifyWithLLM(m)
}
```

## Performance Breakdown

Stop blindly chasing "state-of-the-art" and look at the actual cost-to-value ratio:

**Resource Usage:** Swapped a power-hungry GPU for a few megabytes of RAM.**Latency:** Moved from seconds of waiting for an LLM response to sub-millisecond CPU execution.**Reliability:** A hard-coded rule is 100% predictable; a 7B model is a probabilistic guess.

If you're building an AI workflow, stop treating the LLM as the default. Use a practical tutorial approach: map your data, identify the obvious patterns, and use the smallest possible tool for the job. Most of the time, a basic classifier from a decade ago beats a modern LLM on efficiency and speed.

[Next Building a Deep Learning Framework from Scratch →](/en/threads/2426/)

## All Replies （3）

A

Did this with a small regex script for my filters and barely noticed a difference.

0

M

Try a simple keyword filter first, usually handles 90% of the noise anyway.

0

J

Switched to a tiny BERT model for similar tagging and it's basically instant now.

0