Are Local LLMs Actually Worth It?

Hey everyone, I'm Wolf, and I survived another week!

This is update #2 in my journey building Wolf.IA.

Last week, I talked about finally hitting an MVP and having to face the hard part: marketing and distribution. Specifically for Wolf.Engine, a platform designed for building multimodal AI agents.

(Hey, feel free to give it a spin, too! Pretty please? wolf.ia.br/engine)

The feedback I received last week pointed out that my second project (focused on the Brazilian accounting framework) would actually be much easier to market and sell right now. And you guys are totally right. The Engine is still my pride and joy, but I need a cash cow to keep the operation alive.

So right now, I am:

During development, I decided to benchmark a few local models via Ollama + OpenCode. The whole point of building Wolf.Park right now is to slash my API costs, so if local models can shoulder any of the workload, that's a huge win.

The catch: My GPU is an RTX 4060 Ti with only 8GB of VRAM. That's tight for heavier models.

Here were the response times for generating a simple Python script to read and update text files:

┌────────────────┬───────────────┐
│     MODEL      │ Response Time │
├────────────────┼───────────────┤
│ gemma4:12b     │ 12m 29s       │
│ gemma4:26b     │ 8m 48s        │
│ gemma4:e4b     │ 1m 37s        │
│ gemma4:e2b     │ 31s  (!!)     │
│ ornith-1.5:9b  │ 11m 41s       │
│ qwen3.8:27b    │ 25m 40s       │
└────────────────┴───────────────┘

(I also tested qwen3.6 and phi4, but both were disqualified: the former hung for over 30 minutes without outputting anything, and the latter lacked support for tool usage in my setup).

All the tested models managed to produce files and functional code, but the quality varied:

gemma4:e4b and gemma4:e2b included unnecessary (though harmless) imports;qwen3.8:27b relied on lazy design patterns, dumping values into global variables instead of passing arguments between functions;ornith-1.5:9b used overly generic exception handling. However, one thing surprised me: ornith-1.5:9b was the only model that, after writing the script, actually tested its own code. It was also the only one that took the time to write a detailed explanation of how the script works.

Based on this small (and admittedly limited) experiment, I drafted this multi-agent workflow:

[User Request]
       │
       ▼
1. LOCAL GENERATION
   ├── Agent 0: gemma4:e4b ──> Generates base code
   ├── Agent 1: gemma4:e4b ──> Independent reviewer (harsh critic persona)
   ├── Agent 2: gemma4:e2b ──> Runs local linter & style review
   └── Agent 3: gemma4:e2b ──> Generates pytest unit tests
       │
       ▼
2. CLOUD REVIEW & ESCALATION
   └── DeepSeek v4 Pro ──> Reviews code + generated tests
       │
       ├───> [Approved] ──> Sent to human review.
       │
       └───> [Rejected: Low Difficulty]
       │         │
       │         └──> Gemma e4b/e2b attempt fixes
       │
       └───> [Rejected: Medium Difficulty or Low with Persistent Error]
       │         │
       │         └──> Ornith-1.5:9b attempts fix
       │
       └───> [Rejected: High Difficulty or Medium with Persistent Error]
       │         │
       │         └──> DeepSeek v4 Pro attempts fix
       │
       └───> [Architectural Impasse / 3rd Round Failure / High with Persistent Error]
       │         │
       │         └──> Gemini 3.8 Flash attempts fix
       │
       └───> [Unresolved Impasse / 4th Round Failure / Critical Escalation]
                 │
                 └──> GPT 6 Astra attempts fix ──> Sent to human review.

Will this actually work in practice? I don't know yet. Wolf.Park is being built to make orchestrating this kind of pipeline seamless, but for now... I'll be testing this scheme manually throughout the week.

Drop a comment below if you have any suggestions for local models I should benchmark and potentially plug into this pipeline!

Cheers!