Adapted from three posts first published on the Novaeon Studio LinkedIn page (9, 14 and 28 October 2026).
Want an agent that makes better decisions? Don't fine-tune your big model. Train a small one next to it.
Unsloth published a strong example. Daniel Han and team turned Qwen3.5 0.8B into a decision model: a small scoring head on top of the LLM, LoRA rank 64, one epoch. Average accuracy across three decision benchmarks rose from 20.7% to 74.3%, on 4 GB of VRAM. The model writes no text. It scores the options you give it and returns a choice with a probability.
That is the pattern we favour for agent decisions like which tool, which model, escalate or not. Three reasons to keep the decider small and separate:
How to get training data: let the big model act as teacher and label examples, then correct the labels with real outcomes. The bottleneck is data, not GPUs. So we log every live decision.
Should one big model decide everything, including whether it is the right model for the job? We measured that on our own agent stack. The answer was no.
Every incoming task needs three quick decisions: who handles it (answer directly, hand off, or split into parallel jobs), which model does the work (our local model or a stronger cloud model), and whether it needs step-by-step reasoning.
We built a benchmark of 60 realistic requests, each with a fixed reference label, and gave the same rubric to two judges: a small 9B decision model that returns calibrated probabilities, and our main local LLM, a 35B mixture-of-experts model.
Alone, the main model picked the right handler 92% of the time. But of the 20 tasks that needed the stronger model, it kept 9 for itself: debugging, a security review, an email sent on someone's behalf. A model is not a neutral judge of its own limits.
The small model made different mistakes. Their under-escalations overlapped on exactly one task.
So we combined them: a task stays local only if both judges say so. Any doubt goes up. Result on the 60 tasks: handler 92% (55/60, unchanged), model tier 94% (49/52, media tasks excluded), reasoning on/off 97% (58/60). One under-escalation left, and it is that shared one.
Two more findings:
In live use, a routing decision takes a median of about 1.5 seconds, fully on local hardware. The expensive model only gets the work that needs it.
Can a 4-billion-parameter model decide which release gates an agent change needs? We wanted to know, so we built it in the open.
Every change to an agent is a release: a prompt edit, a new tool, a model swap, a wider permission, a removed human approval step. Each one needs a risk class and the right gates: regression tests, security review, business approval, a rollback plan, a staged rollout.
So we published two things on Hugging Face:
On the held-out test split of 341 cases:
Better. Not good enough to approve anything on its own.
What it cannot do, and the dataset card says so:
So the adapter is a decision aid. Anything rated high or critical goes to a human.
Why open? Because "trust our classifier" is not an argument. Data, rubric, filters and scores are public, so anyone can test it against their own change records: huggingface.co/NovaeonStudio.
How we run local models, evaluation sets and LoRA fine-tuning for clients who keep data in-house: Private AI Lab at Novaeon Studio. Maik Buchenau, co-founder of Novaeon Studio