Disclosure: I built Proceda, the SOP-execution harness used here, and ran these experiments.
I wanted to see how much work an agent could hand to a decision model when the answer is narrower than general-purpose text generation. I tested Qwen 3.8 27B with hosted Jev on 360 SOP-Bench cases across referral abuse detection, email intent, and patient intake.
I tried two insertion points:
With both changes enabled:
There is a tradeoff worth separating out: completion checking alone saved 23.8% of LLM calls but finished 356/360 tasks correctly, versus 357/360 for the baseline. The combined result above is from this particular 360-case subset; it does not establish a general accuracy improvement. Including the final structured-plan preparation cost also reduces the estimated cost saving to 24.6%.
The useful architectural pattern is to supply evidence, define allowed choices, validate the result, and keep an LLM fallback. The implementation and experiment reports are public:
Which decisions in your agents have you successfully offloaded, and how do you validate confidence thresholds before allowing them to bypass the main LLM?