wmo serve
tool hits exactly that pain point by routing tasks to smaller, distilled models.The core logic is simple: it uses your existing agent traces to identify patterns. Instead of hitting a frontier model every time, it distills that knowledge into a smaller model and routes the repetitive stuff there.
Here is how the workflow actually functions:
-
Trace Capture: It takes the logs/traces your agents are already generating.
-
Distillation: It uses those traces to refine a specialized smaller model.
-
Routing: A router decides in real-time if a request needs the "big" model or if the custom distilled model can handle it.
-
Compaction: It strips out noise to reduce token usage.
For a local setup, you just provide the traces and an OpenRouter key. It spins up an OpenAI-compatible endpoint locally. As more traces come in, the system keeps training, meaning the smaller model gets more accurate over time.
From a deployment perspective, this is a practical way to build a production AI workflow where you aren't paying "frontier prices" for basic tasks. The claim is a 40%+ reduction in costs while maintaining equivalent quality.
If you want to check out the demo, the link is here:
https://www.youtube.com/watch?v=2_m4Ze6mdko
For those who don't want to manage the infrastructure, there is a hosted version available via their waitlist:
https://experientiallabs.ai
Next Saying "no" to a feature or a deploy is actually a high-value →