TL;DR: I fine-tuned LiquidAI's LFM2.5 (230M and 350M) into a generic front-end agent that runs entirely in the browser - no server, no API key, no cloud costs.
It doesn't just chat; it calls real tools to browse a catalog, answers grounded questions, and manages a cart.
The trick: it's trained on interaction patterns, not domain facts, so the same weights drive a coffee store, an absurdist emporium, or a corner grocer - with zero retraining.
Live demo on Github pages. Most "AI assistant" features are a text box wired to a frontier model in someone's data center.
That's fine, but it means a network round-trip per turn, a bill per token, and your users' inputs leaving the device.
I wanted the opposite: an agent small enough to ship with the page.
Load it once, run it on the user's own hardware (WebGPU if available, CPU/WASM otherwise via wllama), and let it actually do things in the UI instead of just describing them.
The bet was that a small model can't hold a useful amount of world knowledge, but it can learn a compact set of behaviors well enough to be useful.
The model does not know what a "BrewCraft Pico" is, or that it costs $699.
It knows how to:
Everything domain-specific is injected at runtime.
Each turn, the host app hands the model a compact context: the items currently on screen (with ids and prices), the cart, and any retrieved knowledge.
The model grounds strictly in that. Swap the store, swap the injected context - the same weights work.
That's why the demo ships three storefronts on one model.
And critically, they were held out of training entirely.
If the model can run a store it never saw, the generalization works. Three design choices carry most of the weight.
A frozen tool roster.
Early on I tried teaching the model to read arbitrary tool schemas - variable tool names and arguments per training example so it wouldn't memorize a fixed set.
For a 230M model, that was too much to ask; it garbled calls.
So the roster is now fixed: eight tools with stable names (list_items
, get_item
, search_knowledge
, add_to_cart
, remove_from_cart
, clear_cart
, checkout
, navigate
) that the model learns by name.
A small, memorizable action space.
The one place variety survives is the filter set on list_items
, which the model reads from the injected schema.
RAG as a tool, not a pipeline.
Retrieval is just list_items
(catalog) and `search_knowledge`
(guides, policies).
The model decides when to call them and grounds its reply in the results.
The backend is swappable - BM25, vector, hybrid - because only the result shape is the contract.
The demo uses in-browser BM25; nothing leaves the machine.
Grammar-constrained decoding.
Tool calls are decoded against a GBNF grammar, so every call is syntactically valid and - the important part - every id the model emits is one that actually exists in the injected context.
It literally cannot hallucinate a product id.
That single constraint removes a whole class of failures that would otherwise sink a model this small.
The pipeline is synthetic-data distillation:
One deliberate choice worth flagging for anyone doing the same: train on the pattern, not a blocklist.
Some LLM providers might cache requests; this would duplicate training data, where we expect variety. To avoid this, seed each request appropriately.
The quality of the teaching model is of course extremely important; I limited to only Apache 2.0 licensed ones, and the best bang for buck I found at the time of writing was Qwen3 30B through Openrouter.ai.
Well, yes, but actually no.
What works:
Structured tool calls are reliable. Add an item by name or by position, ask a price, get a grounded answer, check out - the core loop holds up, including on domains it never trained on.
For a 150 MB model running on your laptop with no server, that still feels a lot like magic.
**What doesn't (yet):**
230M parameters spread across many domains is thin.
During training, generalized use-cases compete for the same limited capacity.
Multi-turn fidelity is the weakest spot - over a long conversation it can drift toward the shape it was trained on rather than the specific thing you just said.
Because of this, the training regime limits to only 2 turns of conversation patterns, so does the JS runtime with a sliding window; the agent never sees more than 2 turns of conversations.
The 350M variant buys real headroom at ~1.5x the footprint, and you can switch between them in the demo to feel the difference.
I think this is an interesting frontier: instead of "can a giant model do this" (obviously), we ask how small can you go and still be genuinely useful on-device.
Beyond the cool factor:
Built on LiquidAI LFM2.5. Model weights inherit the LFM Open License v1.0; code and this post are Apache-2.0 / CC-BY