{"slug": "running-an-llm-agent-entirely-in-your-browser", "title": "Running an LLM agent entirely in your browser", "summary": "A developer fine-tuned LiquidAI's LFM2.5 (230M and 350M) into a generic front-end agent that runs entirely in the browser, with no server, API key, or cloud costs. The agent calls real tools to browse a catalog, answer grounded questions, and manage a cart, and is trained on interaction patterns rather than domain facts, allowing the same weights to drive different storefronts without retraining.", "body_md": "**TL;DR**: I fine-tuned LiquidAI's LFM2.5 (230M and 350M) into a generic front-end agent that runs *entirely in the browser* - no server, no API key, no cloud costs.\n\nIt doesn't just chat; it calls real tools to browse a catalog, answers grounded questions, and manages a cart.\n\nThe trick: it's trained on interaction *patterns*, not domain facts, so the same weights drive a coffee store, an absurdist emporium, or a corner grocer - with zero retraining.\n\n[Live demo](https://lajosbencz.github.io/frontend-agent/) on Github pages.\n\nMost \"AI assistant\" features are a text box wired to a frontier model in someone's data center.\n\nThat's fine, but it means a network round-trip per turn, a bill per token, and your users' inputs leaving the device.\n\nI wanted the opposite: an agent small enough to ship *with the page*.\n\nLoad it once, run it on the user's own hardware (WebGPU if available, CPU/WASM otherwise via [wllama](https://github.com/ngxson/wllama)), and let it actually *do things* in the UI instead of just describing them.\n\nThe bet was that a small model can't hold a useful amount of world knowledge, but it *can* learn a compact set of behaviors well enough to be useful.\n\nThe model does **not** know what a \"BrewCraft Pico\" is, or that it costs $699.\n\nIt knows how to:\n\nEverything domain-specific is injected **at runtime**.\n\nEach turn, the host app hands the model a compact context: the items currently on screen (with ids and prices), the cart, and any retrieved knowledge.\n\nThe model grounds strictly in that. Swap the store, swap the injected context - the same weights work.\n\nThat's why the demo ships three storefronts on one model.\n\nAnd critically, they were **held out of training entirely**.\n\nIf the model can run a store it never saw, the generalization works.\n\nThree design choices carry most of the weight.\n\n**A frozen tool roster.**\n\nEarly on I tried teaching the model to read *arbitrary* tool schemas - variable tool names and arguments per training example so it wouldn't memorize a fixed set.\n\nFor a 230M model, that was too much to ask; it garbled calls.\n\nSo the roster is now **fixed**: eight tools with stable names (`list_items`\n\n, `get_item`\n\n, `search_knowledge`\n\n, `add_to_cart`\n\n, `remove_from_cart`\n\n, `clear_cart`\n\n, `checkout`\n\n, `navigate`\n\n) that the model learns by name.\n\nA small, memorizable action space.\n\nThe one place variety survives is the *filter set* on `list_items`\n\n, which the model reads from the injected schema.\n\n**RAG as a tool, not a pipeline.**\n\nRetrieval is just `list_items`\n\n(catalog) and `search_knowledge`\n\n(guides, policies).\n\nThe model decides when to call them and grounds its reply in the results.\n\nThe backend is swappable - BM25, vector, hybrid - because only the *result shape* is the contract.\n\nThe demo uses in-browser BM25; nothing leaves the machine.\n\n**Grammar-constrained decoding.**\n\nTool calls are decoded against a GBNF grammar, so every call is syntactically valid and - the important part - every id the model emits is one that actually exists in the injected context.\n\nIt literally cannot hallucinate a product id.\n\nThat single constraint removes a whole class of failures that would otherwise sink a model this small.\n\nThe pipeline is synthetic-data distillation:\n\nOne deliberate choice worth flagging for anyone doing the same: **train on the pattern, not a blocklist.**\n\nSome LLM providers might cache requests; this would duplicate training data, where we expect variety. To avoid this, seed each request appropriately.\n\nThe quality of the teaching model is of course extremely important; I limited to only Apache 2.0 licensed ones, and the best bang for buck I found at the time of writing was [Qwen3 30B through Openrouter.ai](https://openrouter.ai/qwen/qwen3-30b-a3b-instruct-2507).\n\nWell, yes, but actually no.\n\n**What works:**\n\nStructured tool calls are reliable. Add an item by name or by position, ask a price, get a grounded answer, check out - the core loop holds up, including on domains it never trained on.\n\nFor a 150 MB model running on your laptop with no server, that still feels a lot like magic.\n\n**What doesn't (yet):**\n\n230M parameters spread across many domains is *thin*.\n\nDuring training, generalized use-cases compete for the same limited capacity.\n\nMulti-turn fidelity is the weakest spot - over a long conversation it can drift toward the shape it was trained on rather than the specific thing you just said.\n\nBecause of this, the training regime limits to only 2 turns of conversation patterns, so does the JS runtime with a sliding window; the agent never sees more than 2 turns of conversations.\n\nThe 350M variant buys real headroom at ~1.5x the footprint, and you can switch between them in the demo to feel the difference.\n\nI think this is an interesting frontier: instead of \"can a giant model do this\" (obviously), we ask *how small can you go* and still be genuinely useful on-device.\n\nBeyond the cool factor:\n\n*Built on LiquidAI LFM2.5.\nModel weights inherit the LFM Open License v1.0; code and this post are Apache-2.0 / CC-BY*", "url": "https://wpnews.pro/news/running-an-llm-agent-entirely-in-your-browser", "canonical_source": "https://dev.to/lajosbencz/running-an-llm-agent-entirely-in-your-browser-5foe", "published_at": "2026-07-15 20:59:54+00:00", "updated_at": "2026-07-15 21:10:05.237312+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-tools", "developer-tools"], "entities": ["LiquidAI", "LFM2.5", "Qwen3 30B", "Openrouter.ai", "wllama", "Github Pages"], "alternates": {"html": "https://wpnews.pro/news/running-an-llm-agent-entirely-in-your-browser", "markdown": "https://wpnews.pro/news/running-an-llm-agent-entirely-in-your-browser.md", "text": "https://wpnews.pro/news/running-an-llm-agent-entirely-in-your-browser.txt", "jsonld": "https://wpnews.pro/news/running-an-llm-agent-entirely-in-your-browser.jsonld"}}