Living House: Adaptive UI and docs using in-browser inference

Developer theletterf released Living House, an experimental fake home-automation app whose interface and in-product documentation are rewritten on the fly by a small language model running in the browser via WebLLM. Each UI element carries a fixed ID and a fixed description of its real function, and the model may only rename or re-describe registry entries — it cannot create or delete them — based on signals such as hover time, hesitation and most-clicked items. The experiment tests whether LLM-driven adaptive UIs and docs stay usable, echoing earlier adaptive-menu efforts like Office 2000's and Krzysztof Z. Gajos's 2009 finding that "evaluation results of adaptive GUIs have been mixed.

Living House: Adaptive UI and docs using in-browser inference One of my favorite video games of all time is Control https://www.remedygames.com/games/control . The main location of the game, the brutalist Oldest House, reacts to the player’s actions and rearranges itself dynamically, moving slabs of concrete around like a sentient being. Now, what if user interfaces and docs could react dynamically to the user and seek to adapt to their behavior like Control’s levels? Would it still be usable? Would the documentation drift too much? I explored these questions through an experimental application that uses in-browser LLMs. I’ve been a fan of Control since it came out, but what spurred me to action was WorldTUI https://simedw.com/2026/10/21/world-model-terminal/ , an experiment by Simon Edwardsson that sought to replicate a full CLI experience through a fine-tuned small language model. What I find deeply fascinating about it is the fact that there is no real CLI underneath, just the pretense of one. At the same time, the idea of using a small LLM in the browser was new to me; I immediately started to wonder what other applications a small model could have for documentation and design. Adaptive UIs are not a new thing. The venerable Office 2000 had adaptive menus https://www.itwriting.com/blog/2594-six-abandoned-features-from-the-history-of-microsoft-office.html that hid least used items a feature that was abandoned in later versions . The infamous Clippy also tried to detect user intent and suggest actions. This failed most of the time, partly because the suggestions got in the way more times than they helped. Experiments on adaptive UIs went on, though. For example, SUPPLE++ https://www.eecs.harvard.edu/~kgajos/papers/2007/gajos07automatically.shtml by Krzysztof Z. Gajos proposed a way of generating UIs for impaired users. In 2009, Gajos summarized https://www.eecs.harvard.edu/~kgajos/papers/2009/AIMag09-AUIs.pdf the situation thus: Adaptive graphical user interfaces GUIs have the potential to improve performance and user satisfaction by automatically tailoring the presentation of functionality to each individual user. In practice, however, many challenges exist, and evaluation results of adaptive GUIs have been mixed. Parodies of adaptive UIs had more success: I still remember Desktop Goose https://samperson.itch.io/desktop-goose and User Inyerface https://userinyerface.com/ as grim reminders of the horror of user-unfriendly UIs and the nightmare of apps that try too hard to be smart. More recently, though, experiments like Neural-OS https://neural-os.com/ show potential for GenAI to turn user experience into intelligent UIs, in the sense of anticipating user intent and adapting fully to their needs. Consider workflows affected by network lag, which can be mitigated by predictions done by LLMs. Another example is Generative UIs https://research.google/blog/generative-ui-a-rich-custom-visual-interactive-user-experience-for-any-prompt/ , which are generated on the fly by models. Enter the Living House Living House https://github.com/theletterf/living-house is my experiment in LLM-powered adaptive UIs and docs. At its core, Living House is a fake home automation service that reshapes itself by analyzing user behavior, like how long the user hesitates and hovers over a label or a button, or what are the most clicked items. Each UI element has a fixed ID with a fixed description of what it really does; anything else is ripe for modification by a small LLM that runs in the browser through WebLLM https://webllm.mlc.ai/ . The model edits the UI as well as the in-product docs. It all happens as you go. Every part of the app is an entry in a registry, keyed by a permanent ID. As the app renders from the UI registry, changing an entry is the only way to change the UI. The registry covers most areas and is protected against deletion or creation it’s a fixed contract that only allows renaming and style modifications . This protects the core intent of the app while it opens up the possibility of fully renaming or restyling elements, including the docs, which quote the former labels if a change occurs. The docs in the help look like this: On {{nav overview}}, choose {{sc movie}}. It switches off {{lr ceiling}}, … What does the LLM act upon? Living House collects seven different kinds of signals, looking for hesitation and interactions. | Signal | Logged as | Default threshold | Read as | |---|---|---|---| | Click, switch, slider, stepper | click, set | Every action | Use; repeated use is a habit | | Page change | nav | Every change | Navigation; 3 in a row without acting is searching | | Hovering over a control | hover | 0.3 s | Interest | | Hesitating on a control | hesitate with an ID | 1.5 s without clicking | Confusion: the label is unclear | | Hesitating on a page | hesitate without an ID | 3 s without doing anything | Confusion: what they need isn’t visible | | Selecting text | select | Any selection of 2+ characters | What the person wants | | Pointing at text | read | 0.6 s | What the person wants | A round is one call to the model that turns recent activity into at most two proposed changes. The default model is Qwen2.5 Instruct, 0.5B, which is pretty small, quantized to 4 bits and executed locally on WebGPU through WebLLM https://github.com/mlc-ai/web-llm . Everything is local and not very performant. The model isn’t fine-tuned either, which could perhaps help get better results, but as an experimental platform it works quite well at the various intensity levels, which are defined by how often the LLM runs following a user interaction. The more aggressive the option, the more often the model suggests changes as if it was getting chaotic . Since the model might be wrong when proposing something, every change proposal is checked in the app before it is queued, and checked again just before it is applied: | Change | Applies to | Rejected when | |---|---|---| | text | Any element with text | Empty, same as now, over 24 characters on a control or 120 elsewhere | | style | Any element | Not property: value pairs, invalid per CSS.supports, or loads anything external url , image-set , @import, expression , javascript: | | hint | Controls | Empty, over 48 characters, same as now | | move first | Controls | Already first | | pin | Actions and switches | Already pinned, or 4 already pinned | A change only reaches the UI either because the model suggested it and got validated or because the user asked for it. For example, I asked to rename a menu and the app duly complied. By default, Living House asks for permission before changing anything it can get real fun when you turn that off and let the UI evolve in front of your eyes . A UI element is off limits when the user undid a change to it, when it has reached the intensity cap, or when the exact change was declined before. Off-limits elements are left out of the model’s context and schema. Help or docs usage also feeds adaptation. Looking something up means it was hard to find, so the model sees looked up lr\ blinds "Blinds" in the help and the base prompt asks it to make that element easier to find. This is interesting for us tech writers because it shows that docs usage can inform product usability directly and feed adaptive UI changes not just the other way around . Seeing the documentation evolve together with the UI, in real time, and remember what things used to be called, was one of the most impressive things I took away from this experiment. Change, trust, intelligence, and the role of docs The possibilities of LLM-powered adaptive UIs are so numerous that I struggle to count all the questions in my head. For example, assuming that state is preserved between runs, should the UI tweak itself according to context and user state? If the user wakes up in the morning, could we have docs that lean towards simpler terms, especially if usage patterns seem more lethargic? What about context? If it’s a cold day, should the model in a home control app adapt to present options for heat boosts first? Or is the whole idea of a fluid, organic UI that shapeshifts entirely heretic and alien to usability principles? What unsettles us about the concept of a machine that wears us like a smart glove? If adaptive UIs powered by AI become a thing, protecting the core terminology and setting up semantic guardrails would be almost a moral obligation, so as to avoid unintended drift in meaning with all the risks that that carries with it . Customer support would have to rely on the bits of docs that don’t evolve, the fixed beacons in the text; docs would be split into dynamic and fixed parts, clearly indicated. In-app docs could also be tested in real time for coherence against the core app contract, so as to avoid dangerous mutations. While labels and positions can change, writers must define the identities, purposes, and relationships that remain stable. But that’s also true for good old deterministic systems. What’s different with AI is that changes become hyperlocal, different for every session. I don’t know if I would like a UI to change under my eyes based on usage patterns, but I can definitely see the appeal of a tool that adapts to the user to the point that usage friction is almost entirely eliminated except the one presented by other factors, like performance or the underlying mental model, which is read-only in this experiment . I find the Oldest House from Remedy’s Control compelling because you cannot really trust it. Software that rearranges itself around us will need to earn trust, though. The way to do that, I suppose, will be through the parts that never move or shift meaning. There’s a role for writers to play there. Featured image is The Gothic Arch, plate 14 from Imaginary Prisons 1761 , by Giovanni Battista Piranesi.