How to Run OUI-1 with vLLM for Generative UI

Thesys published OUI-1, a 26B-parameter diffusion model with 4B active parameters finetuned via LoRA from Google's DiffusionGemma 26B-A4B-it, which generates UI screens in openui-lang and scores 71.7% on the Generative UI Benchmark versus 13.0% for its base model. vLLM 0.24 added block-diffusion serving support, and running OUI-1 with FP8 quantization cuts the weight footprint to about 25.8 GiB, rendering a light screen in roughly one second and dense screens in three to six seconds on an A100 80GB using 48 denoising steps. Tool calling works through Gemma 4's native format with --tool-call-parser gemma4, though temperature and seed are ignored by the sampler and tool_choice="required" is silently ignored because vLLM has no structured outputs for diffusion models yet.

How to Run OUI-1 with vLLM for Generative UI A practical guide to serving OUI-1 with vLLM: FP8 quantization, GPU memory needs, tool-calling setup, and OpenAI-compatible API calls. What is OUI-1 and why does it need special serving setup? OUI-1 is a diffusion model built specifically to generate user interface screens, published by Thesys as a finetune of Google’s DiffusionGemma 26B-A4B-it. Instead of writing tokens one at a time like a standard chat model, it writes UI code in blocks: 256 tokens at once, starting from noise and resolving each token as the model becomes confident in it. That block-diffusion approach is what makes it fast about a second per screen on one GPU but it also means the serving stack needs to understand diffusion-specific settings like canvas length and denoising steps, not just the usual sampling parameters. vLLM 0.24 added support for this, which is why it’s the recommended way to run OUI-1 in production. TL;DR - OUI-1 is a 26B-parameter model with only 4B active parameters, finetuned via LoRA from Google’s DiffusionGemma and merged back into bf16 weights. - It writes UI screens in openui-lang , a declarative format where each line is a component wired into a root, meant to be rendered by OpenUI’s React, Vue, or Svelte renderers. - Serving it with vLLM 0.24 or newer and FP8 quantization brings the weight footprint down to about 25.8 GiB , small enough to share a GPU if you set --gpu-memory-utilization . - On an A100 80GB , a light screen renders in about a second and dense screens take three to six seconds, using 48 denoising steps as specified in the checkpoint’s own generation config. - The model scores 71.7% on the Generative UI Benchmark, a 5.5x jump over its 13.0% base model score, using a shared system prompt and validator across all tested models. - Tool calling works out of the box through Gemma 4’s native format: pass --tool-call-parser gemma4 and the model returns standard OpenAI tool calls , then writes the screen from the tool result on the next turn. - Two important quirks: temperature and seed are ignored by the sampler, and tool choice="required" is silently ignored too, since vLLM has no structured outputs for diffusion models yet. One coffee. One working app. You bring the idea. Remy manages the project. How do you install and start the vLLM server? The setup is a single pip install followed by a single vllm serve command. You need vLLM 0.24 or newer, since that’s the version where block-diffusion serving support landed. pip install "vllm =0.24" vllm serve thesysdev/OUI-1 --trust-remote-code --max-model-len 16384 --quantization fp8 \ --served-model-name OUI-1 --max-num-seqs 4 \ --enable-auto-tool-choice --tool-call-parser gemma4 A few things worth noting about these flags. --max-model-len 16384 matches the context length OUI-1 is served at. --quantization fp8 is what gets the weight footprint down to roughly 25.8 GiB. --tool-call-parser gemma4 is required if you want structured tool calls back, since OUI-1 inherited Gemma 4’s native tool-call format rather than using a generic parser. The sampler settings, meanwhile, are not flags you need to pass at all. The 256-token canvas size comes from config.json , and the entropy-bound sampler entropy bound 0.1 plus the 48 denoising steps come from generation config.json . Everything documented about OUI-1’s speed and benchmark score was measured with exactly this command line and no extra tuning. How much GPU memory does OUI-1 actually need? At FP8 quantization, the weights take about 25.8 GiB. vLLM will then fill the remaining GPU memory with KV cache by default, which is fine if you have the card to yourself but a problem if you’re sharing it with other workloads. In that case, pass --gpu-memory-utilization explicitly to cap how much vLLM tries to claim. One hardware caveat matters here: A100s don’t have native FP8 support. vLLM runs FP8 weight-only quantization through Marlin on Ampere cards, which shrinks memory use for real but doesn’t give you the compute speedup that native FP8 hardware like H100s would provide. So the memory savings are legitimate on an A100, but don’t expect FP8-level throughput gains from the arithmetic side. If you’d rather run in bf16 through Transformers instead of vLLM, budget for about 52 GiB of GPU memory, which fits on one A100 80GB or H100. How do you call OUI-1 once it’s running? OUI-1 speaks the OpenAI chat completions format, so any existing OpenAI client library works against it once vLLM is serving it locally. python from openai import OpenAI client = OpenAI base url="http://localhost:8000/v1", api key="local" system = open "system-prompt.txt" .read brief = """Status page for the platform team. Build a single screen for this. It must show: 1. current uptime percentage for the API this month 2. a short note on the most recent incident and when it was resolved Cover every numbered item.""" r = client.chat.completions.create model="OUI-1", messages= {"role": "system", "content": system}, {"role": "user", "content": brief} , max tokens=4096, stream=False, print r.choices 0 .message.content The system prompt is where the component library lives. OUI-1 needs to know what components exist and their signatures before it can write valid openui-lang for them. You can generate that system prompt from your own component library with npx @openuidev/cli generate <library.ts --out system-prompt.txt , or use the reference prompt shipped in the generative-ui-bench repo to try the model without building your own library first. Remy doesn't build the plumbing. It inherits it. Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something. Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want. A couple of behavioral quirks to plan around: temperature and seed are accepted but ignored, since the diffusion sampler runs its own fixed schedule and no per-request seed gets threaded through. That means two identical requests can come back with differently worded screens, even though the layout logic should be consistent. Also, max tokens=4096 covers every screen size tested in the benchmark, though the benchmark itself runs at 8192 for headroom. Streaming works too, with one difference from autoregressive models: because output arrives one 256-token canvas at a time, a screen that fits in a single canvas arrives as one chunk rather than a token-by-token trickle. How does tool calling work with OUI-1? OUI-1 keeps Gemma 4’s native tool-call format, so with --tool-call-parser gemma4 set on the server, it returns standard OpenAI-style tool calls objects for any tools you pass in the request. The pattern is a two-turn exchange: the model calls the tool on the first turn, then writes the actual screen from the tool’s returned values on the next turn. Getting this to work reliably takes one addition to your system prompt: an explicit instruction telling the model to call the tool before answering, rather than inventing plausible-looking numbers. Something like: “When the user asks about the weather or a stock price you MUST call the matching tool first and output nothing else in that turn; write the screen from the returned values on the next turn.” With that instruction in place, testing against weather and stock price tools showed the model calling the correct tool with correct arguments on 9 out of 9 relevant asks, and correctly not calling any tool on 3 unrelated asks, using the returned numbers verbatim in the resulting screen. One limitation worth flagging: leave tool choice at its default "auto" . Passing "required" or naming a specific function is accepted by the API but silently ignored, because vLLM doesn’t yet support structured outputs for diffusion models. In that case the model just generates an ordinary screen, but the response will report finish reason: "tool calls" even though tool calls comes back empty. Don’t branch your application logic on finish reason alone; check whether tool calls actually has content. Is OUI-1 worth deploying over a general-purpose chat model? That depends on what you’re building. OUI-1 is explicitly not a general chat model, it’s narrow by design: generating UI screens for OpenUI-based applications where latency is a hard constraint and self-hosting is preferred over an API call to a much larger model. If your use case is exactly that, generating layouts on the fly in response to user requests, its speed advantage roughly a second for light screens, three to six seconds for dense ones on an A100 and its benchmark performance 71.7% versus 13.0% for the un-finetuned base model make a real case for it. If you need a model that can also hold a conversation, answer general questions, or do anything outside UI generation, OUI-1 isn’t that model, and you’d want it running alongside something else rather than in place of it. Frequently Asked Questions What GPU do I need to run OUI-1? At FP8 quantization through vLLM, the weights take about 25.8 GiB, which fits comfortably on an A100 80GB with room for KV cache. Running in bf16 through Transformers instead needs about 52 GiB, still within reach of a single A100 80GB or H100. Does OUI-1 support streaming responses? Remy is new. The platform isn't. Remy is the latest expression of years of platform work. Not a hastily wrapped LLM. Yes, but output arrives in 256-token canvas chunks rather than token by token, since it’s a block-diffusion model. A screen that fits within one canvas will arrive as a single chunk instead of a gradual stream. Why are temperature and seed ignored? The model uses the checkpoint’s own entropy-bound sampler and fixed 48-step denoising schedule rather than standard autoregressive sampling. No per-request seed is plumbed through, so identical requests can produce differently worded though structurally similar screens. Can I use function calling with OUI-1? Yes. Set --tool-call-parser gemma4 when starting the vLLM server, add an instruction to your system prompt telling the model to prefer tool calls over invented data, and it returns standard OpenAI-format tool calls . Just leave tool choice at "auto" , since "required" and named functions are accepted but ignored. How is OUI-1’s output rendered into an actual UI? It outputs openui-lang, a declarative format with one component per line wired into a root. You render that output using @openuidev/react-lang or the Vue and Svelte equivalents, and validate it with @openuidev/lang-core before displaying it.