# Designing Generative Vision UI

> Source: <https://roboflow.com/blog/designing-generative-vision-ui>
> Published: 2026-11-20 20:30:41+00:00

**Key takeaway:** Roboflow's agent assembles vision apps from prebuilt components for boxes, masks, keypoints, counts over time, and filterable event galleries, with color, type, and spacing exposed as design tokens so branding and light or dark mode are app settings rather than redesigns.

For most of the history of software, someone else decided what the user's screen looked like. A team studied a few thousand people, designed the interface that would work for most of them, and shipped it. If someone’s job was a little different from the average, they learned how to work around the gaps.

Now a user can describe the software they want in a sentence and watch a working version appear. At Roboflow, we're building for the world where anyone can describe an app that uses computer vision and have it built and hosted.

This has fundamentally shifted our design work. Instead of designing the end screens to be as flexible as possible, we’re designing a system and tool that can make those screens come to life.

Dynamic UIs are New

Design has always started with a guess about the user. We interview people, look for patterns, and commit to one layout, because building a second one costs months. Good design teams guess well. But they can still leave out the person whose need shows up once in ten thousand accounts.

When an agent can build an interface from a description, the guess moves. We no longer have to predict the exact screen. The person who needs it says what they want to see, looks at the result, and asks for changes until it fits. The interface gets made by the person doing the job, in about the time it takes to explain it.

Components that Build with Vision

Half of this already exists. With Roboflow Agent, users describe what a camera should look for, and the Agent helps pick the models, builds the Workflow, and runs it. We've talked to people with no background in computer vision or programming who built their first functional Workflow this way. The next thing a user wants is a screen to monitor their models and see workflows actually running.

Writing a web page is close to a solved problem for an agent. Plenty of tools will produce a dashboard in a minute. Vision applications are harder, because a lot of functionality behind the screen has to exist first. The app needs a model that recognizes what the user wants to see, a Workflow to turn detections into insights, and a place to run.

Roboflow already holds these pieces for users. Now, we’re building a tool to natively create the interface right where the model, data, and project lives. A few principles follow from it:

An app talks to a user’s models, Workflows, devices, and data.

Hosting is automatic.

An app stays private to a user’s workspace until they decide to share it.

An app starts from standard, tested design components to ensure the first version reads well and has a stable jumping off point.

That last principle hides a lot of design work. Vision output is awkward material for an interface. It can be an image with shapes drawn over it, a score between zero and one, a count that changes many times a second. A general-purpose component kit has a table and a bar chart, and nothing that knows how to draw a detection.

So we built components and primitives that do. Displaying boxes, masks, and keypoints over the image, handling counts over time, displaying a gallery of past events users can filter, showing the alerts triggered from incoming detections. The Agent assembles apps from what we’ve built instead of writing new code each time, which is why two apps made by different people may still look related.

Even so, users can still customize each app. Color, type, and spacing live in the design system as tokens, so the user’s logo, brand color, and light or dark mode are settings on the app. Nobody has to redesign anything to make it their own.

Users can ask for whatever they want, and the app is connected to something real from its first minute.

Designing the Defaults

When a person describes an app, they don't specify type sizes or how a chart behaves. Those decisions come from the components and defaults we give the Agent. If we get the defaults right, someone's first app reads clearly and cleanly, and they never have to know why. If we get something wrong, every generated app inherits the mistake.

The same is true of everything a person forgets to ask for. Nobody describes the empty state, the loading state, or what the screen should say if the camera feed drops. Each component has to arrive with those already designed, because the Agent will place it in situations we never drew.

Interaction design changes too. We think about what happens between the request and the result. The preview should stay live with the conversation, so users see each change land. We want each change to be a version they can return to, so trying an idea costs nothing.

Vision adds more complexity, because there is so much variety to what it can do and so much people want to do with the output. The same detection can feed a count, trigger an alert, go to someone for review, or go back into a dataset to improve the model. Someone who watches a live camera needs a different screen from someone who checks last week's trends, yet both are working from the same output. Our components and defaults have to hold up across all of those uses.

This is why leaning into this work is challenging for us, but so rewarding for the user. Users get the kind of complex solution that we would build for them, made for their own tasks and goals. They do not need to know the specific how to route data or what components to use. Everything is built through language, something that is novel, empowering, and a huge shift for the field. We’re shifting with it to make sure that at the end of it, the user gets exactly what they need.

Tell Us What You'd Make

If there's something you've wanted a camera to tell you and never had a screen for, we’d love to hear about it. Those are the apps we want to make easy. You can start today by describing what you want a camera to see to Roboflow Agent, or set up a meeting with us to chat about your needs and requests.
