Amigo walks around the desktop, follows the cursor with his eyes, and says “No more caramba.” Move the cursor back and forth over him and he closes his eyes with a little heart above his head.
He's one of five pets in the AgenticOS desktop app. You can also pick a ghost, a small terminal on legs, a sprout, or a round creature with an antenna. They share the same movements, but each has its own appearance and lines.
AgenticOS is an open-source project I'm building at Vstorm for creating and using AI agents. The pet is a small extra in its Tauri desktop app. This article is about how that extra works: drawing the characters, deciding what they do, and keeping them on the desktop beside the main window.
Amigo has his own window, so he can stay visible when the main app is minimised.
The artwork starts as a small grid. Each character in the grid represents a colour, and a dot represents a transparent pixel. The body, eyes, feet and other details are placed on top of one another to produce a frame.
That means we can change the expression without drawing the whole character again. A blink uses closed eyes in place of open ones. A wave moves the arm. A hop lifts the parts together and then brings them back down.
Each pet has its own body and coordinates for those parts. The ghost has no feet, while Amigo has a sombrero and moustache. The animation code works with those differences, using the parts that each character provides.
The definitions live in pet-sprites.js. In the same file, a waving animation is described by these two positions:
wave: () => [{ arm: "up" }, { arm: "down" }],
That's an excerpt from the animation table, rather than a standalone program. The renderer combines each position with the selected pet's artwork. Other entries describe blinking, walking, sleeping and hopping in the same way.
For these simple characters, that gives us a shared set of movements to work on. A new pet needs artwork and part positions, then it can use the existing animations. A character with a very different shape or movement would need more work. Sprite sheets would still be a reasonable choice for more detailed, individually drawn animation.
Drawing the frames is separate from choosing what happens next. That second part lives in pet-engine.js.
When left alone, the pet spends most of its time standing still. It occasionally looks around, walks a little, or dozes. At night, it dozes instead of choosing another stroll. When it reaches the edge of the usable screen area, it turns back.
Those decisions are functions with explicit inputs. The choice of the next behaviour takes a random-number function and the hour. Walking takes the current position, direction, elapsed time and screen boundaries. None of those calculations needs to open a desktop window.
That makes the rules easier to check independently. You can supply a particular hour or position and inspect the result without waiting for a pet to happen to do the right thing on screen. The repository includes tests alongside the pet code for these behaviours.
The eyes follow the cursor, with a small area around the centre where they keep looking straight ahead. Otherwise, small pointer movements near the middle would keep changing their direction.
Stroking needs a different rule. A cursor passing over the character once shouldn't trigger the same reaction as moving back and forth over it. The engine tracks changes in direction within a short time window. Enough changes trigger the closed eyes and heart animation.
Clicking makes the pet wave and say something. Double-clicking brings the AgenticOS window forward. Hovering reveals a “+ New chat” button, and the right-click menu lets you change the character or hide it.
These interactions give the pet a few useful links back to the app without requiring it to move constantly. It also respects the operating system's reduced-motion setting by standing still while remaining draggable.
The pet lives in a transparent, always-on-top Tauri window. The main console is a separate window, so minimising or closing that window doesn't remove the pet while the app is still running.
The app remembers the selected character, whether it's visible, and where it was left. On the next launch, you don't need to pick the same pet and drag it back into place.
The console itself loads the web app from the configured AgenticOS server. You still need that server running. Installing the desktop shell doesn't install the platform or make it available offline.
One current limitation is that the pet doesn't show agent activity. A sleeping Amigo doesn't mean your agent has stopped, and a wave doesn't mean a task has finished. Connecting those events to the pet would require an explicit connection between the web app and the desktop shell. For now, its reactions come from the interactions described here.
If you'd like to explore the implementation, start with these three files in desktop/ui:
pet-sprites.js holds the artwork, part positions and animation frames.pet-engine.js handles behaviour, movement and cursor interactions.pet-lines.js contains what each character says.
The desktop documentation covers the Tauri setup and commands for running it against your own AgenticOS installation.
Changing a line in pet-lines.js is a small place to start. Changing the artwork gives you something more visible to play with. You can explore either without needing to understand the agent builder or the rest of the platform first.
Kacper Włodarczyk, building AgenticOS at Vstorm.