{"slug": "wisp-a-quiet-desktop-spirit-local-ai-that-lives-on-your-linux-desktop", "title": "Wisp — A Quiet Desktop Spirit: Local AI That Lives on Your Linux Desktop", "summary": "Developer Ghananil Shirpurkar built Wisp, an open-source MIT-licensed Linux desktop companion that runs as a floating ghost overlay and uses local AI inference through Ollama to analyze screen content on demand. The Electron and TypeScript application captures frames via Wayland desktop portals and PipeWire or X11, sends them over an authenticated loopback connection to a worker running a vision-capable model, and uses Temporal for durable workflow orchestration of observations, retries and cooldowns. No cloud AI inference service is required.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine](https://dev.to/challenges/hacktoberfest-week4-2026-10-26)*\n\nMost AI assistants live in a browser tab or a separate application. You open them, explain what you're doing, provide context, and wait for an answer.\n\nI wanted to explore a different idea.\n\n**What if AI didn't need to be opened every time you needed it? What if it simply lived on your desktop?**\n\nMeet **Wisp**, a floating AI desktop companion for Linux that understands your workspace, observes what's happening on your screen, and provides contextual assistance when you need it.\n\nInspired by the theme *Ghost in the Machine*, Wisp is designed to feel less like another application and more like a quiet presence inside your operating system.\n\nWisp appears as a transparent, floating ghost overlay that stays out of your way. You can interact with it, change its appearance, switch between friendly and after-hours modes, and control its visual effects.\n\nBut the interesting part is what happens behind that little ghost.\n\nImagine you're working on a programming assignment, debugging an error, reading documentation, or navigating an unfamiliar application.\n\nInstead of taking screenshots, switching tabs, and explaining your entire situation to an AI assistant, you can:\n\nThe goal is to reduce the friction between encountering a problem and getting useful assistance.\n\nWisp isn't designed to constantly interrupt you or demand your attention. It is designed to be present when needed and invisible when not.\n\nThe central idea is simple: **an AI companion that understands what you're doing without making you explain everything from scratch.**\n\n**Explore the project:** [Wisp on GitHub](https://github.com/GhananilShirpurkar/wisp)\n\nThe repository README includes a screenshot of Wisp's screen-sharing interface, along with setup instructions and architecture documentation.\n\n**GitHub Repository:** [https://github.com/GhananilShirpurkar/wisp](https://github.com/GhananilShirpurkar/wisp)\n\nWisp is released under the MIT License.\n\nThe repository contains the complete application source, architecture documentation, setup guide, tests, and CI workflows.\n\nWisp is built around one important architectural decision: **AI inference happens locally, not through a cloud AI API.**\n\nRather than building a simple chatbot interface around an API call, I built a system that coordinates desktop interaction, screen capture, vision inference, and background workflows.\n\n| Technology | Responsibility | \n|---|---|\n| Electron | Desktop application and floating ghost overlay | \n| Node.js | Application and worker runtime | \n| TypeScript | Core application and workflow logic | \n| Ollama | Local AI inference | \n| Vision-capable model | Screen understanding and visual analysis | \n| Temporal | Durable workflow orchestration | \n| Linux desktop portals and PipeWire | Wayland screen capture | \n| X11 utilities | Display capture and window metadata | \n\nWisp separates four major responsibilities:\n\n**1. Desktop Interface**\n\nThe Electron application manages the floating ghost, user interactions, screen-sharing controls, visual effects, and communication with the background worker.\n\n**2. Screen Capture**\n\nWhen the user explicitly enables screen sharing, Wisp captures screen frames through the appropriate Linux integration.\n\nOn Wayland, it uses the desktop portal and PipeWire. On X11, it captures the primary display.\n\n**3. Local AI Intelligence**\n\nCaptured frames are passed to a local worker through an authenticated loopback connection.\n\nThe worker sends the images to Ollama, where a vision-capable model analyzes the screen and produces structured observations.\n\nNo cloud AI inference service is required.\n\n**4. Workflow Orchestration**\n\nTemporal manages the observation lifecycle and background execution.\n\nIt handles retries, pending questions, cooldowns, rolling observations, and workflow history rollover. This allows Wisp to maintain context across repeated observation cycles rather than relying on isolated requests.\n\nWisp uses a recurring observation cycle:\n\nThis is what turns Wisp from a screenshot-to-prompt utility into a context-aware desktop system.\n\nWisp's core idea would lose much of its value if it depended entirely on a proprietary cloud AI service.\n\nThe product is built around the idea that your desktop context should remain under your control.\n\nOpen-source AI and local inference make that possible.\n\nWith Ollama, Wisp can use locally available vision models without being tied to a single proprietary provider. Developers can experiment with different models, inspect the inference pipeline, modify the architecture, and build on top of the project.\n\nThere is also a deeper reason.\n\nScreen content can contain source code, private messages, documents, personal information, and sensitive work. Sending all of that to an external AI service introduces a dependency that conflicts with the product's privacy goals.\n\nWisp processes screen images locally. On Linux, screenshot bytes remain in memory during processing and are not stored in Temporal workflow history. Window titles, questions, and model observations are retained in local workflow history.\n\nThe project also makes the boundaries explicit: users control screen-sharing sessions, and stopping a session prevents future captures, although an inference operation already in progress may finish.\n\nOpen innovation makes it possible to build an AI experience where the user has more control over the model, the infrastructure, and their data.\n\n**Your AI should understand your workspace without requiring your workspace to leave your machine.**\n\nI didn't want to build another assistant that occupies a window, waits for a prompt, and disappears when you close it.\n\nI wanted to explore a different relationship between humans and AI.\n\nSomething that feels like it belongs on your desktop.\n\nImagine working late at night, switching between your editor, terminal, and documentation. A little ghost quietly floats in the corner. It doesn't demand attention. It doesn't interrupt every action. But when you need help understanding something on your screen, it's already there.\n\nThat is the experience I wanted to build.\n\nWisp takes the idea of a *ghost in the machine* quite literally: a small, intelligent presence living inside your operating system, observing only when you allow it, understanding through local AI, and helping without taking over your workspace.\n\n**Not another chatbot. Not another browser tab. A quiet desktop spirit.**", "url": "https://wpnews.pro/news/wisp-a-quiet-desktop-spirit-local-ai-that-lives-on-your-linux-desktop", "canonical_source": "https://dev.to/ghananilshirpurkar/wisp-a-quiet-desktop-spirit-local-ai-that-lives-on-your-linux-desktop-1b4c", "published_at": "2026-10-31 18:19:22+00:00", "updated_at": "2026-10-31 18:22:28.672766+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "computer-vision", "ai-agents", "developer-tools"], "entities": ["Wisp", "Ghananil Shirpurkar", "Ollama", "Temporal", "Electron", "Linux", "PipeWire", "Wayland"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/wisp-a-quiet-desktop-spirit-local-ai-that-lives-on-your-linux-desktop", "markdown": "https://wpnews.pro/news/wisp-a-quiet-desktop-spirit-local-ai-that-lives-on-your-linux-desktop.md", "text": "https://wpnews.pro/news/wisp-a-quiet-desktop-spirit-local-ai-that-lives-on-your-linux-desktop.txt", "jsonld": "https://wpnews.pro/news/wisp-a-quiet-desktop-spirit-local-ai-that-lives-on-your-linux-desktop.jsonld"}}