Have you ever wanted a Jarvis? Not a toy demo, but something that actually controls your computer, understands what you're asking, and takes action? That's exactly what I built with NexusOS
NexusOS is an AI-powered operating environment that runs locally. You say "Hey Nexus", it wakes up, listens, figures out what you want, and does it. Open Chrome, search for something, move files around, run a script, or control your smart home devices — all through voice or text, without touching the keyboard
🛠️ What it does:
Voice Pipeline — Listens for a wake word, transcribes with Whisper, and speaks back using TTS. The whole loop takes about 2-3 seconds [2†L14-L17].
Computer Control — Moves your mouse, clicks buttons, and types text using PyAutoGUI
Browser Automation — Built on Playwright, it can open browsers, navigate, fill forms, and read page content back to you [2†L20-L22].
File Management — Create, move, rename, search, and delete files through voice commands .
Autonomous Workflows — Define multi-step routines like "morning routine" that run in sequence .
AI Memory — Remembers things across sessions using vector embeddings, so it actually knows context from previous conversations .
Smart Home — Connects to MQTT, working with Home Assistant and most IoT devices .
Plugin System — Drop a Python file into the plugins folder and it loads automatically .
🧰 Tech Stack
The backend is Python with FastAPI, Whisper for speech recognition, PyAutoGUI for computer control, Playwright for browser automation, and SQLite with vector embeddings for memory. The frontend is React with Tailwind CSS. Everything runs in Docker .
🔗 Get Started
Check out the repo: (https://github.com/isidhartha/nexus-os)