I built Magnus AI, a voice assistant that helps my friend Vashkar plan meals around his allergies

A developer built Magnus AI, a voice-first Windows assistant for friend Vashkar Ghosh, who must plan meals around his allergies. The assistant streams 16 kHz mono microphone audio over a WebSocket to Google's Gemini Live API, returning 24 kHz audio with roughly 250 ms turn detection and barge-in support, and adds self-describing typed tools, Win32 window control, a PyQt6 HUD, local JSON memory with atomic writes, and a fallback ladder across Gemini models on quota errors. The MIT-licensed code is open, but the developer notes the hosted Gemini Live model means voice and screen data leave the user's machine, with an offline open-weight mode planned next.

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend https://dev.to/challenges/hacktoberfest-weekend-2026-10-01 My friend Vashkar Ghosh has to plan his meals around his allergies. That means reading labels, double-checking ingredients and searching for safe options, again and again. So I built him Magnus AI , a voice-first assistant for Windows. Instead of typing and searching, he can just say what he needs. He tells Magnus what he can't eat, and Magnus suggests meals that avoid those ingredients and keeps his preferences saved for next time. Magnus does more than meals, because a helper that lives on his PC should handle the small daily things too. He can: https://drive.google.com/file/d/137xF8eviOjMR5DFXnfyZKzUN IJyS8Br/view?usp=sharing https://drive.google.com/file/d/137xF8eviOjMR5DFXnfyZKzUN IJyS8Br/view?usp=sharing https://github.com/RIDDHIDEV-OPS/Magnus-AI https://github.com/RIDDHIDEV-OPS/Magnus-AI Real-time voice loop. Microphone audio is resampled to 16 kHz mono and streamed over a WebSocket to the Gemini Live API. Replies come back as 24 kHz audio. There's about 250 ms of turn detection, and barge-in lets Vashkar interrupt mid-sentence the way he would with a person. Self-describing tools. Every file in actions/ exposes one typed function. At startup, Magnus turns the docstrings into function declarations, so adding a new skill means dropping in a single file. Windows control. Win32 APIs SetForegroundWindow , AttachThreadInput , psutil and PyAutoGUI let Magnus find, launch and bring windows to the front. Interface. A PyQt6 HUD with an animated orb, live audio waveforms, and CPU/RAM/GPU telemetry served from a small local FastAPI dashboard. Local memory. Preferences and learned context are stored in a local JSON file with thread-safe atomic writes, and it's git-ignored so personal data never reaches the repo. Resilience. If an API quota error hits, Magnus steps down a fallback ladder of Gemini models instead of going silent. I want to be upfront: the model at the heart of Magnus is Google's Gemini Live API, which is not open-weight. What is open is the code MIT-licensed , the Python ecosystem it's built on PyQt6, psutil, PyAutoGUI, FastAPI , and the Open-Meteo data behind its weather tool. Building this for a friend taught me what that trade-off costs. Meal and allergy information is personal, and with a hosted model, his voice and screen go to a server he doesn't control. Open-weight models would fix that by running on his own laptop, working offline, costing nothing per use, and letting me tune the assistant to him. My next step is an offline mode built on an open-weight model so his data never leaves his machine.