I’m building Hoot, a small robot companion for learning. A little owl that can look at what I’m working on, talk things through with me, and help me practice. It’s my noncompetitive project for the Nebius × NVIDIA Global AI Hackathon.
Learning together #
Chinese storytime is my starting point. I’ve been making tools to help me study Chinese for a while, and Hoot builds on that: capture book pages, explore unfamiliar characters, and save words for later practice. I’d like to see where else a little learning companion could be useful.
What’s inside #
-
Conversation and vision: NVIDIA Nemotron 3.5 Super VL throughNebius Token Factory , withHermes managing context and tools.
-
The physical owl: aJetson Orin Nano , camera, USB microphone and speaker, andSense HAT sensors and LED expressions. Visual indicators show when the microphone is recording, when the camera is on, and when Hoot is speaking.
-
English and Mandarin voice: localFaster-Whisper recognition andKokoro speech, or optionalGemini transcription andTTS . In cloud mode, Nemotron still writes the answers.
-
Reading and review: captured pages, editable text,pinyin , saved vocabulary, and practice cards, backed by localSQLite .
-
Web dashboard:Vite andThree.js power the animated owl, camera preview, chat, and device controls.
Hoot on screen #
The dashboard runs in the browser. The Jetson serves it through a local web server, which connects to Hermes running on the device.
It’s a working prototype. In cloud mode, Hoot needs an internet connection to Nebius for the assistant’s reasoning. Speech can already run locally, and a local mode could use a model running directly on the Jetson Orin Nano for the reasoning too.
A shoulder owl? #
I already have a battery for Hoot, and I want to work out a way to mount it on my shoulder. I’d also like to be able to position it in different places around a room, wherever it has a useful view of what I’m working on.
The shoulder mount is still a plan. But having a little owl hanging out on my shoulder while I learn something sounds pretty fun.
On the software side, I want faster responses, better reading alignment, and pronunciation help that notices when I hesitate or get stuck. Still plenty to work on, but I’m glad to have the pieces working together.