# Building a Robot Owl with No Robotics Experience

> Source: <https://mattcool.tech/posts/hoot-a-small-robot-companion-for-learning/>
> Published: 2026-10-20 19:32:06+00:00

I’m building [Hoot](https://github.com/matthewbcool/Hoot), a small robot companion for learning. A little owl that can look at what I’m working on, talk things through with me, and help me practice. It’s my noncompetitive project for the [Nebius × NVIDIA Global AI Hackathon](https://nebiusglobalaihackathon.devpost.com/).

## Learning together

Chinese storytime is my starting point. I’ve been [making tools to help me study Chinese](https://mattcool.tech/posts/hands-free-chinese-flashcard-maker/) for a while, and Hoot builds on that: capture book pages, explore unfamiliar characters, and save words for later practice. I’d like to see where else a little learning companion could be useful.

## What’s inside

- **Conversation and vision:** NVIDIA Nemotron 3.5 Super VL through[Nebius Token Factory](https://docs.tokenfactory.nebius.com/quickstart) , with[Hermes](https://hermes-agent.nousresearch.com/docs/) managing context and tools.
- **The physical owl:** a[Jetson Orin Nano](https://developer.nvidia.com/embedded/learn/getting-started-jetson) , camera, USB microphone and speaker, and[Sense HAT](https://www.raspberrypi.com/documentation/accessories/sense-hat.html) sensors and LED expressions. Visual indicators show when the microphone is recording, when the camera is on, and when Hoot is speaking.
- **English and Mandarin voice:** local[Faster-Whisper](https://github.com/SYSTRAN/faster-whisper) recognition and[Kokoro](https://github.com/hexgrad/kokoro) speech, or optional[Gemini transcription](https://ai.google.dev/gemini-api/docs/transcribe) and[TTS](https://ai.google.dev/gemini-api/docs/speech-generation) . In cloud mode, Nemotron still writes the answers.
- **Reading and review:** captured pages, editable text,[pinyin](https://github.com/zh-lx/pinyin-pro) , saved vocabulary, and practice cards, backed by local[SQLite](https://nodejs.org/api/sqlite.html) .
- **Web dashboard:**[Vite](https://vite.dev/guide/) and[Three.js](https://threejs.org/docs/) power the animated owl, camera preview, chat, and device controls.

## Hoot on screen

The dashboard runs in the browser. The Jetson serves it through a local web server, which connects to Hermes running on the device.

It’s a working prototype. In cloud mode, Hoot needs an internet connection to Nebius for the assistant’s reasoning. Speech can already run locally, and a local mode could use a model running directly on the Jetson Orin Nano for the reasoning too.

## A shoulder owl?

I already have a battery for Hoot, and I want to work out a way to mount it on my shoulder. I’d also like to be able to position it in different places around a room, wherever it has a useful view of what I’m working on.

The shoulder mount is still a plan. But having a little owl hanging out on my shoulder while I learn something sounds pretty fun.

On the software side, I want faster responses, better reading alignment, and pronunciation help that notices when I hesitate or get stuck. Still plenty to work on, but I’m glad to have the pieces working together.