I Let an AI Orb Judge My Facial Expressions While I Code, and Here's What Happened

A deep dive into AURA, the desktop AR companion that watches your face, reads your hand gestures, and — in a previous life — took 35 seconds just to say "hello."

Let me introduce you to ** AURA**, a desktop companion whose entire personality can be summarized as: Per its own README, AURA is built to look at your screen, evaluate your facial expressions, and judge your open browser tabs in real time. No notes. No euphemisms. That's just the mission statement, printed in broad daylight, by the people who made it. Bold. Deranged. Kind of iconic.

It's a semi-transparent holographic orb pretending very hard to be a sentient biological interface, the way a Roomba pretends to have feelings when it gets stuck under the couch. It changes color depending on whether you look focused, happy, or the specific flavor of "deeply stressed by my own code" that only a 2am debugging session can produce. It does not, notably, offer to help you fix the bug. It just watches. Like a nature documentary, except you're the nature.

Before the great rewrite, launching AURA was less "spin up an AI assistant" and more "sit down, we need to talk about your life choices while the computer thinks." It behaved less like software and more like a extremely judgmental houseplant that needed 35 seconds of silent contemplation before it would even acknowledge your existence.

Here's the greatest hits album of suffering, straight from the project's own changelog, presented with the reverence it deserves:

.bat

file meant to start the whole show, had a stray closing parenthesis on line two. The app failed before the app even had a chance to fail properly. It's the software equivalent of tripping over your own shoelaces before you've left the house.Somewhere out there, a Task Manager CPU graph from this era is still in therapy, unpacking the trauma with a licensed professional.

The maintainers apparently got tired of their own creation taking longer to boot than a small Linux distro compiling itself from source on a calculator, and did an honest-to-goodness performance overhaul. The results are, frankly, a little suspicious in a good way — the kind of glow-up you assume must involve at least one crime:

Translation, for the humans in the room: the heavy ML imports got shoved into background threads instead of blocking the front door like a bouncer who insists on checking everyone's ID twice, the two camera-hungry threads got merged into a single, well-mannered CameraManager

that only grabs the webcam once (revolutionary, groundbreaking, someone alert the Nobel committee), and the frontend stopped waiting around for a dev server to compile itself from scratch on every single launch. It now just loads a pre-built static bundle in Electron like a normal, well-adjusted application that has been to therapy.

The telemetry pipeline got a promotion too — from a sad 2Hz polling loop to a snappy 20Hz WebSocket broadcast. Your orb now notices your feelings roughly as fast as your actual friends do, which is either impressive or mildly concerning, depending on how many browser tabs you currently have open and how many of them are, statistically, shopping carts you'll never check out.

Once booted (in a very reasonable 1.5 seconds now, thank you), AURA gets to work reading your face and your hands like a mime interpreting your soul for an audience of one, that audience being you, on a Tuesday, in your pajamas.

Using face-mesh tracking on eyebrow distance and mouth curvature, AURA sorts you into moods and reacts with a whole aesthetic package — color gradient, orb animation, ambient sound, the works, like a mood ring that got a computer science degree and will not stop bringing it up: Hand gestures via MediaPipe Hands trigger real actions on your desktop, turning your hands into a remote control you didn't know you signed up for:

The frontend, rebuilt in Electron with Framer Motion and Tailwind, delivers a full glassmorphism package: a liquid gradient orb, an ambient mood bar across the top of your screen, AR target reticles with crosshairs, a floating quick-action dock, and a tiny picture-in-picture HUD showing your own webcam feed with a face-mesh overlaid on it — so you, too, can watch yourself being watched, in a delightfully unsettling hall-of-mirrors sort of way that nobody asked for but everybody, deep down, kind of wanted.

Under the hood, there's a Python/FastAPI backend built around an AuraBrain

class (yes, it's really called that, no, I did not name it, I would never) that delegates to a `CameraManager`

(perception), `VisionEngine`

(screen OCR via EasyOCR), `VoiceEngine`

(TTS/STT), and `AutomationEngine`

(PyAutoGUI clicks). All of it streams state to an Electron + React frontend over a WebSocket at 20Hz, where components with names like Bubble.tsx

, MoodBar.tsx

, Overlay.tsx

, QuickDock.tsx

, and CameraPreviewHUD.tsx

turn "your webcam thinks you look tired" into actual pixels on your screen, twenty times a second, because your existential dread deserves a smooth frame rate.

It is, essentially, a real-time perception pipeline wearing a very stylish holographic costume, like if ps aux

went to art school.

If you're on Windows, have Python 3.10+ and Node.js 18+, and are emotionally prepared for an orb that watches your face while you write code and occasionally plays melancholic piano at you, the whole thing launches with one double-click of StartAURA.bat . It's MIT licensed, so you're free to fork it, judge its judging code, or build your own emotionally-invested desktop pet that will, unlike your actual pets, never respect your boundaries.

Just maybe don't open it during a live coding interview. The Angry mood's warning tone hitting mid-sentence, right as you're explaining a for

loop, is not the vibe you want your interviewer to hear — and "sorry, that's just my sentient orb, it can tell the interview's going badly" has never once saved anyone's callback.