We’ve written a lot lately about agents gaining senses and self-verification — Pixel Path screenshotting its own renders until the sky stops bleeding through buildings, vision models that draw to reason. earshot adds the one sense everyone forgets: hearing.
The premise is sharp. AI coding agents can now write the code that renders a film, a product demo, or a game — but they still cannot hear what they made. They can’t tell a premium key press from a cheap one, notice that a hit lands 80 ms after the cut, or catch that the music swells in the wrong place. earshot fixes exactly that, and it runs locally on CPU as a CLI, Python API, and MCP server.
Turning Sound Into Evidence #
The core idea is to convert audio into something an agent can reason about in words and numbers, not just play back. A sampling of the commands:
describe— what a file sounds like: calibrated tags (e.g.“single mechanical keyboard key press, z +5.4” ), closest alternative readings, AudioSet labels, and raw measurements (peak dBFS, brightness).match— how well a description fits a sound, with a verdict (strong / plausible / weak / no match) and held-out evidence.find— the best moments inside a long recording, returned as precise sample trims.realness— is this a real/pro-produced recording or code-synthesized? With per-window scores.timeline— a second-by-secondlistening transcript of a whole film: events, sections, loudness, beat grid, and how sound syncs to picture cuts.sheet— an annotated PNG (frames, events, spectrogram, waveform, sync markers) for agents that read images.critique— a launch-video scorecard against a profile of 7 professional launch films, withtimestamped fixes (“nothing reaches body level until 0:01.01” ).
That last one is the kicker: the agent doesn’t just hear, it gets graded against professional work and told what to fix, where.
Making Sound in Code — and Hearing It Back #
The other half is generative, and it closes a feedback loop:
forge— synthesize a soundentirely in code , then improve it by listening to the result. (Every sound in the project’s 22-second launch film was made this way — no recordings, no music track.)fit— translate a real recording into aforgerecipe Python module that re-renders it, with knobs and morphs so the agent can vary it.sfx/kit(new in 0.3) — a bank of201 sound effects stored as code that render by id in seconds, no models, no network.sfxaims a whoosh, hit, click, or note at a moment in the picture;kitrenders a whole film’s effects with measured sync points.
Storing effects as code with fingerprints rather than audio files is a neat touch — it’s diffable, version-controllable, and tiny.
Why It Matters #
This is the same deep pattern we keep flagging: software built to be operated by an AI agent, exposing an MCP server so the model gets a real tool instead of a guess. earshot gives an agent a perception-and-critique loop for audio — generate, listen, score, fix — the exact discipline that separates “the code ran” from “the result is actually good.” As agents produce more video, games, and demos, being deaf to their own output is a glaring blind spot. earshot is a thoughtful, honest (it ships VALIDATION and LIMITS docs), CPU-only answer to it.