Inspiration #
DJing is one of the last creative crafts that software never truly automated — because the hard part was never the buttons, it's the judgment. When to tease the next record in, when to slam the drop, how hard to ride the pitch. AI music tools generate songs; DJ software syncs buttons. Nobody had handed an AI agent the booth itself - the decks, the mixer, the FX, and asked it to be good.
WebMCP made that question askable. Instead of an agent fumbling through a UI built for hands, we could expose the craft as structured tools and see whether an agent could develop taste: compose a set, perform it live, listen to its own work, and fix what sucked. BananaLabs DJ is our answer.
What it does #
BananaLabs DJ is a web DJ workstation that a person and an agent can play together - same decks, same document, same tools.
Upload a handful of tracks and the app measures them in a worker: BPM, musical key, drop points, energy curves, harmony (chroma), vocal regions. From there the agent takes over like a real DJ:
It composes— picks track order by solving a mixability path over real join costs, cuts drop-anchored clips (never radio intros), and compiles every transition from a craft grammar: tease-slams that bleed the next record's build in under the current one, tempo and pitch rides, loop rolls, echo throws, backspins - every drop lands exactly on the 1.It performs— live in the browser on two tempo-matched decks, a 3-band EQ/filter mixer, crossfader, delay-throw FX bus and sampler pads, with SoundTouch keylock and a dosed vinyl "pitch scream" on rides.It listens to itself— review_set bounces the set offline and measures every join from the rendered audio: dead air, level jumps, bass stacking, masked first transients, slam punch. If something's rough, it gain-stages the levels and re-measures until clean.It ships— verify gates the document, then the set plays live, exports to WAV, or shares as a .blset. The human can interrupt at any moment - grab a fader, rewrite a join, or just enjoy the set. Live app:https://bananalabs-sable.vercel.app/· Code (MIT):https://github.com/KushagraAgarwal525/webmcp-dj
How we built it #
The core decision: one document, one pipeline, two performers. Every state change - from a human dragging a crossfader to an agent recompiling a transition - flows through the same SetDoc command pipeline, so human and agent never fork the mix.
On top of that:
WebMCP tool surface— 66 tools registered with document.modelContext.registerTool, in two curated tiers: a compose-first surface (compose, audition, verify, review, play) and a booth-hardware tier (?booth=1) exposing loops, hotcues, EQ, sampler, FX. Per-tool output budgets keep results semantic, never truncated mid-JSON, and every mutation echoes a compile report (where the commit lands, whether it's on the drop):document.modelContext.registerTool({ name: "review_set", description: "Bounce the arrangement offline and MEASURE each join from the rendered audio…", inputSchema: { /* index?, fix? */ }, execute: async (input) => { /* render → measure → report/fix */ }, });
Measurement, not narration— a worker analysis pipeline (STFT chroma, EDM-profile key detection, flux beatgrid with downbeats, salience-based drop detection, vocal regions) gives the composer ground truth; the composer never guesses bar math.The agent's ear— an OfflineAudioContext bounce renders the set with full parity to live playback (loop rolls, backspin rewinds, tempo rides included). That one renderer powers both the Download-WAV feature and review_set, so what the agent measures is exactly what the room hears.The loop is documented in the tools— compose with prepare_set, measure the sound with review_set (fix:true to gain-stage), gate with verify_set, then play. The agent never computes bar math and never plays unreviewed when it follows the descriptions.
React + TypeScript + Vite on the front, Dexie/IndexedDB + OPFS for persistence, deployed as a static app on Vercel.
Challenges we ran into #
The first agent sets were bad. Instant track-to-track switches read as random, not mixed. We rebuilt the transition grammar around handoffs - the tease_slam, so the crowd hears the next record arriving before the drop slams on the 1.
The slam still sounded rough - so we measured it. The drop's first transient was landing mid-sweep and several dB hot. The fix (pre-open the incoming a bar early, a 1/16-bar flick on the 1, and per-entry gain staging) was proven by the agent's own review tool, not by assertion: it caught a +5.9 dB hot slam, staged it, and re-measured clean.
Pitch rides went dull before they screamed. Unlocking keylock for a full 16-bar ride made the record creep thin; dosing the unlock to the final 4 bars - stacked with the loop roll and filter rise, made the festival scream. Along the way we found and fixed a keylock restore leak on deck reloads, a set-clock drift during un-laned overlaps, and a clip-clamping bug that silently destroyed drop anchors.
Agents don't fail like users fail. Sixty flat tools invite an agent to push faders instead of compose — so we curated tiers. Agents narrate confidently and wrongly — so tools return measured truth, not stories. And our own live-debugging harness needed watchdogs and streaming logs after too many "stuck forever" sessions; vite HMR never swaps long-lived audio modules, so every proof runs on cache-disabled reloads.
Accomplishments that we're proud of #
An agent that genuinely DJs— composes with craft vocabulary, performs live, and is good enough that a friend judged the set by ear and approved it.** The agent-ear loop**— review_set measures rendered audio and the agent corrects itself without human intervention. Watching it catch and fix a hot slam on its own was the moment the project took shape.Zero prompt magic required— the loop lives in the tool surface. Our most naive test prompt ("compose a dj set using these songs") drove the full compose → measure → fix → verify → play flow.A complete product, not a demo— persistence, undo/redo, WAV export, shareable .blset sets, a proposal banner for agent edits, a full smoke-test suite, and a CDP eval harness that bounces and scorecards every compose.Real DJ details done right— phrase-snapped drops, chroma auditions of actual overlap windows, loop rolls, backspins, echo throws that ring over the next drop.
What we learned #
The tool surface is the prompt. When a naive instruction failed, the fix belonged in the tools - guidance, defaults, and honest echoes - not in cleverer wording. ** Agents need measurements, not assertions. The single biggest quality jump came from giving the agent an objective ear over its own output. Craft followed from feedback.** Design the grammar, not the script. The agent picks what (recipes, order, intent); the engine owns how (bar math, lane compilation, clock). That separation is what made the agent's choices both free and safe. WebMCP's real unlock is shared state. The magic isn't that an agent can click our buttons - it's that two performers can work one instrument.
What's next for BananaLabs - WebMCP DJ #
Calibration corpus— crowd-confirm drop/hook timestamps to keep sharpening the salience detector.** Musical pitch rides**— semitone-quantized landings keyed by Camelot relation, so rides resolve tonally, not just dramatically.** True two-hand collaboration**— human and agent performing the same set simultaneously (the shared document already supports it; the UX is next).** Deeper measurement**— harmonic dissonance scoring from the bounce, energy-arc analysis, and a crowd-simulation score.** The pattern as a template**— "compose → measure your own output → fix → verify → play" generalizes to video editing, motion design, and any creative app on WebMCP. I'd love to package it as the reference pattern for the agent-native creative web.
Built With #
- codex
- indexeddb
- mcp
- opencode
- opfs
[react](https://devpost.com/software/built-with/react)- soundtouch
[typescript](https://devpost.com/software/built-with/typescript)- vercel
- vite
- webmcp