Show HN: Duetto – pretrained music transformer that plays a piano with you

Riccardo Cataldi released Duetto, an open-source system that pairs a live MIDI keyboard with the pretrained Anticipatory Music Transformer (trained on Lakh MIDI) to generate a second piano part in real time, running the model on an Nvidia Jetson Orin Nano with JetPack 6 while a laptop handles MIDI, FluidSynth sound and a piano roll display. Duetto offers presets including duet, call & response, it comps and it solos, plus controls for density, harmony, role, dialogue, groove, risk, focus, key follow, scale and tempo, and is licensed under PolyForm Noncommercial 1.0.0, which permits noncommercial use, modification and sharing but bars commercial use.

duo-f-blues.mp4 Play a MIDI keyboard and a transformer plays a second piano with you, in real time. It listens like a partner: your key, the chord you play, your pulse, your phrases. It comps while you play and takes the melody when you rest. The model is the pretrained Anticipatory Music Transformer https://github.com/jthickstun/anticipation Lakh MIDI , running on a Jetson; the laptop handles MIDI, sound and the piano roll. duo-you-comp-it-solos.mp4 Above: you play low chords, the model improvises over them the it solos preset . Files in samples/duo https://github.com/RiccardoCataldi/duetto/blob/main/samples/duo . ssh -t jetson sudo jetson clocks after every Jetson reboot ./run duet.sh on the laptop run duet.sh restarts everything: the model server on the Jetson, the SSH tunnel, FluidSynth, main.py on the keyboard and the piano roll. Closing the piano roll stops the duo and saves it in generated/ . Without a Jetson: python -m music transformer.server in one terminal, python main.py in another Ctrl+C stops and saves . Setup - Jetson Orin Nano, JetPack 6 : the repo in ~/Ai-Duet with a venv made with --system-site-packages NVIDIA's torch , the model as safetensors in ~/models/music-medium-800k , power mode MAXN SUPER, and ssh jetson working with a key. - Laptop : FluidSynth with a General MIDI soundfont SOUNDFONT=path/to.sf2 ./run duet.sh for a sampled grand , and a venv with pip install mido python-rtmidi pygame imageio-ffmpeg . - Code changes : the Jetson does not pull from git; copy the files over scp music transformer/ .py jetson:~/Ai-Duet/music transformer/ and run ./run duet.sh again. The session tests need torch: run python -m unittest tests.test duo session there. - Logs : ~/amt server.log on the Jetson; duet.log , fluidsynth.log , pianoroll.log in /tmp on the laptop. The top of the piano roll says what the model hears key, chord, tempo and what it is doing. Keys: F fullscreen, K show or hide the controls, Esc close. Controls are sliders drag them; hold Shift to drag finely and steppers click the arrows . The wheel or the arrow keys over a control turn it, and the wheel elsewhere scrolls the panel. A double click resets a control, and the bottom of the panel says what the control under the mouse does. A control is dimmed while what it shapes is not happening. - AI mind - Presets: duet , call & response , it comps it plays the harmony, you improvise , it solos you play the harmony, it improvises , free - Musical controls: density, harmony, role, dialogue, groove, risk, focus, key follow - Range: lowest , highest octave - Scale: root and scale force it into a scale C major, D harmonic minor, F minor pentatonic... instead of your key - Tempo: bpm fixes its tempo instead of following yours, metronome clicks it - Presets: - Your sound , AI sound : instrument, volume, pan, envelope, reverb and delay, in FluidSynth - Advanced : every knob of the model, described in KNOBS in music transformer/duo.py https://github.com/RiccardoCataldi/duetto/blob/main/music transformer/duo.py , where the defaults are too Knobs and sound are kept in ~/.config/ai-duet/ . PolyForm Noncommercial 1.0.0 https://github.com/RiccardoCataldi/duetto/blob/main/LICENSE : use it, change it and share it freely for anything noncommercial personal, study, research, education, hobby . Commercial use is not allowed. The model itself is the Anticipatory Music Transformer https://github.com/jthickstun/anticipation , under its own license.