By Tania Alwan
Stem separation, also called music source separation, takes a finished stereo mix and estimates the individual parts that went into it: vocals, drums, bass and everything else. It powers karaoke apps, remix tools and AI cover workflows. It's also a nice example of an "impossible" inverse problem that deep learning made practical.
This post covers how separation works, how quality is measured, and how to run an open-source model yourself in a few lines of Python.
A stereo mix is, roughly, a sum of sources:
mix(t) = vocals(t) + drums(t) + bass(t) + other(t)
Given only mix, we want to estimate each term. Mathematically that's underdetermined: infinitely many combinations sum to the same signal. Models solve it by learning what each kind of source usually sounds like from large sets of multitrack recordings.
The classic deep-learning approach works in the time-frequency domain.
import numpy as np
import librosa
y, sr = librosa.load("mix.wav", sr=44100, mono=True)
S = librosa.stft(y, n_fft=4096, hop_length=1024)
mag, phase = np.abs(S), np.angle(S)
vocal_mask = model.predict(mag) # shape == mag.shape, values in [0, 1]
vocals = librosa.istft(vocal_mask * mag * np.exp(1j * phase), hop_length=1024)
Weakness: reusing the mixture phase limits quality. When two sources overlap in the same bin, a single mask can't fully separate them, which is where "watery" artefacts come from.
Waveform models like the original Demucs operate directly on raw audio with a convolutional encoder-decoder, sidestepping the phase problem. Hybrid models combine a spectrogram branch and a waveform branch. Hybrid Transformer Demucs (HT Demucs), released by Meta's research team, adds transformer layers between the two branches to model longer-range context.
Other strong families include band-split architectures, which process different frequency bands with dedicated sub-networks. The open research community iterates on these quickly, often through public challenges.
The standard metric is SDR (signal-to-distortion ratio), in decibels; higher is better. Benchmarks typically use the MUSDB18 dataset, which contains multitrack songs split into vocals, drums, bass and other.
Keep two caveats in mind:
Always listen, not just read numbers.
Demucs is open source and installable with pip.
python -m pip install -U demucs
demucs song.mp3 # default model, 4 stems
demucs --two-stems=vocals song.mp3 # vocals + no_vocals only
Outputs are written to a separated/ folder, organised by model name and track. Check demucs --help for the model options available in your installed version.
Practical notes:
import subprocess, pathlib
for f in pathlib.Path("tracks").glob("*.wav"):
subprocess.run(["demucs", "--two-stems=vocals", str(f)], check=True)
If you're comparing models for a project, a simple, repeatable test beats impressions.
import soundfile as sf, numpy as np
mix, _ = sf.read("song.wav")
stems = [sf.read(f"separated/htdemucs/song/{s}.wav")[0] for s in ("vocals","drums","bass","other")]
residual = mix - np.sum(stems, axis=0)
print("residual RMS:", np.sqrt(np.mean(residual**2)))
and why they happen
Run it yourself when:
Use a hosted service when:
On the hosted side there are options like LALAL.AI, Moises and the Voicestars stems splitter, which outputs six stems (vocals, drums, bass, guitars, piano, other). Pricing models differ (credits, subscriptions, per-minute rates), so compare against your expected volume.
Take a song you own, run demucs --two-stems=vocals on it, and listen to the outputs on headphones. Then compare against a hosted tool to see which artefacts each one leaves. If you want a browser comparison point, the Voicestars stems splitter is one option.
Disclosure: this article was written in partnership with Voicestars.