A background-removal API can finish successfully while the exported file is still wrong for the next step. A video that plays on a black background might have lost its transparency—or the player might simply be displaying it against black. Those are different problems.
I work on unbg.video, a video background remover and editor. The project has export paths for WebM, ProRes 4444 MOV, and separate H.264 color and alpha files. Here is a practical way to reason about those outputs, with a small FFmpeg experiment you can run without an AI service.
“Download video” hides several different jobs:
| Next step | Output to consider | What to verify |
|---|---|---|
| Display transparent footage on a website | WebM with alpha | Actual transparency in the target browser and device |
| Continue compositing in an editor | ProRes 4444 MOV with alpha | Import behavior and alpha interpretation in that editor |
| Transfer color and a separate matte | H.264 color + grayscale alpha files | Matching dimensions, timestamps, duration, and mask convention |
| Publish a finished scene with its new background | Flattened H.264 MP4 | The final composition; transparency is no longer needed |
The extension alone does not describe the whole contract. A MOV file does not automatically contain alpha. An ordinary H.264 MP4 export is not a portable RGBA deliverable. In the two-file workflow, the second MP4 stores a grayscale matte as picture data; it is not an embedded alpha channel in the color file.
For straight alpha, the basic compositing relationship is:
output = alpha * foreground + (1 - alpha) * background
Here alpha is normalized from 0 to 1: zero reveals the background, one keeps the foreground, and intermediate values produce partial coverage. For physically meaningful blending, color-space handling also matters; this equation is a conceptual starting point.
A useful consequence: you cannot safely recover transparency by deleting every black pixel. Black clothing and shadows are legitimate foreground content. Keep the matte.
Also establish whether color is straight or premultiplied. With premultiplied color, RGB already includes multiplication by alpha. Multiplying again can darken the edges. If the silhouette looks right but has a dark fringe, inspect this convention before blaming the segmentation model.
This synthetic example creates one second of red video and a horizontal grayscale ramp, combines them, and writes ProRes 4444. It needs an FFmpeg build with prores_ks.
ffmpeg \
-f lavfi -i 'color=c=red:s=64x64:r=10:d=1,format=rgb24' \
-f lavfi -i 'nullsrc=s=64x64:r=10:d=1,format=gray,geq=lum=255*X/W' \
-filter_complex '[0:v][1:v]alphamerge,format=yuva444p10le[v]' \
-map '[v]' -an \
-c:v prores_ks -profile:v 4 -pix_fmt yuva444p10le \
foreground.mov
Then extract a frame of the stored alpha:
ffmpeg -i foreground.mov \
-vf 'alphaextract,format=gray' -frames:v 1 -update 1 mask.png
The expected mask is a horizontal dark-to-light ramp, not a solid white frame. The red image should become progressively more opaque when composited over a background.
This example was run locally with FFmpeg 8.0.1 for this article: the output decoded as ProRes 4444 with an alpha-capable pixel format, and the extracted 8-bit alpha ramp ranged from 0 to 251 (the last column is 63/64 of full scale). It is an export smoke test, not evidence of browser compatibility or segmentation quality. Audio is deliberately omitted.
FFmpeg documents alpha extraction and merging and the ProRes encoder options. When replacing the synthetic inputs with real color and matte files, first align their dimensions and timeline. Resetting a timestamp does not repair missing or mismatched frames.
The project stores multiple video assets with color/alpha variants and can package the H.264 pair into a ZIP. That matters: a single videoUrl cannot tell a consumer whether it has a finished composition or only half of a compositing input.
For a new API, a small discriminated union can make that distinction explicit. This is an illustrative design, not a copy of the project's implementation:
type VideoExport =
| { kind: 'transparent'; url: string; format: 'webm' | 'prores4444' }
| { kind: 'matte-pair'; colorUrl: string; alphaUrl: string }
| { kind: 'flattened'; url: string; format: 'mp4' };
Only mark a pair ready when both assets are available. Give files explicit names such as clip_color.mp4 and clip_alpha.mp4, and document which mask values mean opaque. Otherwise the ZIP merely moves the ambiguity to the user.
My suggested acceptance checklist is short:
The useful product question is: Can the recipient use this file for their next step without guessing? That question should shape the export format, the API response, and the download label.