NVIDIA released Nemotron 3.5 Lightning two weeks ago: a 30B-A3B hybrid
Mamba MoE agent model with 1M context, text only. Tonight I released
Lightning-Omni: the same model reading images and understanding speech,
with no training involved.
The trick is a geometry coincidence that is not really a coincidence.
Lightning shares the exact backbone shape of NVIDIA’s Nemotron-3-Nano-Omni
(nemotron_h, hidden 2688, 52 layers, same 131072 vocabulary). The Omni’s
projectors were trained by NVIDIA to translate C-RADIO vision features and
Parakeet audio features into precisely that embedding geometry. So the
towers and projector attach to Lightning zero-shot: download, load, works.
What I measured before claiming anything:
Every quantization passes text, image, and audio gates before upload; the
graded matrix ships in the repository and grows as tiers land:
Runtime: image, text and audio run on stock llama.cpp with these files.
Video needs my fork (the Nemotron video graph is not upstream yet), with
prebuilt arm64 CUDA binaries:
This is also a small existence proof for a general pattern: within an
architecture family, perception can be a detachable organ. Any future
nemotron_h checkpoint with this geometry can be given the same eyes and
ears in an evening. Questions welcome, and if you try it on your own
hardware I would love to see results.