{"slug": "i-gave-nemotron-3-5-lightning-eyes-and-ears-with-zero-training-because-the-let", "title": "I gave Nemotron 3.5 Lightning eyes and ears with zero training, because the architecture let me", "summary": "NVIDIA's Nemotron 3.5 Lightning, a 30B-A3B hybrid Mamba MoE agent model released two weeks ago, has been extended to process images and speech without any training, thanks to a geometry coincidence with NVIDIA's Nemotron-3-Nano-Omni. The extension, called Lightning-Omni, attaches the Omni's projectors to Lightning zero-shot, enabling image and audio understanding on stock llama.cpp, with video support via a fork. This demonstrates that perception can be a detachable organ within an architecture family.", "body_md": "NVIDIA released Nemotron 3.5 Lightning two weeks ago: a 30B-A3B hybrid\n\nMamba MoE agent model with 1M context, text only. Tonight I released\n\nLightning-Omni: the same model reading images and understanding speech,\n\nwith no training involved.\n\nThe trick is a geometry coincidence that is not really a coincidence.\n\nLightning shares the exact backbone shape of NVIDIA’s Nemotron-3-Nano-Omni\n\n(nemotron_h, hidden 2688, 52 layers, same 131072 vocabulary). The Omni’s\n\nprojectors were trained by NVIDIA to translate C-RADIO vision features and\n\nParakeet audio features into precisely that embedding geometry. So the\n\ntowers and projector attach to Lightning zero-shot: download, load, works.\n\nWhat I measured before claiming anything:\n\nEvery quantization passes text, image, and audio gates before upload; the\n\ngraded matrix ships in the repository and grows as tiers land:\n\nRuntime: image, text and audio run on stock llama.cpp with these files.\n\nVideo needs my fork (the Nemotron video graph is not upstream yet), with\n\nprebuilt arm64 CUDA binaries:\n\nThis is also a small existence proof for a general pattern: within an\n\narchitecture family, perception can be a detachable organ. Any future\n\nnemotron_h checkpoint with this geometry can be given the same eyes and\n\nears in an evening. Questions welcome, and if you try it on your own\n\nhardware I would love to see results.", "url": "https://wpnews.pro/news/i-gave-nemotron-3-5-lightning-eyes-and-ears-with-zero-training-because-the-let", "canonical_source": "https://discuss.huggingface.co/t/i-gave-nemotron-3-5-lightning-eyes-and-ears-with-zero-training-because-the-architecture-let-me/179222#post_1", "published_at": "2026-08-24 19:41:47+00:00", "updated_at": "2026-08-24 19:44:56.071686+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "ai-research", "ai-products"], "entities": ["NVIDIA", "Nemotron 3.5 Lightning", "Nemotron-3-Nano-Omni", "Lightning-Omni", "C-RADIO", "Parakeet", "llama.cpp"], "alternates": {"html": "https://wpnews.pro/news/i-gave-nemotron-3-5-lightning-eyes-and-ears-with-zero-training-because-the-let", "markdown": "https://wpnews.pro/news/i-gave-nemotron-3-5-lightning-eyes-and-ears-with-zero-training-because-the-let.md", "text": "https://wpnews.pro/news/i-gave-nemotron-3-5-lightning-eyes-and-ears-with-zero-training-because-the-let.txt", "jsonld": "https://wpnews.pro/news/i-gave-nemotron-3-5-lightning-eyes-and-ears-with-zero-training-because-the-let.jsonld"}}