{"slug": "mlvc-microsoft-builds-an-ml-video-codec-for-npus", "title": "MLVC: Microsoft Builds an ML Video Codec for NPUs", "summary": "Microsoft has open sourced ML Video Codec (MLVC), a neural network-based video codec that reduces bandwidth to about one-eighth of H.264 while maintaining comparable quality, using Neural Processing Units (NPUs) for efficient processing. The codec is already powering video calls in Microsoft Teams, and the company is seeking third-party contributions to expand hardware support and improve performance.", "body_md": "TL;DR — Key Takeaways\n\n**Microsoft says MLVC can slash video bandwidth requirements to about one-eighth of H.264 while maintaining comparable quality.****NPUs are central to the codec’s efficiency, handling neural video processing with far less power than GPUs.****Microsoft has open sourced MLVC to accelerate support for more hardware platforms and improve real-time performance and resilience.**\n\nLooking for outside help, Microsoft has[ open sourced](https://techcommunity.microsoft.com/blog/linuxandopensourceblog/announcing-the-open-source-release-of-ml-video-codec-mlvc/4539875) a neural network-based video codec that promises to cut storage and bandwidth costs to one-eighth of what can be achieved by hardwired codecs, as shown in early tests against rate-distortion objectives. The technology is built on a new type of coprocessor called a Neural Processing Unit (NPU).\n\n“Neural video codecs have surpassed classical codecs in coding efficiency but remain impractical for deployment due to cross-platform incompatibility and high computational cost,” the researchers write in an [arXiv paper](https://arxiv.org/abs/2606.28027).\n\nThe company’s ML Video Codec (MLVC) aims to productize the neural video codec for everyday use, so it can be deployed in streaming video, teleconferencing, cloud gaming, surveillance, augmented and virtual reality and many other areas.\n\nIn the best practice of drinking your own champagne, Microsoft is already using the technology to power video calls in its Microsoft Teams enterprise workspace software, with the traditional codecs serving as backup.\n\n**Codecs are Long-Term Projects**\n\nBuilding a new generation of video codecs is difficult work. Videos are huge files: A codec compresses a bandwidth-heavy video at the point of origin and then decodes it at the destination for the user.\n\nEach new generation of codecs (such as H.264/AVC and H.265/HEVC) tends to be used for a long time, given the immense amount of engineering that goes into the individual elements such as motion estimation, transforms and entropy modeling.\n\nSo in the age of AI, it would be natural to do the job not with fixed algorithms, but rather with end-to-end learned neural compression.\n\nMLVC builds on a whole family of ML codecs, [DCVC (Deep Contextual Video Compression) family of NVC (Neural Video Codec)](https://github.com/microsoft/DCVC), which the company has been working on since 2021.\n\nThe video is passed through a transformer layer, values are trained against [rate-distortion](https://en.wikipedia.org/wiki/Rate%E2%80%93distortion_theory) objectives, then unpacked at the receiver end using NPUs.\n\n**NPUs Do the Heavy Lifting**\n\nGPUs can do the work themselves, though they require[ a lot more power](https://openaccess.thecvf.com/content/WACV2024/papers/van_Rozendaal_MobileNVC_Real-Time_1080p_Neural_Video_Compression_on_a_Mobile_Device_WACV_2024_paper.pdf#:~:text=Neural%20codecs%20are%20quickly%20closing%20the%20gap,show%20the%20number%20of%20Multiply%2DAccumulate%20(MAC)%20operations.) for the task. Intel, Qualcomm and Apple all have NPUs alongside their CPU cores. MLVC uses no more than 50% of these chips’ native capability to leave room for other tasks.\n\nThe results look promising: MLVC-encoded video requires only about 12% of the bandwidth that H.264-encoded video requires. For a 360p video at 30 fps, H.264 chews up about 1 Mbps, whereas MLVC requires roughly 122 kbps for the same quality.\n\nFor these benchmarks, Microsoft used the [P.910 subjective test](https://www.itu.int/rec/t-rec-p.910/en), with the recently-open sourced [Video Conferencing Dataset (VCD)](https://github.com/microsoft/VCD) ([Demo here](https://aka.ms/mct/media/azLinOss/0724261)).\n\n**Open Source Leads the Way**\n\nThe company has open sourced the work, hoping to get third-party contributions. The team needs help across the board, but particularly seeks more NPU ports (the AMD NPU, for instance, is not yet supported) and work that pushes the frontier of rate distortion.\n\nThe[ MLVC repository](https://github.com/microsoft/mlvc), shared under the MIT License, includes the model source code, trained model weights, training scripts and platform conversion scripts to target different NPUs and runtimes. The code is mostly written in Python, with some C++ at the edges.\n\nThe group’s next goals are to stabilize 540p real-time performance, continue expanding hardware coverage, and improve loss resilience.", "url": "https://wpnews.pro/news/mlvc-microsoft-builds-an-ml-video-codec-for-npus", "canonical_source": "https://techstrong.ai/articles/mlvc-microsoft-builds-an-ml-video-codec-for-npus/", "published_at": "2026-07-27 15:15:47+00:00", "updated_at": "2026-07-27 15:39:24.877128+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-products", "ai-infrastructure", "ai-research"], "entities": ["Microsoft", "ML Video Codec (MLVC)", "Neural Processing Unit (NPU)", "Microsoft Teams", "H.264", "DCVC (Deep Contextual Video Compression)", "Intel", "Qualcomm"], "alternates": {"html": "https://wpnews.pro/news/mlvc-microsoft-builds-an-ml-video-codec-for-npus", "markdown": "https://wpnews.pro/news/mlvc-microsoft-builds-an-ml-video-codec-for-npus.md", "text": "https://wpnews.pro/news/mlvc-microsoft-builds-an-ml-video-codec-for-npus.txt", "jsonld": "https://wpnews.pro/news/mlvc-microsoft-builds-an-ml-video-codec-for-npus.jsonld"}}