{"slug": "litert-js-google-s-high-performance-web-ai-inference", "title": "LiteRT.js, Google's high performance Web AI Inference", "summary": "Google announced LiteRT.js, a JavaScript binding of its LiteRT inference library that enables high-performance AI model execution directly in web browsers via WebAssembly. The tool supports hardware acceleration across CPU, GPU, and NPU, offering up to 3x speedups over prior web solutions while enhancing user privacy and reducing server costs.", "body_md": "We are excited to announce [LiteRT.js](https://developers.google.com/edge/litert/web), a JavaScript binding of LiteRT for running AI directly inside the web browser. By bringing the trusted on-device inference library [LiteRT](https://ai.google.dev/edge/litert) to the web, web developers can now run ML and AI models with maximum performance entirely locally. This means enhanced user privacy, zero server costs, and ultra-low latency for real-time experiences. For developers with existing .tflite models, LiteRT.js makes deployment to mobile and desktop web browsers smoother than ever, serving as a powerful evolution from TensorFlow.js for executing .tflite models.\n\nWhile prior web AI solutions like TensorFlow.js relied on less performant JavaScript-based kernels, we are now making our native, cross-platform runtime with all its optimizations directly available to web developers through WebAssembly. LiteRT.js unlocks impressive performance by running your .tflite models directly in the browser leveraging the state-of-the-art hardware acceleration of LiteRT, including [XNNPACK](https://github.com/google/XNNPACK) for CPU, [ML Drift](https://developers.googleblog.com/litert-maximum-performance-simplified/#:~:text=MLDrift%3A%20Best%20GPU%20Acceleration%20Yet) for GPU, and the upcoming WebNN for NPUs.\n\nOur initial release provides all the tools needed to get started, including the new [LiteRT.js npm package](https://www.npmjs.com/package/@litertjs/core) and a [collection of demos](https://codepen.io/collection/PoJBoq) showcasing real-world implementation.\n\nWith LiteRT.js, web developers can integrate models into their apps written in JavaScript or TypeScript to handle complex tasks like text generation, object detection, and audio processing entirely client-side. As LiteRT.js shares a unified cross-platform stack with LiteRT, your web applications automatically benefit from the latest performance upgrades, quantization improvements, and hardware optimizations developed for Android, iOS, and desktop.\n\nBy leveraging LiteRT's lowering flow and runtime, you get simple conversion of models from a variety of Python ML frameworks and native hardware acceleration across all major accelerators (CPU / GPU / NPU). To help you unlock these AI capabilities easily, here are the main highlights of LiteRT.js:\n\n**1.PyTorch conversion & tailored quantization**\n\nWith [LiteRT Torch](https://github.com/google-ai-edge/ai-edge-torch), PyTorch models can be converted in a single step, making them instantly ready to leverage advanced browser-based hardware acceleration. Get started today by following the[ LiteRT Torch guide](https://github.com/google-ai-edge/ai-edge-torch/blob/main/docs/pytorch_converter/getting_started.ipynb).\n\nFor further optimization, [AI Edge Quantizer](https://github.com/google-ai-edge/ai-edge-quantizer) allows you to configure tailored quantization schemes across different model layers. This achieves substantial size reductions and performance gains while preserving overall model quality. Explore the [quantization colab](https://github.com/google-ai-edge/ai-edge-quantizer/blob/main/colabs/selective_quantization_isnet.ipynb) to see this in action.\n\n**2.Native hardware acceleration across CPU, GPU, and NPU**\n\nLiteRT.js enables high-performance AI inference for a diverse variety of hardware backends.\n\nReady to accelerate your web applications? Dive into the [LiteRT.js documentation](https://developers.google.com/edge/litert/web/get_started) to get started.\n\nTo demonstrate the real-world impact of the unified runtime and hardware-accelerated backends, we evaluated LiteRT.js against existing web solutions. Across classical computer vision and audio processing models, LiteRT.js delivers significant speedups—outperforming other web runtimes by up to 3x across both CPU and GPU inference.\n\nTo ground these claims in real-world efficiency, we benchmarked popular AI models using LiteRT.js across three distinct web execution backends: **CPU (via XNNPACK)**, **WebGPU**, and **WebNN (via Apple CoreML).** For demanding real-time applications like object tracking, audio transcription, or image manipulation, leveraging the GPU or NPU via WebGPU or WebNN delivers 5-60x speedup compared to standard CPU execution, ensuring lower latency without compromising performance.\n\nTo see LiteRT.js in action, explore our live implementations. LiteRT.js demo source code is available on the [LiteRT GitHub repository](https://github.com/google-ai-edge/LiteRT/tree/main/litert/js) and via [Ultralytics](https://github.com/ultralytics/ultralytics).\n\nUltralytics is an artificial intelligence company that specializes in building computer vision tools and models. It is best known as the creator of the YOLO (You Only Look Once) framework, family of real-time object detection and image segmentation models.\n\nWe are excited to share [official LiteRT export](https://docs.ultralytics.com/integrations/litert) support built directly into the Ultralytics Python package. Easily deploy [Ultralytics YOLO](https://www.ultralytics.com/yolo/yolo26?utm_source=googlelitert&utm_medium=referral&utm_campaign=blog&utm_content=yolo26) models across mobile, edge, and browsers—and go from compilation to runtime in just a few lines of code.\n\n[Depth Anything - monocular depth estimation](https://goo.gle/depth3d) showcases how to transform a standard webcam feed into an interactive 3D point cloud in real-time. Powered by LiteRT.js via WebGPU, it uses the Depth-Anything-V2 model to instantly calculate depth data and map video pixels into a responsive 3D space.\n\nUpscale images by 4x in the browser using the [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) model with LiteRT.js, which works by upscaling 128x128 pixel patches to 512x512 which are then reassembled into the final image.\n\nIntegrating LiteRT.js into your development workflow is straightforward, whether you’re launching a fresh implementation or migrating an existing application to our high-performance runtime. LiteRT.js abstracts the complexities of hardware-level optimization, enabling you to deliver responsive, privacy-focused experiences without the overhead of manual platform tuning.\n\nThe following snippet highlights the streamlined process for initializing, compiling, and running a `.tflite`\n\nmodel with GPU acceleration. Using clean, modern JavaScript, you can load your model, feed input tensors, and capture high-speed inference results in real-time. For more detailed instructions, demos, and guidance, please refer to our documentation [here](https://developers.google.com/edge/litert/web/get_started).\n\n``` js\nimport { loadLiteRt, loadAndCompile, Tensor } from '@litertjs/core';\n\nawait loadLiteRt('path/to/wasm/directory/');\n\nconst model = await loadAndCompile('path/to/your/model.tflite',{ accelerator: webgpu });\n\nconst inputTypedArray = new Float32Array(1 * 3 * 244 * 244);\n\nconst inputTensor = new Tensor(inputTypedArray, [1, 3, 244, 244]);\n\nconst results = await model.run(inputTensor);\n\n// results is a Tensor stored on GPU. To move it to CPU & convert to a typedArray we use\nconst resultArray = (await results[0].moveTo('wasm')).toTypedArray();\n```\n\nWe are committed to continually expanding LiteRT.js performance, model coverage, and developer tooling. Looking ahead, our development roadmap centers on advancing WebNN integration for native NPU performance and delivering highly optimized support for on-device generative AI.\n\nUltralytics, for providing YOLO26 media and performance data. Jason Mayes for LiteRT.js demos.", "url": "https://wpnews.pro/news/litert-js-google-s-high-performance-web-ai-inference", "canonical_source": "https://developers.googleblog.com/litertjs-googles-high-performance-web-ai-inference/", "published_at": "2026-07-09 21:07:43.908390+00:00", "updated_at": "2026-07-09 21:07:46.257758+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-tools", "ai-infrastructure", "developer-tools"], "entities": ["Google", "LiteRT.js", "TensorFlow.js", "XNNPACK", "ML Drift", "WebNN", "LiteRT Torch", "AI Edge Quantizer"], "alternates": {"html": "https://wpnews.pro/news/litert-js-google-s-high-performance-web-ai-inference", "markdown": "https://wpnews.pro/news/litert-js-google-s-high-performance-web-ai-inference.md", "text": "https://wpnews.pro/news/litert-js-google-s-high-performance-web-ai-inference.txt", "jsonld": "https://wpnews.pro/news/litert-js-google-s-high-performance-web-ai-inference.jsonld"}}