{"slug": "litert-js-run-ai-models-in-the-browser-no-server", "title": "LiteRT.js: Run AI Models in the Browser, No Server", "summary": "Google launched LiteRT.js on July 9, a JavaScript runtime that runs AI models entirely in the browser via WebAssembly, eliminating server costs and data leaving the device. It outperforms TensorFlow.js and ONNX Runtime Web by up to 3x on CPU/GPU and up to 60x using WebGPU, with a simplified model conversion pipeline from PyTorch to .tflite.", "body_md": "Google shipped [LiteRT.js](https://developers.googleblog.com/litertjs-googles-high-performance-web-ai-inference/) on July 9 — a JavaScript runtime that executes AI models entirely inside the browser with no server required. It runs up to 3x faster than TensorFlow.js and ONNX Runtime Web on CPU and GPU benchmarks, and between 5x and 60x faster when it taps the GPU via WebGPU. One `npm install`\n\n, and your per-query inference cost drops to zero.\n\n## What LiteRT.js Actually Is\n\nLiteRT is Google’s successor to TensorFlow Lite — a runtime for on-device AI inference. LiteRT.js is the JavaScript binding of that same runtime, compiled to WebAssembly, so it runs in browsers. It loads `.tflite`\n\nmodels and executes them client-side. No API calls. No GPU server. No data leaving the device.\n\nThe key departure from TensorFlow.js is architectural. TF.js runs JavaScript kernels — fast by JS standards, slow by native code standards. LiteRT.js routes through WebAssembly, which means you get the same optimized runtime Google uses on Android, now running in Chrome or Firefox.\n\n## Three Backends: Pick Your Hardware\n\nLiteRT.js exposes three execution backends, and the choice matters for performance:\n\n**wasm**— CPU inference via XNNPACK, Google’s highly optimized kernel library with multi-threading and relaxed SIMD. Works in every modern browser, no setup required.**webgpu**— GPU inference via ML Drift, Google’s current-generation GPU engine. Targets the WebGPU API. Stable in Chrome, Edge, and Firefox. This is the one you want for demanding workloads.**webnn**— NPU inference via the[WebNN API](https://learn.microsoft.com/en-us/windows/ai/directml/webnn-overview). Experimental. Requires Chrome or Edge with flags enabled, plus Windows 11 24H2+. Do not ship this to production today.\n\nFor most real-time applications — object detection, image processing, audio transcription — reach for `webgpu`\n\n. WebNN will matter in 2027 when the spec stabilizes and NPU hardware is everywhere. For now, it is a preview.\n\n## Performance Numbers\n\nGoogle’s benchmarks, run on an M4 MacBook Pro, show LiteRT.js delivering up to 3x speedups over TensorFlow.js and ONNX Runtime Web on both CPU and GPU. Move from `wasm`\n\nto `webgpu`\n\nand demanding workloads — real-time object tracking, audio processing, image manipulation — jump 5x to 60x.\n\nTake those numbers seriously but not literally. Performance scales with your user’s GPU, thermal envelope, browser driver quality, and model size. The headline numbers represent an upper bound on modern Apple Silicon, not an average across the fleet.\n\n## Getting Started\n\nInstall from [npm](https://www.npmjs.com/package/@litertjs/core):\n\n```\nnpm install @litertjs/core\n```\n\nCopy the Wasm files from `node_modules/@litertjs/core/wasm/`\n\nto your public directory — LiteRT.js needs to load them at runtime. Then, a typical inference flow:\n\n``` js\nimport { loadLiteRt, loadAndCompile } from '@litertjs/core';\n\n// Initialize the Wasm runtime\nawait loadLiteRt('path/to/your/wasm/');\n\n// Load and compile the model, selecting a backend\nconst model = await loadAndCompile(\n  '/models/mobilenet_v2.tflite',\n  { accelerator: 'webgpu' }\n);\n\n// Run inference\nconst results = await model.run(inputTensor);\nconst output = await results[0].data();\n\n// Always clean up\ninputTensor.delete();\nresults[0].delete();\n```\n\nMemory management is manual. Call `.delete()`\n\non every tensor or you will leak. This is a WebAssembly reality, not a LiteRT.js quirk.\n\n## Model Conversion: The Old Pain Is Gone\n\nThe previous TensorFlow.js workflow involved four conversion steps: PyTorch → ONNX → TensorFlow → TensorFlow.js format. Each handoff introduced potential shape mismatches, quantization issues, and debugging sessions.\n\nLiteRT.js collapses this to one step. PyTorch models convert to `.tflite`\n\ndirectly via [LiteRT Torch](https://developers.google.com/edge/litert/web/get_started). TensorFlow and JAX models follow the same path. If you have been avoiding browser AI inference because the conversion pipeline was painful, that excuse is now gone.\n\n## What You Can Build\n\nGoogle shipped working demos: YOLO-based object detection running in the browser with no server, a webcam feed converted to a 3D point cloud using Depth-Anything-V2 in real time, and 4x image upscaling via Real-ESRGAN running entirely client-side.\n\nThe cost math is straightforward. On-device inference runs roughly 90% cheaper than cloud equivalents for high-volume apps. A $0.50-per-query cloud call becomes $0.05 on-device, and at thousands of queries per hour, that is not a marginal savings — it is a different business model.\n\n## The TensorFlow.js Verdict\n\nTensorFlow.js is not going anywhere. It has years of ecosystem depth, community tutorials, and compatibility with the SavedModel format. But for new projects where you control the model format and care about performance, LiteRT.js is the better starting point. See the [official GitHub repo](https://github.com/google-ai-edge/LiteRT) for model examples and roadmap.\n\nThe browser is now a serious AI inference runtime. LiteRT.js is the clearest evidence of that yet. The only thing holding it back from production ubiquity is WebNN’s experimental status — and that clock is running.", "url": "https://wpnews.pro/news/litert-js-run-ai-models-in-the-browser-no-server", "canonical_source": "https://byteiota.com/litert-js-run-ai-models-in-the-browser-no-server/", "published_at": "2026-07-11 20:07:48+00:00", "updated_at": "2026-07-11 20:15:05.464607+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-tools", "ai-infrastructure", "developer-tools"], "entities": ["Google", "LiteRT.js", "TensorFlow.js", "ONNX Runtime Web", "WebGPU", "WebNN", "XNNPACK", "ML Drift"], "alternates": {"html": "https://wpnews.pro/news/litert-js-run-ai-models-in-the-browser-no-server", "markdown": "https://wpnews.pro/news/litert-js-run-ai-models-in-the-browser-no-server.md", "text": "https://wpnews.pro/news/litert-js-run-ai-models-in-the-browser-no-server.txt", "jsonld": "https://wpnews.pro/news/litert-js-run-ai-models-in-the-browser-no-server.jsonld"}}