Show HN: Tiny ONNX Runtime Web Worker Starter

A developer released a starter kit for running ONNX Runtime Web inside a Web Worker to keep browser-based machine learning inference off the UI thread, improving performance on mobile devices. The project provides reusable plumbing including a classic worker, promise-based client wrapper, and optional React hook, with instructions for copying runtime files and models into a public folder.

A small starter for running ONNX Runtime Web inside a Web Worker. I pulled this pattern out after dealing with browser-side inference that was fine on desktop, then painful on mobile. Moving the model load and session.run work into a worker keeps the UI thread free, and the app feels much less fragile while inference is running. This repo is just the reusable plumbing: public/onnx-worker/onnxWorker.js classic worker, loads ONNX Runtime src/lib/OnnxWorkerClient.js promise-based wrapper around postMessage src/hooks/useOnnxWorker.js optional React hook example/DemoComponent.jsx tiny usage example test/OnnxWorkerClient.test.js client protocol tests Bring your own model, inputs, preprocessing, and UI. The worker uses importScripts to load ort.min.js , so it needs to be a classic worker with stable static URLs. I first tried the more obvious new URL "../worker/onnxWorker.js", import.meta.url approach. It can work in some builds, but in Next.js it is easy to hit dev/prod differences: the worker may be emitted under a hashed URL, or dev tooling may inject code that a classic worker cannot parse. Serving the worker and ONNX Runtime files from public/onnx-worker/ is boring, but boring is exactly what I want here. Install ONNX Runtime Web in your app: npm install onnxruntime-web Copy the worker and runtime files into your app's public folder: mkdir -p public/onnx-worker cp public/onnx-worker/onnxWorker.js your-app/public/onnx-worker/onnxWorker.js cp your-app/node modules/onnxruntime-web/dist/ort.min.js your-app/public/onnx-worker/ort.min.js cp your-app/node modules/onnxruntime-web/dist/ .wasm your-app/public/onnx-worker/ Put your model somewhere public too: public/models/my-model.onnx js "use client"; import { useOnnxWorker } from "./src/hooks/useOnnxWorker.js"; export function MyInferenceButton { const { status, progress, error, run } = useOnnxWorker { modelUrl: "/models/my-model.onnx", workerUrl: "/onnx-worker/onnxWorker.js", runtimeUrl: "/onnx-worker/ort.min.js", wasmPaths: "/onnx-worker/", } ; async function handleClick { const outputs = await run { input: { type: "float32", dims: 1, 3, 224, 224 , data: new Float32Array 1 3 224 224 , }, } ; console.log outputs ; } return <button onClick={handleClick} disabled={status == "ready"} {status === "running" ? "Running..." : "Run inference"} {progress ? ${progress.percent}% : ""} {error ? ${error} : ""} </button ; } js import { OnnxWorkerClient } from "./src/lib/OnnxWorkerClient.js"; const client = new OnnxWorkerClient { workerUrl: "/onnx-worker/onnxWorker.js", onProgress: progress = console.log progress , } ; await client.preload { modelUrl: "/models/my-model.onnx", runtimeUrl: "/onnx-worker/ort.min.js", wasmPaths: "/onnx-worker/", } ; const outputs = await client.run { modelUrl: "/models/my-model.onnx", runtimeUrl: "/onnx-worker/ort.min.js", wasmPaths: "/onnx-worker/", inputs: { input: { type: "float32", dims: 1, 4 , data: new Float32Array 1, 2, 3, 4 , }, }, } ; preload loads ONNX Runtime and the model before the first run. run sends tensor-like inputs to the worker and resolves serialized ONNX output tensors. dispose terminates the worker and rejects pending requests.- The worker serializes requests and keeps one cached session. requestTimeoutMs is optional. By default there is no timeout. Input tensors should match what ONNX Runtime expects: { type: "float32", dims: 1, 3, 224, 224 , data: new Float32Array ... } npm test The tests cover the client/worker message protocol. You should still test with your real model in a browser, because ONNX Runtime's wasm files and execution providers can vary by version. A worker keeps inference off the UI thread, but it does not magically make big models cheap. If you are working with images, crop and resize before inference when you can. That one change often matters more than any clever code. This starter was extracted from production code while building Lumli https://lumli.io . Lumli is a privacy-first browser-native AI studio where AI models run locally on your device. MIT