{"slug": "show-hn-tiny-onnx-runtime-web-worker-starter", "title": "Show HN: Tiny ONNX Runtime Web Worker Starter", "summary": "A developer released a starter kit for running ONNX Runtime Web inside a Web Worker to keep browser-based machine learning inference off the UI thread, improving performance on mobile devices. The project provides reusable plumbing including a classic worker, promise-based client wrapper, and optional React hook, with instructions for copying runtime files and models into a public folder.", "body_md": "A small starter for running ONNX Runtime Web inside a Web Worker.\n\nI pulled this pattern out after dealing with browser-side inference that was\nfine on desktop, then painful on mobile. Moving the model load and\n`session.run()`\n\nwork into a worker keeps the UI thread free, and the app feels\nmuch less fragile while inference is running.\n\nThis repo is just the reusable plumbing:\n\n```\npublic/onnx-worker/onnxWorker.js   classic worker, loads ONNX Runtime\nsrc/lib/OnnxWorkerClient.js        promise-based wrapper around postMessage\nsrc/hooks/useOnnxWorker.js         optional React hook\nexample/DemoComponent.jsx          tiny usage example\ntest/OnnxWorkerClient.test.js      client protocol tests\n```\n\nBring your own model, inputs, preprocessing, and UI.\n\nThe worker uses `importScripts()`\n\nto load `ort.min.js`\n\n, so it needs to be a\nclassic worker with stable static URLs.\n\nI first tried the more obvious `new URL(\"../worker/onnxWorker.js\", import.meta.url)`\n\napproach. It can work in some builds, but in Next.js it is\neasy to hit dev/prod differences: the worker may be emitted under a hashed URL,\nor dev tooling may inject code that a classic worker cannot parse.\n\nServing the worker and ONNX Runtime files from `public/onnx-worker/`\n\nis boring,\nbut boring is exactly what I want here.\n\nInstall ONNX Runtime Web in your app:\n\n```\nnpm install onnxruntime-web\n```\n\nCopy the worker and runtime files into your app's public folder:\n\n```\nmkdir -p public/onnx-worker\ncp public/onnx-worker/onnxWorker.js your-app/public/onnx-worker/onnxWorker.js\ncp your-app/node_modules/onnxruntime-web/dist/ort.min.js your-app/public/onnx-worker/ort.min.js\ncp your-app/node_modules/onnxruntime-web/dist/*.wasm your-app/public/onnx-worker/\n```\n\nPut your model somewhere public too:\n\n```\npublic/models/my-model.onnx\njs\n\"use client\";\n\nimport { useOnnxWorker } from \"./src/hooks/useOnnxWorker.js\";\n\nexport function MyInferenceButton() {\n  const { status, progress, error, run } = useOnnxWorker({\n    modelUrl: \"/models/my-model.onnx\",\n    workerUrl: \"/onnx-worker/onnxWorker.js\",\n    runtimeUrl: \"/onnx-worker/ort.min.js\",\n    wasmPaths: \"/onnx-worker/\",\n  });\n\n  async function handleClick() {\n    const outputs = await run({\n      input: {\n        type: \"float32\",\n        dims: [1, 3, 224, 224],\n        data: new Float32Array(1 * 3 * 224 * 224),\n      },\n    });\n\n    console.log(outputs);\n  }\n\n  return (\n    <button onClick={handleClick} disabled={status !== \"ready\"}>\n      {status === \"running\" ? \"Running...\" : \"Run inference\"}\n      {progress ? ` ${progress.percent}%` : \"\"}\n      {error ? ` ${error}` : \"\"}\n    </button>\n  );\n}\njs\nimport { OnnxWorkerClient } from \"./src/lib/OnnxWorkerClient.js\";\n\nconst client = new OnnxWorkerClient({\n  workerUrl: \"/onnx-worker/onnxWorker.js\",\n  onProgress: (progress) => console.log(progress),\n});\n\nawait client.preload({\n  modelUrl: \"/models/my-model.onnx\",\n  runtimeUrl: \"/onnx-worker/ort.min.js\",\n  wasmPaths: \"/onnx-worker/\",\n});\n\nconst outputs = await client.run({\n  modelUrl: \"/models/my-model.onnx\",\n  runtimeUrl: \"/onnx-worker/ort.min.js\",\n  wasmPaths: \"/onnx-worker/\",\n  inputs: {\n    input: {\n      type: \"float32\",\n      dims: [1, 4],\n      data: new Float32Array([1, 2, 3, 4]),\n    },\n  },\n});\n```\n\n`preload()`\n\nloads ONNX Runtime and the model before the first run.`run()`\n\nsends tensor-like inputs to the worker and resolves serialized ONNX output tensors.`dispose()`\n\nterminates the worker and rejects pending requests.- The worker serializes requests and keeps one cached session.\n`requestTimeoutMs`\n\nis optional. By default there is no timeout.\n\nInput tensors should match what ONNX Runtime expects:\n\n```\n{\n  type: \"float32\",\n  dims: [1, 3, 224, 224],\n  data: new Float32Array(...)\n}\nnpm test\n```\n\nThe tests cover the client/worker message protocol. You should still test with your real model in a browser, because ONNX Runtime's wasm files and execution providers can vary by version.\n\nA worker keeps inference off the UI thread, but it does not magically make big models cheap. If you are working with images, crop and resize before inference when you can. That one change often matters more than any clever code.\n\nThis starter was extracted from production code while building [Lumli](https://lumli.io).\n\nLumli is a privacy-first browser-native AI studio where AI models run locally on your device.\n\nMIT", "url": "https://wpnews.pro/news/show-hn-tiny-onnx-runtime-web-worker-starter", "canonical_source": "https://github.com/makan0713/onnx-web-worker-starter", "published_at": "2026-07-10 14:14:18+00:00", "updated_at": "2026-07-10 14:35:59.588846+00:00", "lang": "en", "topics": ["developer-tools", "machine-learning", "ai-tools"], "entities": ["ONNX Runtime Web", "Next.js", "React"], "alternates": {"html": "https://wpnews.pro/news/show-hn-tiny-onnx-runtime-web-worker-starter", "markdown": "https://wpnews.pro/news/show-hn-tiny-onnx-runtime-web-worker-starter.md", "text": "https://wpnews.pro/news/show-hn-tiny-onnx-runtime-web-worker-starter.txt", "jsonld": "https://wpnews.pro/news/show-hn-tiny-onnx-runtime-web-worker-starter.jsonld"}}