{"slug": "building-a-browser-based-receipt-scanner-with-litert-js", "title": "Building a browser-based receipt scanner with LiteRT.js", "summary": "Google's LiteRT.js, a browser runtime for on-device AI inference using WebAssembly and WebGPU, enables developers to run standard .tflite models directly in the browser, as demonstrated in a tutorial for building an OCR receipt scanner that processes data locally without external APIs. The tutorial, published on LogRocket, shows how to use LiteRT.js with React, TensorFlow.js, and LiteRT-LM to preprocess receipt photos, recognize text, and structure results on-device.", "body_md": "\n\n```\nAdvisory boards aren’t only for executives. Join the LogRocket Content Advisory Board today →\n```\n\nGoogle recently launched LiteRT.js, a browser runtime for running on-device AI inference with WebAssembly and WebGPU.\n\nLiteRT.js brings Google’s LiteRT runtime, formerly TensorFlow Lite, to the web. Instead of requiring a JavaScript-specific model format, it can run standard `.tflite`\n\nmodels directly in the browser while taking advantage of modern browser hardware acceleration.\n\nIn this tutorial, we’ll look at how LiteRT.js works and build an end-to-end optical character recognition (OCR) receipt scanner. The application will preprocess receipt photos, detect and recognize text locally, reconstruct the document layout, and pass the extracted text to an on-device Gemma model through LiteRT-LM to structure the result.\n\nThe full pipeline runs locally in the browser, so receipt data does not need to be sent to an external inference API.\n\nYou’ll need:\n\nBefore we build the scanner, let’s look at what LiteRT.js changes about running machine learning models in the browser.\n\nTensorFlow Lite was originally designed for mobile and embedded systems, while TensorFlow.js was designed specifically for the web.\n\nAs browsers gained capabilities such as WebAssembly SIMD and WebGPU, however, the gap between native and browser-based inference narrowed. Google subsequently rebranded TensorFlow Lite as LiteRT as part of its broader AI Edge tooling.\n\nLiteRT acts as both a model runtime and part of a broader conversion pipeline. Models originating in frameworks such as TensorFlow, PyTorch, and JAX can ultimately be deployed in the `.tflite`\n\nformat, while LiteRT.js brings that runtime to the browser.\n\nFor web applications, the important distinction is that LiteRT.js can execute `.tflite`\n\nmodels using modern browser compute APIs rather than requiring models to target a JavaScript-specific execution environment.\n\nLiteRT.js can target several execution paths:\n\n| Backend | Role | Best suited for |\n|---|---|---|\n| WebAssembly + XNNPACK | CPU execution and fallback | Broad compatibility and CPU inference |\n| WebGPU | GPU-accelerated compute | Parallel workloads such as neural network inference |\n| WebNN | Emerging hardware abstraction | Direct access to available ML accelerators and NPUs |\n\nWebAssembly provides the browser-side execution environment, while XNNPACK supplies optimized neural network operators.\n\n```\nOver 200k developers use LogRocket to create better digital experiences\nLearn more →\n```\n\nWith browser features such as SIMD and multithreading, this gives LiteRT.js a much faster CPU path than implementing the same numerical operations directly in JavaScript.\n\nWebGPU gives browser applications access to general-purpose GPU compute. LiteRT.js can use that capability to execute highly parallel operations such as matrix multiplication on the user’s GPU.\n\nThis avoids many of the constraints associated with treating WebGL, which was designed primarily for graphics, as a general-purpose compute API.\n\nWebNN is an emerging browser API for neural network acceleration. Where supported, it is intended to provide access to the device’s available ML hardware, including GPUs and NPUs.\n\nLet’s initialize the React application and install the dependencies.\n\nThe application has two main stages:\n\nIf you haven’t already created the project, scaffold a React and TypeScript app with Vite:\n\n```\nnpm create vite@latest document-scanner -- --template react-ts\n```\n\nThen install LiteRT.js:\n\n```\nnpm install @litertjs/core\n```\n\nWe’ll use `@litertjs/tfjs-interop`\n\nto pass tensors between TensorFlow.js and LiteRT.js:\n\n```\nnpm install @litertjs/tfjs-interop\n```\n\nInstall TensorFlow.js:\n\n```\nnpm install @tensorflow/tfjs\n```\n\nThen add its WebGPU backend:\n\n```\nnpm install @tensorflow/tfjs-backend-webgpu\n```\n\nFinally, install LiteRT-LM for the on-device language model:\n\n```\nnpm install --save @litert-lm/core\n```\n\nThe resulting stack looks like this:\n\n| Package | Purpose |\n|---|---|\n`@litertjs/core` |\nLoads and executes `.tflite` models |\n`@litertjs/tfjs-interop` |\nShares tensors between TensorFlow.js and LiteRT.js |\n`@tensorflow/tfjs` |\nTensor manipulation and supporting image operations |\n`@tensorflow/tfjs-backend-webgpu` |\nWebGPU backend for TensorFlow.js |\n`@litert-lm/core` |\nRuns the on-device language model |\n\nThe OCR pipeline needs three files:\n\nYou can find the OCR models, dictionary, and complete project in the [GitHub repository](https://github.com/emmanuelhashy/document-scanner).\n\nThe Gemma model is available from [Hugging Face](https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm).\n\nPlace the required model files inside the project’s `public/models/`\n\ndirectory.\n\nThe code used throughout this tutorial lives under `src/`\n\n, organized into three main directories:\n\n```\nsrc/\n├── litert/\n│   ├── runtime.ts\n│   └── models.ts\n├── ocr/\n│   ├── preprocess.ts\n│   ├── detect.ts\n│   ├── ctc.ts\n│   └── layout.ts\n└── llm/\n    ├── engine.ts\n    └── structureWithLlm.ts\n```\n\nThis separation keeps model initialization, OCR processing, and LLM inference independent from one another.\n\n```\nMore great articles from LogRocket:\n\nDon't miss a moment with The Replay, a curated newsletter from LogRocket\nLearn how LogRocket's Galileo AI watches sessions for you and proactively surfaces the highest-impact things you should work on\n\nUse React's useEffect to optimize your application's performance\nSwitch between multiple versions of Node\nDiscover  how to use the React children prop with TypeScript\nExplore creating a custom mouse cursor with CSS\nAdvisory boards aren’t just for executives. Join LogRocket’s Content Advisory Board. You’ll help inform the type of content we create and get access to exclusive meetups, social accreditation, and swag\n```\n\nWe’ll start with the ML runtime itself.\n\nTwo pieces need to be initialized before inference can run:\n\nInitializing a machine learning backend is relatively expensive. React components can mount, unmount, and re-render frequently, so tying runtime initialization directly to component lifecycle can create duplicate GPU contexts and unnecessary memory pressure.\n\nInstead, we’ll cache initialization at the module level with a shared `Promise`\n\n.\n\nCreate `runtime.ts`\n\n:\n\n``` js\nimport {\n  loadLiteRt,\n  getWebGpuDevice,\n  isWebGPUSupported\n} from '@litertjs/core';\nimport * as tf from '@tensorflow/tfjs';\nimport { WebGPUBackend } from '@tensorflow/tfjs-backend-webgpu';\nimport { WASM_PATH } from '../ocr/config';\n\nexport interface RuntimeInfo {\n  webgpu: boolean;\n  tfjsBackend: string;\n}\n\nlet runtimePromise: Promise<RuntimeInfo> | null = null;\n\nexport function initRuntime(): Promise<RuntimeInfo> {\n  if (!runtimePromise) {\n    runtimePromise = doInit().catch((err) => {\n      runtimePromise = null;\n      throw err;\n    });\n  }\n\n  return runtimePromise;\n}\n\nasync function doInit(): Promise<RuntimeInfo> {\n  if (isWebGPUSupported()) {\n    await tf.setBackend('webgpu');\n    await tf.ready();\n    await loadLiteRt(WASM_PATH);\n\n    const device = getWebGpuDevice();\n\n    if (device) {\n      tf.removeBackend('webgpu');\n      tf.registerBackend(\n        'webgpu',\n        () => new WebGPUBackend(device, device.adapterInfo)\n      );\n\n      await tf.setBackend('webgpu');\n      await tf.ready();\n\n      return {\n        webgpu: true,\n        tfjsBackend: tf.getBackend()\n      };\n    }\n  }\n\n  await loadLiteRt(WASM_PATH);\n  await tf.setBackend('cpu');\n  await tf.ready();\n\n  return {\n    webgpu: false,\n    tfjsBackend: tf.getBackend()\n  };\n}\n```\n\n`runtimePromise`\n\nensures that multiple callers share the same initialization work. If initialization fails, the promise is reset so a later request can try again.\n\nWhen WebGPU is available, the application initializes the TensorFlow.js WebGPU backend and shares the GPU device with LiteRT.js. Otherwise, it falls back to CPU execution.\n\nOnce the runtime is ready, we can compile the `.tflite`\n\nmodel graphs.\n\nCompilation prepares the model for a particular execution backend. Here, we’ll try the preferred accelerated backend first and fall back when compilation fails.\n\nCreate `models.ts`\n\n:\n\n``` python\nimport { loadAndCompile } from '@litertjs/core';\nimport type {\n  CompiledModel,\n  TensorDetails\n} from '@litertjs/core';\n\nasync function compileWithFallback(\n  url: string,\n  order: readonly Backend[]\n): Promise<LoadedModel> {\n  let lastErr: unknown;\n\n  for (const accelerator of order) {\n    try {\n      const model = await loadAndCompile(url, { accelerator });\n\n      const toSpec = (d: TensorDetails) => ({\n        name: d.name,\n        dtype: d.dtype,\n        shape: Array.from(d.shape)\n      });\n\n      return {\n        model,\n        backend: accelerator,\n        inputs: model.getInputDetails().map(toSpec),\n        outputs: model.getOutputDetails().map(toSpec)\n      };\n    } catch (err) {\n      lastErr = err;\n    }\n  }\n\n  throw new Error(`Failed to compile ${url}: ${String(lastErr)}`);\n}\n```\n\nThe full `models.ts`\n\nimplementation loads the detector, recognizer, and dictionary in parallel with `Promise.all`\n\n.\n\nBefore sending an image through the OCR models, we preprocess it to make recognition more reliable.\n\nThe pipeline:\n\nThe relevant part of `preprocess.ts`\n\nlooks like this:\n\n```\nexport function preprocess(\n  source: CanvasImageSource,\n  srcW: number,\n  srcH: number\n): Preprocessed {\n  const scale = downscaleFactor(srcW, srcH);\n  const w = Math.round(srcW * scale);\n  const h = Math.round(srcH * scale);\n\n  const img = toImageData(source, w, h);\n  const luma = grayscale(img);\n  contrastStretch(img, luma);\n\n  return {\n    image: img,\n    scale\n  };\n}\n```\n\n`downscaleFactor()`\n\nlimits the longest edge to 1600px, preventing unnecessarily large images from increasing inference cost.\n\nGrayscale conversion reduces the image to luminance information, while contrast stretching increases the separation between text and its background.\n\nThe OCR pipeline has three stages:\n\nThe detection model finds text bounding boxes.\n\nWhen WebGPU is available, we want to avoid repeatedly copying tensors between GPU and CPU memory. `@litertjs/tfjs-interop`\n\nprovides `runWithTfjsTensors`\n\n, which lets TensorFlow.js tensors pass directly into the LiteRT.js execution path.\n\nThe relevant code in `detect.ts`\n\nlooks like this:\n\n``` js\nimport * as tf from '@tensorflow/tfjs';\nimport { runWithTfjsTensors } from '@litertjs/tfjs-interop';\n\nexport async function detect(\n  det: LoadedModel,\n  image: ImageData\n): Promise<{ boxes: Box[] }> {\n  const inLayout = {\n    h: det.inputs[0].shape[2],\n    w: det.inputs[0].shape[3],\n    layout: 'nchw' as const\n  };\n\n  const input = buildInput(\n    image,\n    inLayout.w,\n    inLayout.h,\n    inLayout.layout\n  );\n\n  const outputs = await runWithTfjsTensors(det.model, [input]);\n  input.dispose();\n\n  const [region, affinity] = await Promise.all([\n    outputs[0].data(),\n    outputs[1].data()\n  ]);\n\n  tf.dispose(outputs);\n\n  return {\n    boxes: decodeCraftHeatmaps(\n      region,\n      affinity,\n      inLayout.w,\n      inLayout.h\n    )\n  };\n}\n```\n\n`buildInput()`\n\nnormalizes image values into the range expected by the model.\n\nAfter inference, the model’s heatmaps are decoded into bounding boxes using connected-component grouping.\n\nOnce we’ve located the text regions, each region is cropped and passed to the recognition model.\n\nThe recognizer returns a probability distribution over character classes at each timestep. We then decode that output using Connectionist Temporal Classification (CTC).\n\nThe decoder in `ctc.ts`\n\nlooks like this:\n\n```\nexport function ctcGreedyDecode(\n  logits: Float32Array,\n  T: number,\n  numClasses: number,\n  chars: string[]\n) {\n  const path = new Int32Array(T);\n  const probs = new Float32Array(T);\n\n  for (let t = 0; t < T; t++) {\n    path[t] = getArgmax(\n      logits,\n      t * numClasses,\n      numClasses\n    );\n\n    probs[t] = getSoftmaxProbability(\n      logits,\n      t * numClasses,\n      numClasses\n    );\n  }\n\n  let out = '';\n  let prev = -1;\n  const blank = 0;\n\n  for (let t = 0; t < T; t++) {\n    const cls = path[t];\n\n    if (cls !== prev && cls !== blank) {\n      out += chars[cls] ?? '';\n    }\n\n    prev = cls;\n  }\n\n  return {\n    text: out,\n    confidence: calculateAvgConfidence(probs)\n  };\n}\n```\n\nThe decoder takes the most likely character at each timestep, collapses repeated classes, and removes the CTC blank token.\n\nRaw OCR output is not enough for receipts. We also need to preserve relationships such as an item name and its price appearing on the same row.\n\nThe application reconstructs that layout using the coordinates of each detected word.\n\nIn `layout.ts`\n\n:\n\n```\nexport function reconstructLines(\n  words: Word[],\n  rowTolerance = 0.6\n): Line[] {\n  const byY = [...words].sort(\n    (a, b) =>\n      (a.box.y + a.box.h / 2) -\n      (b.box.y + b.box.h / 2)\n  );\n\n  const rows: Row[] = [];\n\n  for (const word of byY) {\n    const matchedRow = findMatchingRow(\n      rows,\n      word,\n      rowTolerance\n    );\n\n    if (matchedRow) {\n      matchedRow.words.push(word);\n    } else {\n      rows.push({\n        words: [word],\n        centerSum: word.box.y + word.box.h / 2\n      });\n    }\n  }\n\n  return rows\n    .map(formatAndInsertSpaces)\n    .sort((a, b) => a.yCenter - b.yCenter);\n}\n```\n\nThe algorithm first groups words whose vertical positions overlap within a tolerance. It then sorts words horizontally and inserts spacing based on their physical distance from one another.\n\nThat gives the LLM a more useful representation of the receipt than a flat list of recognized strings.\n\nOCR gives us text. The next step is turning that text into structured application data.\n\nFor a receipt, that might mean:\n\nWe’ll use an on-device Gemma model through LiteRT-LM to convert the reconstructed OCR output into JSON. If you’re interested in other approaches to [building multi-turn AI agents](https://blog.logrocket.com/building-multi-turn-ai-agents-genkits-agents-api/), Genkit’s Agents API is worth exploring as a complementary option.\n\nLLM runtimes and model resources are relatively large, so loading them as part of the initial application bundle would increase startup cost even for users who never use the feature.\n\nInstead, we’ll dynamically import LiteRT-LM only when the user enables the **Structure with LLM** option.\n\nThe relevant part of `engine.ts`\n\n:\n\n``` python\nimport type { Engine } from '@litert-lm/core';\nimport {\n  LLM_MODEL_PATH,\n  LLM_MAX_TOKENS,\n  LLM_WASM_PATH\n} from '../ocr/config';\n\nlet status: LlmStatus = 'unavailable';\nlet enginePromise: Promise<Engine | null> | null = null;\n\nexport function initLlmEngine(): Promise<Engine | null> {\n  if (!enginePromise) {\n    enginePromise = doInit().catch(() => {\n      enginePromise = null;\n      return null;\n    });\n  }\n\n  return enginePromise;\n}\n\nasync function doInit(): Promise<Engine | null> {\n  const mod = await import('@litert-lm/core');\n\n  await mod.getOrLoadGlobalLiteRtLm(LLM_WASM_PATH);\n\n  return await mod.Engine.create({\n    model: LLM_MODEL_PATH,\n    mainExecutorSettings: {\n      maxNumTokens: LLM_MAX_TOKENS\n    }\n  });\n}\n```\n\nThe module-level promise serves the same role as it did for the main LiteRT.js runtime: only one initialization runs at a time.\n\nThe dynamic `import()`\n\nalso gives the bundler an opportunity to split the LLM runtime into a separate chunk rather than including it in the application’s initial JavaScript bundle. This kind of [lazy evaluation](https://blog.logrocket.com/deep-dive-react-fiber/) is a common pattern in React applications for managing expensive resources.\n\nNext, we provide a system prompt that defines the expected output format and target JSON schema.\n\n`structureWithLlm.ts`\n\nhandles the request:\n\n``` js\nimport { getLlmEngine } from './engine';\nimport {\n  SYSTEM_PROMPT,\n  buildUserPrompt\n} from './prompt';\n\nexport async function structureWithLlm(\n  lines: Line[]\n): Promise<ReceiptData> {\n  const engine = await getLlmEngine();\n\n  if (!engine) {\n    return structureFallback(lines);\n  }\n\n  try {\n    const convo = await engine.createConversation({\n      preface: {\n        messages: [\n          {\n            role: 'system',\n            content: SYSTEM_PROMPT\n          }\n        ]\n      }\n    });\n\n    const prompt = buildUserPrompt(lines);\n    const raw = await convo.sendMessage(prompt);\n\n    await convo.delete();\n\n    const jsonText = extractJson(raw.content);\n\n    return parseAndMapOcrConfidence(\n      jsonText,\n      lines\n    );\n  } catch (err) {\n    return structureFallback(lines);\n  }\n}\n```\n\nIf the LLM cannot initialize or fails during inference, the application falls back to a regex-based parser rather than failing the entire scan.\n\nThe result in the browser looks like this:\n\nNote:The recognition model used in this demo achieves less than 50 percent recognition accuracy in the author’s testing. You can replace it with a more accurate OCR model without changing the overall LiteRT.js pipeline.\n\nLiteRT.js is one of several options for running machine learning workloads in the browser.\n\nThe major differences are the model format, execution architecture, and hardware backends each runtime supports.\n\n| Feature | LiteRT.js | TensorFlow.js | ONNX Runtime Web |\n|---|---|---|---|\n| Model format | `.tflite` |\nTensorFlow.js model formats | `.onnx` |\n| CPU execution | Wasm / optimized native kernels | JavaScript / Wasm backends | Wasm |\n| GPU execution | WebGPU | WebGL / WebGPU, depending on backend | WebGPU |\n| Framework interoperability | Models can originate from multiple ML frameworks through conversion | Strongest with TensorFlow ecosystem | Broad ONNX ecosystem |\n| WebNN path | Emerging/support dependent | Support dependent | Support dependent |\n\nThe most important choice is often the model format you already use.\n\nIf you already have `.tflite`\n\nmodels or are targeting Android, mobile, embedded, and web from the same model pipeline, LiteRT.js is particularly attractive because the browser can use the same deployment format.\n\nTensorFlow.js still has a broader JavaScript-native ecosystem and can be useful when tensor manipulation and model execution both live primarily in JavaScript. When evaluating your [package manager](https://blog.logrocket.com/pnpm-vs-npm-which-package-manager-use/) and dependency setup for these kinds of ML projects, it’s worth considering how each tool affects install times and disk usage.\n\nONNX Runtime Web is a strong fit when your model pipeline already targets ONNX or you need compatibility with models originating across several training frameworks.\n\nLiteRT.js gives web developers another practical path to running machine learning models directly on user devices.\n\nIn this tutorial, we built a browser-based receipt-processing pipeline that uses LiteRT.js for OCR inference, reconstructs the spatial layout of the recognized text, and optionally passes that result through an on-device language model with LiteRT-LM.\n\nThe architecture also demonstrates where LiteRT.js fits alongside existing browser tooling. TensorFlow.js still plays a useful role in tensor handling and interoperability here, while LiteRT.js handles `.tflite`\n\nmodel execution. That makes LiteRT.js less of a wholesale TensorFlow.js replacement and more of a new option for applications built around Google’s AI Edge model ecosystem.\n\nRunning inference locally also changes the application’s deployment model. Images and extracted text can remain on the user’s device, the application can continue working without a round trip to an inference API, and developers can avoid standing up a dedicated backend for every model request. For teams evaluating where AI fits into their broader product strategy, [understanding what AI knowledge product managers need](https://blog.logrocket.com/product-management/ai-skills-product-managers-need/) is increasingly relevant as these on-device capabilities mature.\n\nYou can find the complete project on [GitHub](https://github.com/emmanuelhashy/document-scanner).\n\nIf you’re interested in exploring how AI tooling compares more broadly, the [AI dev tool power rankings](https://blog.logrocket.com/ai-dev-tool-power-rankings/) offer a useful overview of the current landscape.\n\n```\nStop guessing about your digital experience with LogRocket\nGet started for free\n```\n\nLearn how to use the TypeScript Compiler API and AST traversal to extract imports and build a file dependency graph CLI.\n\nStop generating AI slop with Claude Code. Discover 5 actionable developer tips to manage context windows, enforce rules with hooks, and improve code quality.\n\nChoosing between skills and MCP tools comes down to auditability versus flexibility. By building the exact same capability twice, this guide reveals when your agent needs a deterministic tool and when it needs an interpretive skill.\n\nLearn how to replace React state, Context, and event handlers with native HTML and CSS features for dark mode, modals, accordions, carousels, and more.\n\nWould you be interested in joining LogRocket's developer community?\n\nJoin LogRocket’s Content Advisory Board. You’ll help inform the type of content we create and get access to exclusive meetups, social accreditation, and swag.", "url": "https://wpnews.pro/news/building-a-browser-based-receipt-scanner-with-litert-js", "canonical_source": "https://blog.logrocket.com/building-browser-based-receipt-scanner-litert-js/", "published_at": "2026-08-31 13:00:32+00:00", "updated_at": "2026-09-02 06:22:34.289504+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "machine-learning", "computer-vision", "artificial-intelligence"], "entities": ["Google", "LiteRT.js", "LiteRT", "TensorFlow Lite", "TensorFlow.js", "WebAssembly", "WebGPU", "LogRocket"], "alternates": {"html": "https://wpnews.pro/news/building-a-browser-based-receipt-scanner-with-litert-js", "markdown": "https://wpnews.pro/news/building-a-browser-based-receipt-scanner-with-litert-js.md", "text": "https://wpnews.pro/news/building-a-browser-based-receipt-scanner-with-litert-js.txt", "jsonld": "https://wpnews.pro/news/building-a-browser-based-receipt-scanner-with-litert-js.jsonld"}}