# The 7 MB Embedding Model Bringing Semantic Search to the Browser

> Source: <https://sourcefeed.dev/a/the-7-mb-embedding-model-bringing-semantic-search-to-the-browser>
> Published: 2026-07-07 06:03:22+00:00

[AI](https://sourcefeed.dev/c/ai)Article

# The 7 MB Embedding Model Bringing Semantic Search to the Browser

Ternlight proves that client-side vector embeddings are finally practical, bypassing server round-trips and API costs entirely.

[Priya Nair](https://sourcefeed.dev/u/priya_nair)

We have been promised local, browser-native AI for years. But downloading a 2 GB quantized language model over a mobile connection is a hard sell for most web applications. It is slow, eats data caps, and hogs device memory. While running a full generative model locally is often impractical, generating vector embeddings is a completely different story.

Enter [Ternlight Demo](https://ternlight-demo.vercel.app/), an embedding model that runs entirely client-side. At just 7 MB, the model is smaller than many modern hero images, yet it opens the door to fully local semantic search, client-side retrieval-augmented generation (RAG) chunking, and real-time similarity features. By combining ternary quantization with a lightweight WebAssembly engine, it bypasses the latency, cost, and privacy concerns of traditional API-based embedding pipelines.

## The Math Behind the Shrinkage: Ternary Weights

How does an embedding model shrink to 7 MB without becoming completely useless? The secret lies in ternary quantization, specifically using BitLinear layers.

Traditional deep learning models represent weights using 16-bit or 32-bit floating-point numbers. Ternary quantization restricts these weights to just three possible values: -1, 0, and 1. This architecture requires only 1.58 bits per parameter. Beyond the massive reduction in storage size, ternary weights fundamentally alter the compute profile.

Instead of expensive floating-point multiplications, the processor can perform matrix multiplication using simple additions and subtractions. For a browser running on a user's CPU, this is a massive win. It bypasses the need for heavy GPU acceleration for basic text representation tasks.

## The Browser Runtime Stack: WASM, SIMD, and Caching

To run this model efficiently, Ternlight relies on `@ternlight/mini`

, a lightweight engine compiled to [WebAssembly](https://webassembly.org). While WebGPU is the gold standard for heavy parallel workloads, WebAssembly remains the universal fallback for CPU-bound execution.

Modern browsers support WebAssembly SIMD (Single Instruction, Multiple Data). If `WebAssembly.validate()`

returns true for SIMD, the engine can run quantized operations on the CPU roughly 2 to 4 times faster than standard FP32 WASM. This makes the latency of generating a single embedding negligible on modern devices.

To make this practical for production, developers must implement an aggressive caching strategy. You do not want to fetch the 7 MB model payload on every page load. The standard pattern is to check IndexedDB or the Cache API first, download the model once, and cache it for subsequent sessions.

```
flowchart TD
    subgraph Traditional API-Based Search
        Client1[Client Browser] -->|1. Send Query| Server[Backend Server]
        Server -->|2. Get Embedding| OpenAI[Embedding API]
        OpenAI -->|3. Return Vector| Server
        Server -->|4. Query DB| VectorDB[(Vector Database)]
        VectorDB -->|5. Return Results| Server
        Server -->|6. Send Results| Client1
    end
    subgraph Local-First Search
        Client2[Client Browser] -->|1. Load Cached Model| LocalEngine[Local WASM Engine]
        Client2 -->|2. Generate Embedding| LocalEngine
        LocalEngine -->|3. Local Vector Search| LocalIndex[Local Index / IndexedDB]
    end
```

## Developer Angle: Architecture and Implementation

To keep the user interface responsive while generating embeddings, you should never run inference on the main browser thread. Instead, offload the execution to a WebWorker. This architecture, championed by projects like [Mozilla AI](https://mozilla.ai), ensures that the main thread remains free to handle user interactions.

Here is a practical implementation of how you can set up a WebWorker to load the Ternlight model, cache it in IndexedDB, and handle embedding queries.

``` js
// worker.js
import { TernlightEngine } from '@ternlight/mini';

let engine = null;

async function getModelBuffer() {
  const cacheName = 'ternlight-model-cache';
  const modelUrl = '/models/ternlight-mini.wasm';
  const cache = await caches.open(cacheName);
  
  let response = await cache.match(modelUrl);
  if (!response) {
    // Fetch and cache on first load
    await cache.add(modelUrl);
    response = await cache.match(modelUrl);
  }
  
  return await response.arrayBuffer();
}

self.onmessage = async (event) => {
  const { type, payload } = event.data;

  if (type === 'INIT') {
    try {
      const buffer = await getModelBuffer();
      engine = new TernlightEngine(buffer);
      self.postMessage({ type: 'READY' });
    } catch (err) {
      self.postMessage({ type: 'ERROR', error: err.message });
    }
  }

  if (type === 'EMBED') {
    if (!engine) {
      self.postMessage({ type: 'ERROR', error: 'Engine not initialized' });
      return;
    }
    
    const startTime = performance.now();
    const vector = engine.embed(payload.text);
    const latency = performance.now() - startTime;
    
    self.postMessage({
      type: 'EMBED_COMPLETE',
      payload: { vector, latency }
    });
  }
};
```

In your main application code, you instantiate the worker and communicate via standard message passing:

``` js
// app.js
const worker = new Worker(new URL('./worker.js', import.meta.url));

worker.postMessage({ type: 'INIT' });

worker.onmessage = (event) => {
  const { type, payload } = event.data;
  if (type === 'READY') {
    console.log('Ternlight is ready for local inference.');
    worker.postMessage({
      type: 'EMBED',
      payload: { text: 'How do I share state across React components?' }
    });
  }
  if (type === 'EMBED_COMPLETE') {
    console.log(`Generated embedding in ${payload.latency.toFixed(2)}ms:`, payload.vector);
  }
};
```

## The Real-World Trade-offs

Before ripping out your backend pgvector setup, you need to consider the constraints of a 7 MB model.

First, the context window and output dimensionality are naturally limited compared to cloud-hosted models like OpenAI's `text-embedding-3-small`

or Cohere's v3 models. Ternlight is designed for short-to-medium text chunks, such as documentation search, UI autocomplete, or local browser history indexing. If you are trying to embed entire PDF books at once, a 7 MB model will struggle with representation quality.

Second, while 7 MB is incredibly small for an AI model, it is still a non-trivial payload for users on slow mobile connections. You should lazy-load the engine only when the user interacts with a feature that requires semantic search, rather than blocking the initial page bundle.

For documentation sites, offline-first applications, and privacy-sensitive tools where data cannot leave the user's device, Ternlight represents a massive step forward. It proves that we do not always need massive GPU clusters to build intelligent search features.

## Sources & further reading

-
[Ternlight – 7 MB embedding model that runs in browser (WASM)](https://ternlight-demo.vercel.app/)— ternlight-demo.vercel.app -
[Linux Report | Latest Linux News](https://linuxreport.net/)— linuxreport.net -
[3W for In-Browser AI: WebLLM + WASM + WebWorkers](https://blog.mozilla.ai/3w-for-in-browser-ai-webllm-wasm-webworkers/)— blog.mozilla.ai -
[Run AI Models in the Browser with WebGPU & WASM](https://maddevs.io/writeups/running-ai-models-locally-in-the-browser/)— maddevs.io

[Priya Nair](https://sourcefeed.dev/u/priya_nair)· AI & Developer Experience Writer

Priya covers AI frameworks, developer productivity tooling, and the startup ecosystem across South and Southeast Asia, bringing a researcher's rigour and a practitioner's empathy to every story. She is deeply sceptical of benchmarks and asks hard questions so her readers don't have to.

## Discussion 0

No comments yet

Be the first to weigh in.