Browser API for Utilizing On-Device Models A proposal for a Web Models API (navigator.models) would let developers request specific open-weight AI models that run on the user's device, with the browser handling downloading, storing and execution and users granting access per site and model. The proposal, published in the webmachinelearning proposals repository, would support text generation, streaming, embeddings and sessions, and would fail a request for an unsupported model rather than substituting another; an early implementation exists in a Chromium-based browser. Chrome team member Tom pushed back, arguing the Prompt API's value comes from the browser choosing a general-purpose model shared across origins, and pointed to Cross-Origin Storage plus a Prompt API polyfill as the alternative, which the proposal's author called inadequate because sharing model files is not the same as sharing loaded models and managing execution across sites. Repository navigation Description Proposal name Web Models API Short description AI models can run in the browser, but there's no standard way for developers to choose which model to use. They either have to manage the models themselves or rely on whatever the browser provides. This is a problem because an app built for one model may not work with another. The Web Models API navigator.models lets developers request specific open-weight models that run on the user's device. Example use cases 1. Document processing: A user uploads an expense report. The app uses one model to read the document and another to extract structured data. Nothing needs to leave the device. 2. Searching notes: A user searches their notes using natural language. The app uses a local embedding model to find relevant documents, even when offline. 3. Writing assistance: A user asks an app to rewrite an email. The app uses a model the developer has tested and streams the response into the editor. No external API or per-token costs. A rough idea or two about implementation The API would be exposed through navigator.models, supporting text generation, streaming, embeddings, and sessions. Developers request models by ID, with each ID tied to a specific release. If a model isn't supported, the request fails rather than substituting another. The browser handles downloading, storing, and running models, sharing resources across sites when possible. Users grant access per site and model, similar to camera permissions, while each site's data remains isolated. An early implementation exists in a Chromium-based browser. Full explainer and API reference: https://github.com/kenzic/web-models-api-explainer https://github.com/kenzic/web-models-api-explainer Activity 1. Tom from the Chrome team here. For the Prompt API, part of the value-add is really that the browser chooses a general purpose model that it deems suitable for the generally brought forward use cases that all implementing browsers would benchmark their chosen model against and then share it with all origins once it's downloaded. Letting developers specify what model they want would work against this objective. At the same time, we do reckon that developers have a desire to control the model for specific use cases. We're proposing Cross-Origin Storage https://github.com/WICG/cross-origin-storage/ COS for these situations. Together with a Prompt API polyfill https://developer.chrome.com/docs/ai/prompt-api-polyfill that integrates with COS, you would effectively get the Prompt API surface for developers to code against with the cross-origin sharing benefits of the model the developer chooses. 2. Thanks Tom for the quick response. I don't think COS and a polyfill are an adequate substitute here. Choice for developers shouldn't mean giving up browser-managed inference. COS addresses shared storage. The polyfill provides local inference through Transformers.js, but each app still manages its own inference runtime. Sharing model files isn't the same as sharing loaded models and managing execution across sites. A general-purpose benchmark doesn't establish that a model works well for a specific use case. Explicit model selection makes that dependency transparent and gives developers a consistent model to evaluate and test across browsers that support it. The Prompt API can keep its browser-selected default. The Web Models API would complement it by letting apps request a supported model while the browser manages execution and shares resources across sites. Model choice shouldn't require moving inference management back into the application. 3. I agree that running in client-side code will likely be less efficient than running natively, but the earlier comment https://github.com/webmachinelearning/proposals/issues/22 issuecomment-6591148438:~:text=For%20the%20Prompt,against%20this%20objective. still applies. One developer might want a coding-optimized model, another a model that is very creative, someone else a model that's good for describing images, but not the latest version but a version from 2025 from before they enforced refusing CAPTCHA resolving attempts,… You get the point. Some more discussion in webmachinelearning/prompt-api 8 https://github.com/webmachinelearning/prompt-api/issues/8 . 4. Model sprawl is a valid concern although it's still a problem with the polyfill or just using a library like transformers.js. In the CAPTCHA example, that same model would be accessible via a model running on WebGPU. The Web Models API doesn't require every model or historical version be supported. Instead, a registry maintains a collection of supported models which can be requested by the developer and run by the browser. Admittedly, that has tradeoffs, but so does limiting developers to a general-purpose model they can't choose, and is different per browser. I hear the concerns in webmachinelearning/prompt-api 8 https://github.com/webmachinelearning/prompt-api/issues/8 . The proposal prevents sites from enumerating installed models without permission and ties model IDs to specific releases. Remaining questions around fingerprinting, interoperability, and retirement are problems that can be worked through.To be clear, I'm not proposing the Web Models API supersede the Prompt API. The Web Models API is designed to give developers control over which models they use and users control over access. However, I don't think the Prompt API's design goals should define the limits of the web platform. Both approaches have a place, and I think they're both worth developing. What would you consider an acceptable way to balance developer choice with resource sharing in a separate native API? 5. 5. Model sprawl is a valid concern although it's still a problem with the polyfill or just using a library like transformers.js. In the CAPTCHA example, that same model would be accessible via a model running on WebGPU. Correct, the difference maybe being that this is not a new problem for the polyfill approach, whereas with the built-in approach it'd be a new worry: the current worry is does the browser have the API at all, and does it have the default model, but this would introduce a branch point because there would now be a developer preferred model. The Web Models API doesn't require every model or historical version be supported. Instead, a registry maintains a collection of supported models which can be requested by the developer and run by the browser. Admittedly, that has tradeoffs, but so does limiting developers to a general-purpose model they can't choose, and is different per browser. That's right. I guess the tradeoff would be where to stop with browser-supplied models. Model versions say Gemma 3 vs. Gemma 4 , various quantizations maybe, different model sizes like 2B, 4B, 12B,… , expert models,… I hear the concerns in webmachinelearning/prompt-api 8 https://github.com/webmachinelearning/prompt-api/issues/8 . The proposal prevents sites from enumerating installed models without permission and ties model IDs to specific releases. Remaining questions around fingerprinting, interoperability, and retirement are problems that can be worked through. None of these are easy. And to be fair, even with the existing default model, interoperability is a major challenge if you look at the vendors' opinions on the existing Prompt API as is. To be clear, I'm not proposing the Web Models API supersede the Prompt API. The Web Models API is designed to give developers control over which models they use and users control over access. However, I don't think the Prompt API's design goals should define the limits of the web platform. Both approaches have a place, and I think they're both worth developing. I fully understand this. What would you consider an acceptable way to balance developer choice with resource sharing in a separate native API? I guess step one would be to do developer research to understand what model they wish would be supported natively by the browser. Plus all the other challenges you listed yourself "fingerprinting, interoperability, and retirement" . There may be more. 6. Hi, Nico here, one of the maintainers of Transformers.js the runtime the Prompt API polyfill uses . I share the goal of giving developers more control over which model they run, and I agree that sharing model files alone doesn’t avoid every app shipping its own inference runtime. But I’m skeptical about the browser managing a set of open-weight models. My main question is: where do we draw the line? - Which models get accepted? Only a handful verified by the labs that trained them? What about smaller labs, or a new model released next week? - Who maintains that list, and how? For this to be interoperable, the same list would have to work in Chrome, Firefox and Safari. Who owns it, how are models added or deprecated, and how fast can it move compared to how fast models are released? - What about fine-tunes? If the list isn’t fixed and anyone can pin their own fine-tuned version, there’s effectively no list. Browsers would end up downloading, storing, updating and evicting an open-ended set of multi-GB models, which is a huge responsibility to put on the browser. As I see it, either the list is small and curated, which recreates the gatekeeping problem with more steps, or it’s open, with all the problems listed above. On performance: I’m not convinced that browser-managed inference is inherently faster than inference at the application level. Well-optimized custom WebGPU engines can be as fast as, and in many cases faster than, what I’ve seen from the Prompt API implementations so far. They can also be tuned to a specific model architecture and adopt new optimizations without waiting for browser release cycles. I think the current split is the right way forward. On one side, the Prompt API gives a browser-managed, general-purpose model for the common use cases. On the other, the platform exposes the low-level primitives WebGPU, WebNN, Wasm, and Cross-Origin Storage for sharing model files so developers who need a specific or custom model can run their own inference. I’d rather see effort go into making those primitives as fast and as easy to share as possible. That keeps model choice fully open without making browsers responsible for curating and hosting models. Reacted by Thomas Steiner and Chris McKenzie 7. Hey Nico, thanks for the great feedback. Admittedly the registry governance needs to be tightened up, but I do have some thoughts on a path forward. I'll get to those, but first I want to touch upon the split you laid out, as it creates its own set of issues. We already have Prompt, Summarizer, Writer, Rewriter and Translator, with separate proposals for Semantic Embedder https://github.com/explainers-by-googlers/semantic-embedder-api , Vector Store https://github.com/explainers-by-googlers/vector-store-api and Decisions https://github.com/explainers-by-googlers/decisions-api . Where does this stop? Do we keep adding these APIs for each new capability and direct developers to libraries for everything else?Web Models takes a different approach which I probably haven't been clear enough at delineating . The approach is to have an API which supports running models, across different classes, on-device. Generation and embeddings are already defined currently working on extending to support decision models . New inference operations can extend the same contract without introducing another model-management interface for each task. On the registry, I don't agree that our choices are a handful of gatekept models or an unrestricted list. The Public Hash List https://github.com/WICG/cross-origin-storage/blob/main/public-hash-list/phl-explainer.md proposed alongside COS already lays out a cross-vendor approach that I think Web Models can build on.Similar to the PHL proposal, the registry would be selective with predefined acceptance criteria. Maybe 10K downloads as PHL recommends, but I'm not sure on that just yet. However there is precedent and a path forward for this problem. Ultimately, I don't think I alone should set that criteria so it would need to be fleshed out. However, i don't think custom fine-tunes would make sense via the Web Models API, and instead is better served by transformers.js. On Performance, I agree that native isn't necessarily faster, and in the benchmarks I've seen https://github.com/swissspidy/web-ai-evals it's faster than Prompt API, however, I don't believe the WebGPU instance used nano. My concern is what happens across applications. COS shares stored files but it doesn't share loaded models or inference sessions. As more sites run on-device inference, having each manage its own engine leaves cross-site resource sharing unresolved. Web Models proposes putting that coordination in the browser, with user-controlled model access.I think Transformers.js and Web Models have complementary roles here. I'm working on a navigator.models prototype built on Transformers.js to test some of these ideas although admittedly the boundary of the polyfill will be COS and not be able to handle the cross-site resource management . What would you need to see demonstrated to make a browser-managed path for developer-selected models worth pursuing? 8. @tomayac https://github.com/tomayac I think your work on the Public Hash List https://github.com/WICG/cross-origin-storage/blob/main/public-hash-list/phl-explainer.md is a good starting point. Admittedly, with a model registry we'll need additional rules around compatibility and retirement needs, but it gives us a good starting point. I covered this in more detail in my reply to Nico above.Re: another branch point. The Prompt API already requires developers to check if the browser's model is available. Web Models does add a specific model request, which the user can approve or deny, but the trade off is that the developer knows the release to test against. I think the Prompt API's approach shifts the branching to a different level of the application, as Mozilla mentioned https://github.com/mozilla/standards-positions/issues/1213 issuecomment-4347988313 . Developers end up tuning their applications to whichever model they tested against, then face different behavior across browsers without a reliable model ID to test against. I think hiding the model makes this harder to manage.As I mentioned to Nico, Web Models proposes a generalized API for browser-managed inference across model classes. I think that's worth pursuing as we continue introducing separate APIs for embeddings, decisions and other capabilities. I agree developer research is useful. I'd want to understand which specific models and releases developers need for their workloads. I've been discussing this with developers here in the NYC area, but definitely need a more global view on this. Given your PHL work, where do you see that approach becoming impractical when moving from shared storage to browser-managed inference? 9. Thanks for the detailed reply, @kenzic https://github.com/kenzic .On "where does this stop?": we don't actually have those APIs yet. Apart from the ones already shipping in Chrome, they're proposals, and I don't expect all of them to land. That's the point of the standardization process: a vendor brings ideas, and the ecosystem comes together and challenges them. So it stops exactly there, at the standards level. Some will make it, some won't. That isn't a gap a generic API needs to fill.My bigger concern is that running a model is only one part of inference. Every model comes with its own pre- and post-processing, and a browser-managed API would have to support that for every model it accepts, identically across all engines. Otherwise a pinned model ID doesn't give you the same behavior everywhere. Text-to-speech in Transformers.js is a good example. VITS is a single forward pass. SpeechT5 needs a separate vocoder model. Supertonic runs a duration predictor and a denoising loop. We're currently adding Phonon huggingface/transformers.js 1825 https://github.com/huggingface/transformers.js/pull/1825 , which brings its own chat template with per-frame voice placeholders, custom sentence splitting, block-wise generation with noise injection, a model-specific stop threshold and a neural audio codec as decoder. That's one task with four completely different pipelines, and the next model will likely bring yet another one. In a library, that's a PR that ships in days. For browsers, it means every engine has to implement and maintain that logic per architecture.That rules out "bring any model". It also means we're back to a curated list of supported models and architectures, which is gatekeeping, just with more steps. Acceptance criteria like the download threshold in the Public Hash List help with deciding which files are worth sharing, but they don't solve the question of who implements and maintains the inference logic for each model. You already said fine-tunes are better served by Transformers.js, and I think the same reasoning applies to new architectures in general. So I still think the better path is a small set of browser-managed task APIs that survive standardization, plus strong low-level primitives WebGPU, WebNN, Wasm, COS for everything else. Cross-site sharing of loaded models is a real problem, and I'd be happy to explore it, but at the primitive level rather than by making browsers responsible for every model's pipeline. 10. What would you need to see demonstrated to make a browser-managed path for developer-selected models worth pursuing? I think a clear split between what the browser is responsible for and what the app or library is responsible for. Wrapping Transformers.js in a simpler, task-specific API doesn't solve that. The browser would still own all the architecture-specific logic. What would convince me is a concrete example. Take text-to-speech, and show how the browser could expose VITS, Supertonic and Phonon behind one simple API without having to implement and maintain each model's pre-processing, generation loop and decoder itself. But I think that's exactly the contradiction. A developer-friendly API needs abstractions, and those abstractions only work if someone implements the model-specific logic behind them. If the browser offers the abstraction, the browser has to maintain that logic for every model it supports. If it doesn't, developers are back to handling it themselves, which is what libraries like Transformers.js already do on top of the existing primitives. If you can find a clean boundary that avoids this, I think that would be a great starting point. 11. Hey @nico-martin https://github.com/nico-martin My bigger concern is that running a model is only one part of inference. I agree that model specific processing needs an owner, but I disagree that browser-managed inference means each browser has to maintain its own implementation. The browser would keep the public API and resource management, while the model-specific processing could live in shared components. A few ideas on how this could be accomplished: - A shared open-source runtime that browsers integrate. Transformers.js with ONNX is one stack worth exploring. It would take work to adapt, but pipelines could be maintained upstream and reused across browsers. Browser teams maintain the integration with the runtime, while pipeline changes are implemented once upstream. LiteRT-LM https://developers.googleblog.com/en/on-device-genai-in-chrome-chromebook-plus-and-pixel-watch-with-litert-lm/ is another example of an open-source runtime supporting different models and applications which could be useful. - Registered model packages borrowing from llamafile's approach https://github.com/mozilla-ai/llamafile that bundle what's needed to run a model.We wouldn't run arbitrary native executables in the browser, but the idea of shipping the model with its execution dependencies is useful. Hugging Face already packages weights, tokenizer configuration and templates together. Your response-template implementation https://github.com/huggingface/transformers.js/pull/1783 is a nice example: model-specific output parsing rules live in tokenizer config.json , and a shared parser interprets them. - Sandboxed pipeline modules for processing that configuration can't handle. A model package could include portable code, like Wasm, for its preprocessing and generation loop. That code would call a restricted inference interface, while the browser manages execution and resources. These are some ideas on what a path forward could look like, and ultimately it could be a mix of these ideas or something else, but there is precedent for how the community is solving similar challenges. Text-to-speech in Transformers.js is a good example. Yes, VITS, Supertonic and Phonon all handle things differently but that doesn't mean each browser needs to implement those differences. A shared runtime or model package could handle that work. The model ID would cover the processing rules, not just the weights, with shared tests to verify compatibility across browsers. On "where does this stop?": we don't actually have those APIs yet. Apart from the ones already shipping in Chrome, they're proposals, and I don't expect all of them to land.As I understand it, your preferred split puts a browser-selected general purpose model on one side and libraries on the other. But we're already seeing separate proposals for embeddings, decisions, and other capabilities. Whether those proposals make it through standardization doesn't change the underlying issues. The community wants built in browser APIs for these tasks so the surface continues to grow. I'd rather standardize a common model layer and let developers build the features. A new registered model that fits the supported execution contract should expand what applications can do without requiring a separate task API. That rules out "bring any model". That's never been the proposal. The registry would be selective as models need to meet defined execution requirements and shared governance and published admission criteria. Yes, custom fine-tunes are better served by libraries, but I wouldn't put new architectures in the same category. Supporting new architectures can open the door to an entire family of models but that doesn't mean we're committing to support every model built on it. strong low-level primitives WebGPU, WebNN, Wasm, COS for everything else I also think strong lower-level sharing primitives could fit into this. Sharing a loaded model across unrelated sites still requires something outside either site to manage it. If that also coordinates inference, we're getting close to a browser-managed service with a lower-level interface. The main difference is where the pipeline runs. If you can find a clean boundary that avoids this, I think that would be a great starting point Where do you see this separation between browser-managed execution and shared pipeline maintenance breaking down? 12. A shared open-source runtime that browsers integrate. Transformers.js with ONNX is one stack worth exploring. It would take work to adapt, but pipelines could be maintained upstream and reused across browsers. Browser teams maintain the integration with the runtime, while pipeline changes are implemented once upstream. 13. Where do you see this separation between browser-managed execution and shared pipeline maintenance breaking down? Well, thats exactly the hard question that a concrete prototype would have to answer. And also the reason why I don't think it's a good idea: Because I don't really see a separation that could be maintained by different browser, offers a useful API for JS developers and actually improves inference compared to what we have with libraries + strong low-level primitives. 14. Thanks and I appreciate the feedback. I still see value in developer-selected models with browser-managed resource sharing. I'll continue to hammer away at this and explore some of these approaches to see where that gets us. I'll share the results and welcome your feedback as the work progresses. Reacted by Nico Martin 15. Following up on the conformance question from the Web Models API thread: the proposal ties model IDs to specific releases, including weight variant, tokenizer, and input/output defaults, but leaves open how a browser would test that it reproduces the computation an ID promises, especially for embeddings.We built a small conformance pack that fills that gap, and we'd like to offer it to this discussion for use or adaptation: - Fixed test inputs — 32 embedding inputs empty, long, and multilingual cases included and 16 generation prompts run under declared deterministic decoding temperature 0, fixed seed . - Tolerances — cosine similarity ≥ 0.999 against reference vectors for embeddings, a norm check that catches unnormalized output the normalization drift is invisible otherwise , and exact or banded matches for deterministic generation outputs. - A registry-entry schema — each entry must name the weights revision and variant, the quantization scheme, the tokenizer revision, runtime defaults, and embedding normalization, so two different computations cannot share one model ID. - A reference checker Python, standard library only that validates a registry entry against the schema and a browser's outputs against the pack. A submission earns an ID whole or not at all — the proposal's fail-rather-than-substitute rule applied to conformance itself. It ships with a passing sample submission and a failing one whose three defects unnormalized vector, wrong vector, wrong deterministic answer are each named by the checker. One honest limit: the pack's two entries are examples. Their reference vectors come from a small reference implementation included in the pack, not from a named production model; adopting the pack for a real model means publishing a real entry, with real revisions and hashes and reference vectors from that model's reference runtime. The format, schema, tolerances, and checker carry over unchanged.Pack: https://github.com/1OF1Research/web-models-conformance-pack https://github.com/1OF1Research/web-models-conformance-pack Happy to adjust the input set, thresholds, or schema shape to what this group would actually run.