{"slug": "open-sourced-a-local-wan-2-1-mobile-pipeline-for-on-device-video-generation", "title": "Open-sourced a local WAN 2.1 mobile pipeline for on-device video generation", "summary": "Quartz, a from-scratch Rust inference engine developed by Saient, has been open-sourced, enabling on-device video generation with Wan2.1 T2V (1.3B parameters) on a Samsung Galaxy S24's Vulkan GPU, alongside LLM chat and SDXL image generation, with no cloud round-trip. The engine, previously named tinyq4, supports CPU, CUDA, and Vulkan backends and reads GGUF files directly.", "body_md": "Quartz is the on-device inference engine behind [Saient](https://saient.co.uk) —\na from-scratch Rust runtime (no llama.cpp, no ggml, no Candle, no\nPyTorch/libtorch) that runs LLM chat and image generation entirely on a\nphone or desktop, no cloud round-trip.\n\n**It's the engine that made this possible:** Saient mobile generates\nfull text-to-video clips — [Wan2.1 T2V](https://github.com/Wan-Video/Wan2.1),\n1.3B parameters — running end-to-end, locally, on a stock Samsung Galaxy\nS24's Vulkan GPU. Same phone also runs LLM chat and SDXL image\ngeneration locally through this engine. No server, no API key, nothing\nleaves the device.\n\n(The Wan video model itself runs through a separate, MIT-licensed\ncompanion binary — see [Relationship to the Wan video engine](#relationship-to-the-wan-video-engine)\nbelow for how the two fit together. This repo, Quartz proper, is the\nLLM chat + SDXL image half of the stack.)\n\nThis repo covers Quartz: a minimal, dependency-light Rust inference\nruntime for GGUF-format transformer language models, with a bundled\nStable Diffusion (SD1.5/SDXL) image pipeline. CI-grade targets are a\n4-core desktop CPU, a CUDA GPU, and an Android phone's Vulkan GPU, via\na single ~15k-line, mostly-`unsafe`\n\n-free codebase.\n\nThis project was previously developed under the name **tinyq4**. All crate,\nbinary, and environment-variable names now use `quartz`\n\n/ `QUARTZ_*`\n\n; the\ngit history and any external references to `tinyq4`\n\npredate the rename.\n\nQuartz ships as a single binary (`quartz`\n\n) with two capabilities:\n\n**LLM chat inference**— GGUF model loading, tokenization (native GGUF vocab or a`minijinja`\n\n-templated Jinja2 chat template, matching what the source model ships), forward pass, and an OpenAI-compatible HTTP server for streaming completions.**Image generation (SD1.5 / SDXL)**— a from-scratch CLIP text encoder, UNet, VAE, and scheduler implementation, driven by the same binary via`--generate-sd15`\n\n/`--generate-sdxl`\n\n.\n\nVideo generation (Wan2.1 T2V) is a **separate** binary and a separate\ncodebase — see [Relationship to the Wan video engine](#relationship-to-the-wan-video-engine)\nbelow. Don't conflate the two; they share the Quartz brand but not a\nsource tree.\n\n**CPU**— portable Rust, with a hand-rolled AVX2/FMA dot-product path on x86_64 (auto-dispatched at runtime; falls back to scalar otherwise) and a hand-written NEON dot-product path for Q4_K on aarch64 (Android).**CUDA**— optional (`--features cuda`\n\n), custom GEMV kernels per quantization format (`src/cuda_kernels.cu`\n\n,`src/cuda.rs`\n\n), staged/paged weight loading, and a VRAM-aware KV cache that sizes itself to free VRAM rather than a hard token cap.**Vulkan**— optional (`--features vulkan`\n\n), used for both LLM and SDXL compute; this is the backend the Android build ships with, since phones have no CUDA.\n\nReads GGUF files directly (`src/gguf.rs`\n\n) and dequantizes on the fly:\nQ4_K, Q6_K, Q8_0, Q5_0/Q5_1, Q4_0/Q4_1, and the IQ2/IQ3 k-quant formats\n(grid/sign tables in `src/iq2_tables.rs`\n\n/ `src/iq3_tables.rs`\n\n, generated\nfrom upstream `ggml-quants.c`\n\n/ `ggml-common.h`\n\nconstant tables — the\ntables are data, not linked code).\n\n`quartz --server`\n\nexposes an OpenAI-compatible surface:\n\n```\nGET  /health\nGET  /v1/health\nGET  /v1/models\nPOST /v1/chat/completions   (SSE streaming)\n```\n\nThis is what both the desktop and mobile Saient clients speak — same wire format either side.\n\n```\ncargo build --release                       # CPU only\ncargo build --release --features cuda       # + CUDA GEMV kernels\ncargo build --release --features vulkan     # + Vulkan (SD image gen, or Android)\n```\n\nAndroid (arm64-v8a), used by the Saient mobile app:\n\n```\nscripts/build-android-arm64.sh\n# → target/aarch64-linux-android/release/quartz\n```\n\nRequires `ANDROID_NDK_HOME`\n\n(or a discoverable NDK under\n`$ANDROID_SDK_ROOT/ndk`\n\n). `QUARTZ_ANDROID_API`\n\n(default 24) and\n`QUARTZ_ANDROID_FEATURES`\n\n(default `vulkan`\n\n) are overridable.\n\n| Variable | Effect |\n|---|---|\n`QUARTZ_BIND` |\nServer bind address. Defaults to `127.0.0.1` (loopback only); set to `0.0.0.0` to opt into LAN/network access. |\n`QUARTZ_CTX` |\nUpper-bounds the KV cache context length the host app requests. |\n`QUARTZ_CUDA_ARCH` |\nSASS target arch(es) for the CUDA build (`build.rs` ); CI/shipping builds set `all-major` . |\n`QUARTZ_DENSE_FFN_CPU` |\nForces dense-FFN compute onto CPU (CUDA path), for debugging/comparison. |\n`QUARTZ_DEBUG_PROMPT` |\nDumps the fully-rendered prompt before tokenization. |\n`QUARTZ_NO_JINJA` |\nSkips the Jinja2 chat-template path even when the model has no native template. |\n`QUARTZ_NO_WARMUP` |\nSet by the Android host app before every engine start (see note below). |\n\n**Note on QUARTZ_NO_WARMUP:** this flag is set by the mobile app\n(\n\n`plugins/quartz/QuartzEngine.kt`\n\nin the Saient mobile repo) to try to\navoid a full-resident mmap warmup that can get the app OOM-killed on\nmemory-constrained phones. As of this writing the Quartz binary does not\nactually read this variable anywhere — the mmap warmup always runs\nunconditionally (see the `\"quartz: mmap warmup complete\"`\n\nlog line in\n`src/main.rs`\n\n). This was discovered while renaming `TINYQ4_NO_WARMUP`\n\n→\n`QUARTZ_NO_WARMUP`\n\n; it is a pre-existing no-op on the engine side, not\nsomething this rename introduced or fixed. Worth following up on if\nlow-memory devices are still seeing OOM kills at startup.The Saient mobile app also ships `libquartz-wan.so`\n\n, a *second*,\nunrelated binary used only for Wan2.1 text-to-video generation. It is\n**not** built from this repository — it's a pinned fork of\n[leejet/stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp)\n(MIT-licensed, C++/ggml), patched with a small progress-reporting patch\nand cross-compiled for Android/Vulkan. Its build lock lives in the mobile\nrepo at `engine/wan/BACKEND_LOCK.json`\n\nand `engine/wan/saient-progress.patch`\n\n.\nBoth binaries carry the \"Quartz\" name in the mobile app's UI/branding\n(`QuartzService`\n\n, `QuartzEngine.kt`\n\nsupervise both processes), but they\nare two separate codebases with two separate license situations — this\nrepo (Quartz proper) is what's being open-sourced here; the Wan fork's\nown license terms (MIT) and upstream attribution travel with it\nseparately. See the mobile repo's `docs/VIDEO_PIPELINE.md`\n\nfor how the\ntwo engines are orchestrated together on-device.\n\nThis repo has uncommitted local changes beyond the rename performed for\nopen-sourcing (a chat-template fix, KV-cache sizing fix, NEON Q4_K dot\nproduct, and the SD1.5/SDXL modules `sd15.rs`\n\n/`sdxl*.rs`\n\n/`tensor.rs`\n\n/\n`safetensors.rs`\n\n/`vulkan.rs`\n\nare untracked as of this writing). Review\nand commit those separately before or as part of publishing — this\nREADME describes the working tree as it stands, not necessarily the\nlast pushed commit.\n\nMIT — see [ LICENSE](/SaientAI/saient-quartz/blob/master/LICENSE). Matches the license of the bundled Wan\nvideo engine (see below), so the whole on-device stack is MIT.", "url": "https://wpnews.pro/news/open-sourced-a-local-wan-2-1-mobile-pipeline-for-on-device-video-generation", "canonical_source": "https://github.com/SaientAI/saient-quartz", "published_at": "2026-08-02 00:43:19+00:00", "updated_at": "2026-08-02 00:52:24.855071+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "ai-infrastructure", "developer-tools"], "entities": ["Quartz", "Saient", "Wan2.1 T2V", "Samsung Galaxy S24", "Rust", "Vulkan", "CUDA", "SDXL"], "alternates": {"html": "https://wpnews.pro/news/open-sourced-a-local-wan-2-1-mobile-pipeline-for-on-device-video-generation", "markdown": "https://wpnews.pro/news/open-sourced-a-local-wan-2-1-mobile-pipeline-for-on-device-video-generation.md", "text": "https://wpnews.pro/news/open-sourced-a-local-wan-2-1-mobile-pipeline-for-on-device-video-generation.txt", "jsonld": "https://wpnews.pro/news/open-sourced-a-local-wan-2-1-mobile-pipeline-for-on-device-video-generation.jsonld"}}