{"slug": "sana-cpp-nvidia-s-sana-t2i-model-in-c-4-8x-faster-than-pytorch", "title": "Sana.cpp – Nvidia's Sana T2I model in C++, 4.8x faster than PyTorch", "summary": "A new C++ implementation of NVIDIA's Sana 0.6B text-to-image model, named Sana.cpp, runs approximately 4.8 times faster than the PyTorch pipeline on Apple Silicon CPU. The project, developed by GitHub user cconthekeyboard, provides a minimalistic inference pipeline optimized for Apple Silicon, with pre-converted GGUF weights available on Hugging Face. The code is released under the MIT license, while the model weights are subject to NVIDIA's separate terms.", "body_md": "A minimalistic C++ implementation of [Sana](https://github.com/NVlabs/Sana)'s (0.6B) text-to-image\ninference pipeline optimized for Apple Silicon\nCPU — **~4.8x faster than the PyTorch pipeline on Apple Silicon CPU **\n\nRequirements: CMake >= 3.16, a C++17 compiler, and macOS on Apple Silicon\n(the primary target — it links against the `Accelerate`\n\nframework and builds\nwith `-mcpu=native`\n\n; a non-Apple `-march=native`\n\npath exists but is less\nexercised). `llama.cpp`\n\n/`ggml`\n\naren't vendored as source — they're pulled\nautomatically at configure time via CMake `FetchContent`\n\n, pinned to a fixed\ntag, so a plain `cmake`\n\ninvocation is enough to fetch them.\n\n-\n**Build:**\n\n```\ncmake -S . -B build\ncmake --build build -j\n```\n\nThis produces\n\n`sana_infer`\n\n,`bench_full_pipeline`\n\n, and the unit-test binaries, all under`build/`\n\n. -\n**Get the model weights.**`sana_infer`\n\nreads weights from a directory of`.gguf`\n\nfiles (default`../weights`\n\nrelative to the build directory). Pre-converted files are hosted at[doobluhc/sana-cpp-weights](https://huggingface.co/doobluhc/sana-cpp-weights)— fetch them with plain`curl`\n\n, no Python required:\n\n```\n./download_weights.sh weights\n```\n\n-\n**Run inference:**\n\n```\ncd build\n./sana_infer --prompt \"a house by the lake\" --output out.png\n```\n\nRun\n\n`./sana_infer --help`\n\nfor the full option list (`--negative-prompt`\n\n,`--steps`\n\n,`--seed`\n\n,`--guidance`\n\n,`--weights-dir`\n\n,`--gemma-gguf`\n\n, ...).Python 3 with\n\n`torch`\n\nand`diffusers`\n\nis only needed if you also want to run the PyTorch reference benchmarks below — not for any of the steps above.\n\n**Gemma-2 text encoder**(`src/gemma_encoder.*`\n\n) — runs on the vendored[llama.cpp](https://github.com/ggml-org/llama.cpp)/`ggml`\n\ninference engine.**Transformer denoiser**(`src/transformer*.*`\n\n) and**DPM-Solver++ scheduler**(`src/scheduler.*`\n\n) — the diffusion denoising loop.**VAE decoder**(`src/vae*.*`\n\n) — turns final latents into an image.(`sana_infer`\n\n`src/infer_main.cpp`\n\n) — the CLI that chains all three stages end to end and writes a PNG/PPM.(`bench_full_pipeline`\n\n`tests/bench_full_pipeline.cpp`\n\n), paired with`tools/bench_reference_full_pipeline.py`\n\n, so the whole pipeline's inference speed can be timed and compared directly against the PyTorch reference.\n\nThe whole pipeline (encode + denoise + decode) can be timed on both implementations, on the same inputs, and compared directly:\n\n| Stage | C++ | Python reference |\n|---|---|---|\n| Full pipeline (encode + denoise + decode) | `./bench_full_pipeline <warmup> <iters>` |\n`python3 tools/bench_reference_full_pipeline.py` |\n\n```\nctest --test-dir build --output-on-failure\n```\n\nRuns the self-contained unit tests (tensor ops, transformer block, scheduler, Gemma-2 encoder) — none of them need model weights or any external fixture data.\n\n```\nsrc/                  C++ library + sana_infer CLI\ntests/                self-contained unit tests + the full-pipeline benchmark\ntools/                the PyTorch full-pipeline benchmark\ndownload_weights.sh   fetches pre-converted .gguf weights from Hugging Face\n```\n\nMIT — see [ LICENSE](/cconthekeyboard/sana.cpp/blob/main/LICENSE).\n\nThis covers the code in this repo only. The Sana model weights themselves\nare published separately by NVIDIA/Efficient-Large-Model under their own\nterms — check\n[the upstream model's license](https://huggingface.co/Efficient-Large-Model/Sana_600M_1024px_diffusers)\nbefore using them.", "url": "https://wpnews.pro/news/sana-cpp-nvidia-s-sana-t2i-model-in-c-4-8x-faster-than-pytorch", "canonical_source": "https://github.com/cconthekeyboard/sana.cpp", "published_at": "2026-08-17 02:08:15+00:00", "updated_at": "2026-08-17 02:40:34.562846+00:00", "lang": "en", "topics": ["artificial-intelligence", "generative-ai", "ai-tools", "developer-tools"], "entities": ["NVIDIA", "Sana", "Apple Silicon", "PyTorch", "Hugging Face", "llama.cpp", "ggml", "cconthekeyboard"], "alternates": {"html": "https://wpnews.pro/news/sana-cpp-nvidia-s-sana-t2i-model-in-c-4-8x-faster-than-pytorch", "markdown": "https://wpnews.pro/news/sana-cpp-nvidia-s-sana-t2i-model-in-c-4-8x-faster-than-pytorch.md", "text": "https://wpnews.pro/news/sana-cpp-nvidia-s-sana-t2i-model-in-c-4-8x-faster-than-pytorch.txt", "jsonld": "https://wpnews.pro/news/sana-cpp-nvidia-s-sana-t2i-model-in-c-4-8x-faster-than-pytorch.jsonld"}}