Sana.cpp – Nvidia's Sana T2I model in C++, 4.8x faster than PyTorch A new C++ implementation of NVIDIA's Sana 0.6B text-to-image model, named Sana.cpp, runs approximately 4.8 times faster than the PyTorch pipeline on Apple Silicon CPU. The project, developed by GitHub user cconthekeyboard, provides a minimalistic inference pipeline optimized for Apple Silicon, with pre-converted GGUF weights available on Hugging Face. The code is released under the MIT license, while the model weights are subject to NVIDIA's separate terms. A minimalistic C++ implementation of Sana https://github.com/NVlabs/Sana 's 0.6B text-to-image inference pipeline optimized for Apple Silicon CPU — ~4.8x faster than the PyTorch pipeline on Apple Silicon CPU Requirements: CMake = 3.16, a C++17 compiler, and macOS on Apple Silicon the primary target — it links against the Accelerate framework and builds with -mcpu=native ; a non-Apple -march=native path exists but is less exercised . llama.cpp / ggml aren't vendored as source — they're pulled automatically at configure time via CMake FetchContent , pinned to a fixed tag, so a plain cmake invocation is enough to fetch them. - Build: cmake -S . -B build cmake --build build -j This produces sana infer , bench full pipeline , and the unit-test binaries, all under build/ . - Get the model weights. sana infer reads weights from a directory of .gguf files default ../weights relative to the build directory . Pre-converted files are hosted at doobluhc/sana-cpp-weights https://huggingface.co/doobluhc/sana-cpp-weights — fetch them with plain curl , no Python required: ./download weights.sh weights - Run inference: cd build ./sana infer --prompt "a house by the lake" --output out.png Run ./sana infer --help for the full option list --negative-prompt , --steps , --seed , --guidance , --weights-dir , --gemma-gguf , ... .Python 3 with torch and diffusers is only needed if you also want to run the PyTorch reference benchmarks below — not for any of the steps above. Gemma-2 text encoder src/gemma encoder. — runs on the vendored llama.cpp https://github.com/ggml-org/llama.cpp / ggml inference engine. Transformer denoiser src/transformer . and DPM-Solver++ scheduler src/scheduler. — the diffusion denoising loop. VAE decoder src/vae . — turns final latents into an image. sana infer src/infer main.cpp — the CLI that chains all three stages end to end and writes a PNG/PPM. bench full pipeline tests/bench full pipeline.cpp , paired with tools/bench reference full pipeline.py , so the whole pipeline's inference speed can be timed and compared directly against the PyTorch reference. The whole pipeline encode + denoise + decode can be timed on both implementations, on the same inputs, and compared directly: | Stage | C++ | Python reference | |---|---|---| | Full pipeline encode + denoise + decode | ./bench full pipeline