A minimalistic C++ implementation of Sana's (0.6B) text-to-image inference pipeline optimized for Apple Silicon CPU — **~4.8x faster than the PyTorch pipeline on Apple Silicon CPU **
Requirements: CMake >= 3.16, a C++17 compiler, and macOS on Apple Silicon
(the primary target — it links against the Accelerate
framework and builds
with -mcpu=native
; a non-Apple -march=native
path exists but is less
exercised). llama.cpp
/ggml
aren't vendored as source — they're pulled
automatically at configure time via CMake FetchContent
, pinned to a fixed
tag, so a plain cmake
invocation is enough to fetch them.
Build:
cmake -S . -B build
cmake --build build -j
This produces
sana_infer
,bench_full_pipeline
, and the unit-test binaries, all underbuild/
. -
Get the model weights.sana_infer
reads weights from a directory of.gguf
files (default../weights
relative to the build directory). Pre-converted files are hosted atdoobluhc/sana-cpp-weights— fetch them with plaincurl
, no Python required:
./download_weights.sh weights
Run inference:
cd build
./sana_infer --prompt "a house by the lake" --output out.png
Run
./sana_infer --help
for the full option list (--negative-prompt
,--steps
,--seed
,--guidance
,--weights-dir
,--gemma-gguf
, ...).Python 3 with
torch
anddiffusers
is only needed if you also want to run the PyTorch reference benchmarks below — not for any of the steps above.
Gemma-2 text encoder(src/gemma_encoder.*
) — runs on the vendoredllama.cpp/ggml
inference engine.Transformer denoiser(src/transformer*.*
) andDPM-Solver++ scheduler(src/scheduler.*
) — the diffusion denoising loop.VAE decoder(src/vae*.*
) — turns final latents into an image.(sana_infer
src/infer_main.cpp
) — the CLI that chains all three stages end to end and writes a PNG/PPM.(bench_full_pipeline
tests/bench_full_pipeline.cpp
), paired withtools/bench_reference_full_pipeline.py
, so the whole pipeline's inference speed can be timed and compared directly against the PyTorch reference.
The whole pipeline (encode + denoise + decode) can be timed on both implementations, on the same inputs, and compared directly:
| Stage | C++ | Python reference |
|---|---|---|
| Full pipeline (encode + denoise + decode) | ./bench_full_pipeline <warmup> <iters> |
|
python3 tools/bench_reference_full_pipeline.py |
ctest --test-dir build --output-on-failure
Runs the self-contained unit tests (tensor ops, transformer block, scheduler, Gemma-2 encoder) — none of them need model weights or any external fixture data.
src/ C++ library + sana_infer CLI
tests/ self-contained unit tests + the full-pipeline benchmark
tools/ the PyTorch full-pipeline benchmark
download_weights.sh fetches pre-converted .gguf weights from Hugging Face
MIT — see LICENSE.
This covers the code in this repo only. The Sana model weights themselves are published separately by NVIDIA/Efficient-Large-Model under their own terms — check the upstream model's license before using them.