My first public release of Last-llama.cpp.zig, a standalone Windows runtime
and CLI for running local GGUF language models with Zig and llama.cpp.
It provides separate CPU and NVIDIA CUDA workers, a simple command-line
interface, and a versioned JSON-lines protocol that Zig applications can call
directly. It supports explicit backend selection, structured JSON output,
model aliases, lifecycle cleanup, and runtime/model identity information. It
does not download models automatically or require a hosted model API,
users supply their own GGUF model.
This started as a small part of a much larger system I am building. I needed
more control over local models than I was getting through third-party services,
especially around the exact model, backend, structured output, lifecycle, and
evidence of what actually ran. I found Deins/llama.cpp.zig, which showed me
that the Zig- to-llama.cpp bridge was possible and inspired the early work.
The active interface was later replaced with a narrower project owned
implementation, while retaining the original attribution and provenance.
I am sharing this mainly because it might save another Zig developer some work
or provide a useful starting point for a local model integration. This is not a
promise that I will maintain it indefinitely; my main focus remains the larger
project it came from. If it is useful to you, please feel free to fork it,
adapt it, and take it in the direction you need.
The current v0.1.0 release is unsigned, targets Windows x64, and includes
both CPU and CUDA support. Prebuilt binaries, source archives, checksums, and
qualification records are available on the
github .com/ AktionStudio/last-llama.cpp.zig/releases
The qualified source build uses Zig
0.17.0-dev.1676+c9dc9b798. Reproducing the generated C header binding uses Zig
0.14.1 as a separate documented helper. Users of the prebuilt Windows package
do not need Zig installed.
I am working on a simulator platform and needed this to work with my project which
also is being made using Zig.
This project was mainly motivated by my ADHD ignoring some paper work
and a substantial amount of AI tools were used during code and architecture
analysis, implementation and refactoring, testing, release engineering, and documentation.
I am putting it out publicly because it may be useful to somebody
else. If it is not a fit for this Showcase, that is completely fine; the intent
is simply to share the work rather than leave a potentially useful component
buried inside a private project.
Have Fun!