cd /news/artificial-intelligence/llama-cpp-and-ggml-hit-v1-0-stable-a… · home › topics › artificial-intelligence › article
[ARTICLE · art-170832] src=byteiota.com ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

llama.cpp and ggml Hit v1.0: Stable API, FP8 Cache, 35 Functions Gone

Llama.cpp and ggml shipped v1.0.0 on November 13, 3.5 years after Georgi Gerganov first ran a 7B language model on a MacBook CPU, committing both libraries to semantic versioning for the public C API in libllama and libggml. The release removes 35 deprecated functions from llama.h, including llama_set_warmup and the defrag_thold option, moves clip.h into mtmd, renames llama_set_embd_state_src to llama_set_ctx_state_src, and turns off unauthenticated tool access and private-URL media loading by default in llama-server. It also adds FP8 KV caching in the OCP f8_e4m3 format, which cuts the KV cache from 3,584 MiB to 1,792 MiB at a 32,768-token context on CPU only, and a host-memory MoE cache controlled by moe_cache_size and moe_cache_cpu_share.

read4 min views10 publishedNov 16, 2026
llama.cpp and ggml Hit v1.0: Stable API, FP8 Cache, 35 Functions Gone
Image: Byteiota (auto-discovered)

llama.cpp and ggml shipped v1.0.0 on November 13 — 3.5 years after Georgi Gerganov first proved you could run a 7B language model on a MacBook CPU with no GPU and no Python. The release isn’t a rewrite. It’s a contract. Both libraries now follow semantic versioning: major bumps signal breaking C API changes, minor versions add features, patches fix bugs. For the ecosystem that has quietly grown up around llama.cpp — Ollama, LM Studio, enterprise deployments — this is the moment a critical dependency becomes something you can confidently pin.

What “Stable” Actually Covers #

The semver guarantee applies to the public C API in ./include — specifically libllama (inference) and libggml (tensor operations). That’s meaningful. What it doesn’t cover: Python bindings, the llama-server HTTP API, or GGUF format versioning. Those remain outside the versioned contract. If you’re building directly against the C library, you have a real stability promise. If you’re building a UI on top of llama-server’s JSON endpoints, that’s still a matter of convention rather than contract.

This distinction matters. llama.cpp moves fast. The semver commitment doesn’t slow it down — it just makes breaking changes harder to miss.

What Actually Breaks on Upgrade #

This isn’t a no-op upgrade. The 1.0 release removes 35 deprecated functions from llama.h. If your C library code has been ignoring deprecation warnings, those are compile errors now. Specific removals include llama_set_warmup, the defrag_thold configuration option and its CLI flag, and the clip.h header, which has moved into mtmd. The function llama_set_embd_state_src was renamed to llama_set_ctx_state_src.

llama-server users face a different kind of breakage. Two behaviors that were on by default are now off: unauthenticated tool access and media from private URLs. To restore them, you now need explicit flags — --allow-unauthenticated-tools and --media-allow-private-urls. This is the right call from a security standpoint; SSRF, CORS, and DNS rebinding vulnerabilities were patched alongside this change. But if you’re running llama-server in a network-accessible setup and haven’t read the release notes, your deployment will silently break.

The distributed RPC protocol also moved to version 9. If you run llama.cpp nodes in a split-compute setup, upgrade all nodes at the same time.

FP8 KV Cache: Real Numbers, Honest Caveat #

The headline new feature is FP8 KV caching using the OCP f8_e4m3 format. At a 32,768-token context, the KV cache drops from 3,584 MiB to 1,792 MiB — a clean 50% reduction. For anyone running long-context sessions or RAG pipelines hitting memory constraints, that’s worth testing.

The caveat: FP8 KV caching is currently confirmed on CPU only. GPU acceleration isn’t validated yet. If GPU inference matters to you, the q8_0 quantization option lands at 1,904 MiB — nearly identical savings — while maintaining confirmed GPU support. Test FP8 against your hardware before relying on it in production.

./llama-cli -m model.gguf -ctk f8_e4m3 -ctv f8_e4m3 -c 32768

./llama-cli -m model.gguf -ctk q8_0 -ctv q8_0 -c 32768

MoE Cache: Big Models on Tight VRAM #

v1.0 adds a host-memory cache for mixture-of-experts models using LRFU/ClassLRU replacement, controlled by moe_cache_size and moe_cache_cpu_share. If you’re running Mixtral, Qwen-MoE, or DeepSeek-MoE on hardware where the full model doesn’t fit in VRAM, this gives llama.cpp a way to manage expert overflow in host memory without collapsing entirely to CPU inference.

Upgrade Checklist #

  • Build your current code with deprecation warnings enabled and fix everything before upgrading
  • Search scripts and configs for --defrag-thold and remove it
  • If you use llama-server in a network-accessible setup, add explicit flags for any previously-default behaviors you need
  • For distributed RPC setups: upgrade all nodes simultaneously (RPC protocol v9)
  • Test FP8 cache against your specific hardware before production use

Ollama and LM Studio Users: Nothing to Do Yet #

Ollama and LM Studio bundle their own llama.cpp version and ship updates independently — you’re not on v1.0 yet. GGUF model files are compatible across versions, so nothing needs converting. When these tools ship their next updates incorporating v1.0, you’ll inherit the FP8 cache and MoE improvements without any action.

The semver commitment is what makes that transition predictable. For a project that powers nearly everything in local AI inference, predictable matters. Check the full upgrade guide and the community analysis for more detail.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @llama.cpp 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/llama-cpp-and-ggml-h…] indexed:0 read:4min 2026-11-16 · —