llama.cpp and ggml Hit v1.0: Stable API, FP8 Cache, 35 Functions Gone Llama.cpp and ggml shipped v1.0.0 on November 13, 3.5 years after Georgi Gerganov first ran a 7B language model on a MacBook CPU, committing both libraries to semantic versioning for the public C API in libllama and libggml. The release removes 35 deprecated functions from llama.h, including llama_set_warmup and the defrag_thold option, moves clip.h into mtmd, renames llama_set_embd_state_src to llama_set_ctx_state_src, and turns off unauthenticated tool access and private-URL media loading by default in llama-server. It also adds FP8 KV caching in the OCP f8_e4m3 format, which cuts the KV cache from 3,584 MiB to 1,792 MiB at a 32,768-token context on CPU only, and a host-memory MoE cache controlled by moe_cache_size and moe_cache_cpu_share. llama.cpp and ggml shipped v1.0.0 on November 13 — 3.5 years after Georgi Gerganov first proved you could run a 7B language model on a MacBook CPU with no GPU and no Python. The release isn’t a rewrite. It’s a contract. Both libraries now follow semantic versioning https://aiweekly.co/alerts/llamacpp-ships-v100-with-semver-for-libllama-and-ggml : major bumps signal breaking C API changes, minor versions add features, patches fix bugs. For the ecosystem that has quietly grown up around llama.cpp — Ollama, LM Studio, enterprise deployments — this is the moment a critical dependency becomes something you can confidently pin. What “Stable” Actually Covers The semver guarantee applies to the public C API in ./include — specifically libllama inference and libggml tensor operations . That’s meaningful. What it doesn’t cover: Python bindings, the llama-server HTTP API, or GGUF format versioning. Those remain outside the versioned contract. If you’re building directly against the C library, you have a real stability promise. If you’re building a UI on top of llama-server’s JSON endpoints, that’s still a matter of convention rather than contract. This distinction matters. llama.cpp moves fast. The semver commitment doesn’t slow it down — it just makes breaking changes harder to miss. What Actually Breaks on Upgrade This isn’t a no-op upgrade. The 1.0 release removes 35 deprecated functions from llama.h . If your C library code has been ignoring deprecation warnings, those are compile errors now. Specific removals include llama set warmup , the defrag thold configuration option and its CLI flag, and the clip.h header, which has moved into mtmd . The function llama set embd state src was renamed to llama set ctx state src . llama-server users face a different kind of breakage. Two behaviors that were on by default are now off: unauthenticated tool access and loading media from private URLs. To restore them, you now need explicit flags — --allow-unauthenticated-tools and --media-allow-private-urls . This is the right call from a security standpoint; SSRF, CORS, and DNS rebinding vulnerabilities were patched alongside this change. But if you’re running llama-server in a network-accessible setup and haven’t read the release notes https://casrai.org/news/llama-cpp-ggml-1-0-stable-api-release-2026 , your deployment will silently break. The distributed RPC protocol also moved to version 9. If you run llama.cpp nodes in a split-compute setup, upgrade all nodes at the same time. FP8 KV Cache: Real Numbers, Honest Caveat The headline new feature is FP8 KV caching using the OCP f8 e4m3 format. At a 32,768-token context, the KV cache drops from 3,584 MiB to 1,792 MiB — a clean 50% reduction. For anyone running long-context sessions or RAG pipelines hitting memory constraints, that’s worth testing. The caveat: FP8 KV caching is currently confirmed on CPU only. GPU acceleration isn’t validated yet. If GPU inference matters to you, the q8 0 quantization option lands at 1,904 MiB — nearly identical savings — while maintaining confirmed GPU support. Test FP8 against your hardware before relying on it in production. FP8 KV cache — 50% memory reduction at 32K context CPU validated ./llama-cli -m model.gguf -ctk f8 e4m3 -ctv f8 e4m3 -c 32768 Need GPU acceleration? q8 0 gives similar savings with confirmed GPU support: ./llama-cli -m model.gguf -ctk q8 0 -ctv q8 0 -c 32768 MoE Cache: Big Models on Tight VRAM v1.0 adds a host-memory cache for mixture-of-experts models using LRFU/ClassLRU replacement, controlled by moe cache size and moe cache cpu share . If you’re running Mixtral, Qwen-MoE, or DeepSeek-MoE on hardware where the full model doesn’t fit in VRAM, this gives llama.cpp a way to manage expert overflow in host memory without collapsing entirely to CPU inference. Upgrade Checklist - Build your current code with deprecation warnings enabled and fix everything before upgrading - Search scripts and configs for --defrag-thold and remove it - If you use llama-server in a network-accessible setup, add explicit flags for any previously-default behaviors you need - For distributed RPC setups: upgrade all nodes simultaneously RPC protocol v9 - Test FP8 cache against your specific hardware before production use Ollama and LM Studio Users: Nothing to Do Yet Ollama and LM Studio bundle their own llama.cpp version and ship updates independently — you’re not on v1.0 yet. GGUF model files are compatible across versions, so nothing needs converting. When these tools ship their next updates incorporating v1.0, you’ll inherit the FP8 cache and MoE improvements without any action. The semver commitment is what makes that transition predictable. For a project that powers nearly everything in local AI inference, predictable matters. Check the full upgrade guide https://www.systemdesign.academy/explained/llama-cpp-v1 and the community analysis https://dev.to/delimitter/llamacpp-and-ggml-ship-v100-semantic-versioning-comes-to-the-local-ai-stack-1ag4 for more detail.