Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them

A study of all 305 eligible code-capable GGUF tags across 15 model lines in the official Ollama registry found that none of the public model registries examined documented functional testing of their quantized LLM files, meaning developers can download non-functional artifacts. The finding matters because developers rely on these registries to run quantized large language models locally, and the work frames how to catch broken releases.

Developers download quantized large language models LLMs from public registries to run them locally. None of the registries we examined documented functional testing of their files. We tested all 305 eligible code-capable GGUF tags the llama.cpp format across 15 model lines in the official Ollam