{"slug": "run-glm-5-3-flash-locally-gsq-and-rco-quantization-explained", "title": "Run GLM 5.3 Flash Locally: GSQ and RCO Quantization Explained", "summary": "An independent research group in Austria has developed two quantization techniques, GSQ and RCO, that compress Z.ai's 320-billion-parameter GLM 5.3 Flash vision-language model from roughly 320GB at full precision to under 140GB while losing only about two points on a 2,000-question benchmark. GSQ tests several rounding candidates per weight before locking in a value, while RCO allocates a fixed 137GB bit budget across layers so sensitive layers retain more precision. The demonstrated setup still requires two 80GB H100 GPUs to split the load, and multilingual quality remains weaker on low-resource languages, a limitation inherited from the original GLM 5.3.", "body_md": "# Run GLM 5.3 Flash Locally: GSQ and RCO Quantization Explained\n\nHow GSQ and RCO quantization shrink the 320B GLM 5.3 Flash model to under 140GB, and how to build llama.cpp and serve it on local GPUs.\n\n## What is GLM 5.3 Flash, and why does it need special quantization?\n\nGLM 5.3 Flash is a 320 billion parameter vision-language model from Z.ai (the lab behind the GLM family), built to handle both images and text. At full precision, a model this size needs roughly 320GB or more of storage just to hold the weights, which puts it out of reach for almost anyone outside a data center. Two quantization techniques called GSQ and RCO, developed by an independent research group in Austria, compress that same model down to under 140GB while staying within about two points of the full model’s score on a 2,000-question benchmark. That’s the gap between “impossible to run at home” and “runs on two workstation GPUs.”\n\n## TL;DR\n\n- **GLM 5.3 Flash** is a 320B parameter image-and-text model that normally requires around 320GB of storage at full precision.\n- **GSQ and RCO quantization** compress it to under 140GB while losing only about two points on a 2,000-question benchmark.\n- **GSQ operates weight by weight** , testing several rounding options for each number before locking in whichever one preserves model quality best.\n- **RCO operates layer by layer** , allocating a fixed total bit budget across the model so sensitive layers keep more precision and less important ones get compressed harder.\n- **Running the quantized model still requires serious hardware** , specifically two 80GB H100 GPUs to split the load in the demonstrated setup.\n- **Vision capability survives quantization** , with the model able to describe and roughly reconstruct a reference image into HTML/CSS.\n- **Multilingual quality is inconsistent** , performing well on major languages but noticeably weaker on low-resource ones, a limitation inherited from the original GLM 5.3.\n\n## Remy is new. The platform isn't.\n\nRemy is the latest expression of years of platform work. Not a hastily wrapped LLM.\n\n## How does GSQ quantization work?\n\nQuantization is the process of shrinking every number inside a model from its native 16-bit representation down to something much smaller, like 4 bits or fewer. Multiply 16 bits by 320 billion parameters and you get an enormous file. Cutting that down to a handful of bits per number makes the file dramatically smaller, at the cost of some precision.\n\nThe catch is that when you shrink a number, you have to round it to one of a small set of allowed values. Most quantization methods just round to the nearest available value and move on. That’s fast, but it leaves accuracy on the table.\n\nGSQ takes a smarter approach. Instead of committing to the nearest rounding option immediately, it tests several candidate values for each weight, checks which one keeps the model performing best, and only then locks in its final choice. It’s the difference between guessing and actually verifying before committing. This happens at the level of individual weights inside the model, which is why it’s described as working at the “small picture” level.\n\n## How does RCO quantization work?\n\nIf GSQ handles individual numbers, RCO handles something much bigger: how many bits each layer of the model gets in total.\n\nNot every layer in a 320 billion parameter model matters equally. Some layers are more sensitive to compression and need to stay closer to full precision, while others can be squeezed hard with little effect on output quality. RCO’s job is to look at the entire model and distribute a fixed size budget, in this case 137GB, across all the layers so that the important ones keep more detail and the whole model still lands exactly on the target size.\n\nIn practice, RCO runs first at the “big picture” level, deciding the bit budget for each layer. Then GSQ runs at the “small picture” level, deciding how to make the best use of that budget for every weight inside each layer. RCO decides where to spend the budget. GSQ decides how to spend it well.\n\n## What is GGUF, and why does it matter here?\n\nGGUF is the file format used to package quantized model weights so that tools like llama.cpp can load and run them efficiently. After GSQ and RCO compress the numbers inside GLM 5.3 Flash, the result still needs to be stored in a format that inference engines understand. GGUF is that container. It’s the reason a 320B parameter model quantized down to roughly 3.5 bits per weight on average can be downloaded, loaded, and served without needing the original training framework.\n\n## What hardware do you need to run GLM 5.3 Flash locally?\n\nEven after quantization to under 140GB, this is not a model you run on a laptop or a single consumer GPU. The demonstrated setup used two NVIDIA H100 GPUs with 80GB of VRAM each, giving 160GB of combined VRAM, enough to split the roughly 137GB quantized model across both cards with headroom for context and activations.\n\n## \nPlans first.\n*Then code.*\n\nRemy writes the spec, manages the build, and ships the app.\n\nThe build process itself involves cloning and compiling llama.cpp from source (a build using 16 CPU threads took around 15 to 20 minutes), downloading the quantized GGUF weights from Hugging Face via the CLI, and then launching the server with a command that splits the model across both GPUs. This is enterprise-tier hardware. Running large multi-GPU inference sessions for testing and benchmarking is not cheap either, with cloud GPU time for a single extended session easily running into tens of dollars.\n\n## How good is GLM 5.3 Flash after quantization?\n\nOn vision tasks, the quantized model held up well. Given a reference image of an aurora over a snowy landscape, it was able to generate an HTML/CSS reconstruction that captured the diagonal sweep of the aurora, the layered pink and violet color bands, and the general composition of the scene, close to what a full-precision model would be expected to produce.\n\nOn multilingual generation, results were mixed. Asked to produce short, playful text across dozens of languages in their native scripts, the model performed solidly on major world languages but was noticeably weaker on low-resource languages, sometimes repeating or looping. This isn’t a quantization artifact specifically. The same weakness showed up in the original, unquantized GLM 5.3, meaning it’s a property of the base model’s training data rather than something introduced by GSQ or RCO.\n\nOn a multi-step reasoning problem combining population genetics and conditional probability, the model worked through the allele frequency calculations and arrived at the correct final answer, though it printed out some of its intermediate tool-call formatting in the process and took noticeably longer to reason through the problem than the unquantized version.\n\n## Is running GLM 5.3 Flash with GSQ and RCO worth it?\n\nIf your goal is fitting a 320B parameter vision-language model onto hardware you can actually own or rent at a reasonable multi-GPU tier, GSQ and RCO make that possible in a way standard quantization methods often can’t match at this size, since the two-point benchmark gap against the full model is a strong result for compressing a model by more than half its footprint. The tradeoffs are real: multilingual coverage of less common languages remains weak, reasoning can take longer and occasionally loop, and the hardware bar (dual 80GB-class GPUs) is still well beyond typical consumer setups. For teams already comparing large open models like GLM against alternatives such as Qwen, this quantized build is worth testing directly against your own workload before committing infrastructure to it.\n\n## Frequently Asked Questions\n\n### What does GSQ stand for and what problem does it solve?\n\nGSQ improves the rounding step in quantization. Rather than rounding each weight to the nearest allowed value by default, it tests multiple candidate values and picks the one that best preserves model performance, reducing the accuracy loss that normally comes from aggressive compression.\n\n### What does RCO do differently from standard quantization?\n\nRCO decides how much of a fixed total bit budget each layer of the model receives, giving more precision to sensitive layers and compressing less important ones harder, so the entire model lands at an exact target file size while minimizing overall quality loss.\n\n### How small does GLM 5.3 Flash get with this quantization?\n\nThe 320 billion parameter model, which normally needs roughly 320GB of storage, is compressed to about 137GB, an average of roughly 3.5 bits per parameter, while staying within about two benchmark points of the full-precision version.\n\n### Can GLM 5.3 Flash run on a single GPU?\n\n### Built like a system. Not vibe-coded.\n\nRemy manages the project — every layer architected, not stitched together at the last second.\n\nNot in this configuration. The demonstrated setup used two 80GB H100 GPUs with the model split across both, since even a 137GB quantized model exceeds the VRAM of a single H100.\n\n### Does quantization hurt the model’s vision capabilities?\n\nVision support remained intact in testing. The quantized model successfully analyzed a reference image and reconstructed its key visual elements, colors, and composition into HTML and CSS, suggesting GSQ and RCO preserve multimodal capability along with text performance.", "url": "https://wpnews.pro/news/run-glm-5-3-flash-locally-gsq-and-rco-quantization-explained", "canonical_source": "https://www.mindstudio.ai/blog/glm-5-3-flash-gguf-gsq-rco-local/", "published_at": "2026-09-15 00:00:00+00:00", "updated_at": "2026-09-15 17:18:27.639696+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure", "computer-vision", "ai-tools"], "entities": ["GLM 5.3 Flash", "Z.ai", "GSQ", "RCO", "llama.cpp", "H100"], "alternates": {"html": "https://wpnews.pro/news/run-glm-5-3-flash-locally-gsq-and-rco-quantization-explained", "markdown": "https://wpnews.pro/news/run-glm-5-3-flash-locally-gsq-and-rco-quantization-explained.md", "text": "https://wpnews.pro/news/run-glm-5-3-flash-locally-gsq-and-rco-quantization-explained.txt", "jsonld": "https://wpnews.pro/news/run-glm-5-3-flash-locally-gsq-and-rco-quantization-explained.jsonld"}}