# What Are GSQ and RCO? Das Lab's New LLM Quantization Method > Source: > Published: 2026-09-08 00:00:00+00:00 # What Are GSQ and RCO? Das Lab's New LLM Quantization Method GSQ and RCO are new compression techniques from ISTA's Das Lab that shrink LLMs per-weight while targeting an exact size with minimal accuracy loss. ## What are GSQ and RCO? GSQ (Gumbel Softmax Quantization) and RCO (Riemannian Constrained Optimization) are two linked techniques for compressing large language models, developed by Das Lab at the Institute of Science and Technology Austria (ISTA), the same group behind GPTQ. GSQ decides how many bits each individual weight in a model needs to stay close to its original value. RCO then takes those per-weight sensitivity signals and spends a fixed total size budget across the whole model, pushing more bits toward weights that matter and fewer toward ones that don’t, so the final compressed model lands on an exact target size with as little accuracy loss as possible. ## TL;DR - **GSQ works weight by weight** , testing bit depths like two, three, or four bits per value and learning which precision keeps each weight closest to its original number. - **RCO works at the whole-model level** , allocating a fixed size budget (for example, a target of roughly 11.8 GB) across all tensors based on how sensitive each one is. - **Sensitive tensors get more bits and tolerant tensors get fewer** , which is the core idea behind squeezing a model down without a proportional accuracy hit. - **Das Lab is the same team behind GPTQ** , a quantization method already used widely across the AI industry, which gives GSQ and RCO some built-in credibility. - **“Task lossless” is the term Das Lab uses** for quantized models that perform essentially the same as their full-precision originals on real benchmarks despite being a fraction of the size. - **A hands-on test of a quantized Qwen3.8 27B model** showed retained long-form reasoning, working code generation, bug-catching on a tricky SQL query, and correct multi-step chemistry reasoning, all while running in well under half the memory of the unquantized model. - **Multilingual coverage was the weak spot** , with major languages holding up well but several low-resource languages coming out barely coherent, a limitation that also showed up in the original unquantized model. ## Plans first. *Then code.* Remy writes the spec, manages the build, and ships the app. ## Why does quantization matter for running LLMs locally? Modern language models store their learned parameters, called weights, at high numerical precision. That precision is expensive: it takes more memory to store and more compute to move around during inference. A 27 billion parameter model stored at full precision can require far more VRAM than most consumer or even prosumer GPUs have available. Quantization reduces the number of bits used to represent each weight. Instead of storing a weight at 16 bits, you might store it at four, three, or even two bits. Done carelessly, this destroys the model’s ability to reason and generate coherent text, because tiny numerical differences compound across billions of calculations. Done well, it lets a model with tens of billions of parameters run on a single high-end GPU, or even a laptop, while keeping most of its original capability. The challenge every quantization method has to solve is the same: which weights can tolerate aggressive compression, and which ones will break the model if you touch them. ## How does GSQ decide bit depth per weight? GSQ, short for Gumbel Softmax Quantization, addresses that challenge at the most granular level possible: one weight at a time. For every weight in the model, GSQ evaluates multiple candidate bit depths, such as two-bit, three-bit, or four-bit representations, and learns which one keeps that particular weight closest to its original trained value. Not all weights behave the same way under compression. Some barely shift no matter how aggressively you round them down. Others are highly sensitive, and even a small loss of precision changes the model’s output in ways that cascade through later computations. GSQ’s job is to find that sweet spot independently for every weight, rather than applying one blanket bit depth across an entire tensor or the whole model. That weight-by-weight granularity is what distinguishes it from cruder quantization schemes that assign a single precision level to large blocks of parameters regardless of how sensitive individual values inside those blocks actually are. ## How does RCO allocate a compression budget? Knowing how sensitive each weight is doesn’t automatically tell you how to compress the entire model. That’s the problem RCO, Riemannian Constrained Optimization, is built to solve. It operates one level above GSQ: once sensitivity information exists for every weight, RCO decides how to distribute a fixed overall size budget across all the tensors in the model. The practical goal is hitting an exact file size target, for example around 11.8 GB, while wasting as little accuracy as possible along the way. Tensors identified as more sensitive get allocated more bits. Tensors that tolerate heavy compression get fewer. Rather than compressing everything uniformly, RCO treats the size budget like a resource to be spent strategically, weighted by where accuracy is actually at risk. The name comes from the mathematical optimization approach used to solve this allocation problem under the constraint of hitting a specific total size, drawing on Riemannian geometry, a branch of mathematics used for optimization problems where the space of possible solutions isn’t flat. ### Everyone else built a construction worker. We built the contractor. One file at a time. UI, API, database, deploy. Together, GSQ supplies the per-weight precision signal, and RCO turns that signal into a model-wide compression plan that lands on a specific size. ## Is task lossless quantization actually lossless? “Task lossless” doesn’t mean mathematically identical to the original model. It means the quantized model performs essentially the same as the full-precision version on real benchmarks, despite being dramatically smaller. Das Lab applied GSQ and RCO to compress a 27 billion parameter Qwen3.8 model down to several sizes, with builds as small as roughly 8 to 12 GB, and labeled a build in that range as their recommended, task lossless configuration. In hands-on testing of one such quantized build served locally through llama.cpp, the model retained its long chain-of-thought reasoning style, in some cases thinking through a single problem for over ten minutes without looping or degrading into incoherence. It produced a working, visually accurate HTML and canvas animation on a first attempt, caught a deliberately hidden logical bug in a long nested SQL query (a descending sort silently swapped for ascending), and worked through a multi-step chemistry problem involving solubility equilibrium and mass balance to a correct final numerical answer. VRAM use for that build came in at just over 28 GB on a 48 GB GPU, well below what the unquantized 27 billion parameter model would require. The one area that showed clear weakness was multilingual output. Major world languages like Mandarin, Spanish, and Arabic came through solidly, and the model handled a constructed script (Elder Futhark runes) and fictional languages (Klingon, Quenya) honestly, flagging missing vocabulary rather than hallucinating fluency. But several low-resource languages produced barely coherent output, a limitation that appears to trace back to the original model’s training rather than to the quantization process itself. ## How does this compare to other quantization methods? Most widely used quantization approaches, including earlier methods like GPTQ, apply a fixed bit width across large groups of weights or an entire model, sometimes with some per-layer variation. That’s computationally simpler but leaves accuracy on the table, because within any given tensor some weights are far more sensitive than others. The GSQ and RCO combination pushes the sensitivity analysis down to the individual weight level and then treats the overall compression budget as an optimization problem to be solved across the whole model, rather than a fixed rule applied uniformly. The tradeoff is complexity: finding an optimal bit depth per weight and then solving a global allocation problem takes more computation upfront than simpler uniform quantization schemes. The payoff, based on early testing of released models, is a compressed model that holds onto reasoning quality, coding ability, and factual accuracy closer to its full-precision original, at a size small enough to run on a single consumer or prosumer GPU. ## Frequently Asked Questions ### What does GSQ stand for and what does it actually do? GSQ stands for Gumbel Softmax Quantization. It evaluates each individual weight in a model against multiple candidate bit depths and selects the precision level that keeps that specific weight closest to its original trained value, rather than applying one bit depth to an entire tensor or layer. ## Remy is new. The platform isn't. Remy is the latest expression of years of platform work. Not a hastily wrapped LLM. ### What does RCO stand for and how is it different from GSQ? RCO stands for Riemannian Constrained Optimization. While GSQ scores individual weights for sensitivity, RCO uses that information to allocate a fixed total size budget across the entire model, giving more bits to sensitive tensors and fewer to tolerant ones, so the final model hits an exact target size. ### Who developed GSQ and RCO? Das Lab, a research group at the Institute of Science and Technology Austria (ISTA), developed both methods. The same lab previously created GPTQ, a quantization technique already used across the AI industry. ### Does quantization with GSQ and RCO cause accuracy loss? Some loss is inherent to any quantization process, but Das Lab describes certain compressed builds as “task lossless,” meaning they perform essentially the same as the full-precision model on real benchmarks. Hands-on testing of a compressed 27 billion parameter model showed strong results in coding, bug detection, and multi-step scientific reasoning, though multilingual performance on low-resource languages showed clear weaknesses. ### Can I run a GSQ/RCO-quantized model on a consumer GPU? It depends on the specific build size. Das Lab released multiple sizes for their compressed models, with some builds as small as roughly 8 to 12 GB, which fits within the VRAM of many consumer and prosumer GPUs, well below what the equivalent full-precision model would require.