Negative squaring – pre-tilted 3-bit quantization beat naive 4-bit

Researchers introduced negative squaring, a pre-tilting technique that adjusts weights against expected rounding errors before quantization, achieving 77% error reduction in a toy recurrent network with 3-bit quantization outperforming naive 4-bit. The method clips tilts to half a quantization step to preserve reasoning, but scaling to real LLMs remains unsolved.

Code accompanying the writeup "Negative Squaring: Pre-Tilting Weights to Preserve Reasoning in Quantized Models" July 2026 . See negative-squaring-paper 1.md in this repository for the full plain-language paper. Before quantizing a model, tilt each weight against the error the rounding will cause across the model's whole multi-step reasoning trajectory — clipped to half a quantization step, so the tilt only decides which way borderline weights round. | File | What it does | Key result | |---|---|---| 1 first experiment.py | Random-search pre-tilt vs naive 4-bit quantization on a 12-layer, 30-step recurrent toy network | ~18% trajectory error removed; decision flips 14/20 → 8/20 | 2 gradient attempt.py | Straight-through gradient search, unconstrained | Backfires — test error gets worse documented negative result | 3 final with clipping.py | Gradient + random + combo searches, with tilts clipped to half a quantization step | 77% error removed; decision flips 20/50 → 4-5/50 | Requires only Python 3 and numpy: pip install numpy python 3 final with clipping.py Each script is self-contained, seeded, and reproduces the numbers in the writeup. Runtime is seconds to a few minutes on any laptop. To reproduce the 3-bit and 2-bit cliff results, change BITS = 4 to 3 or 2 on line 4 of script 3. - Toy scale: ~49k weights, tanh recurrence, not a transformer. - The toy's dynamics dampen errors; real LLMs often amplify them. Untested there. - Full-trajectory backprop is expensive at real scale; the clipping constraint shrinks the search space only near-boundary weights matter but efficient scaling is unsolved. If you have compute and want to try trajectory-aware rounding on a real sub-1B model, or you know prior literature that already does this AdaRound optimizes rounding decisions per-layer; we're looking for whole-trajectory versions , please reach out in the thread or open an issue here.