{"slug": "i-taught-myself-integer-only-neural-net-training-in-rust-and-ran-into-a-real", "title": "I taught myself integer-only neural net training in Rust — and ran into a real quantization problem", "summary": "A 10th-grade student built Green-AI, a Rust project exploring energy-efficient AI training using only integer arithmetic, and discovered a 'dead zone' problem where fixed-point weight updates stall when errors become small. The student developed an Adaptive Bitshift Learning (ABSL) fix that outperforms stochastic rounding in accuracy, achieving a final weight error of 5 versus 77 for standard fixed-point training.", "body_md": "I'm a 10th grade student teaching myself Rust by building something instead of just reading theory. The project is called Green-AI — an attempt to explore more energy-efficient AI training by using only integer arithmetic, no floating point anywhere except for timing measurements.\n\nTurns out this breaks in a genuinely interesting way, and fixing it taught me more about numerical precision than any tutorial did.\n\n**The problem: a \"dead zone\" in fixed-point training**\n\nA standard weight update in my integer neuron looks like this:\n\n``` php\nfn update(&self, error: i32, input: i32) -> i32 {\n    (((error as i64) * (input as i64)) >> 14) as i32\n}\n```\n\nThat >> 14 is doing the job floating-point division normally does — it scales the update down to a sane range. The problem: once the error gets small enough, (error * input) >> 14 rounds down to 0, even though the error isn't actually zero. The weights just... stop updating. Training silently stalls.\n\nI benchmarked this \"Standard\" approach over 1000 runs: it converges fast, but lands with a final weight error of 77 compared to the true target weights. It gets stuck in the dead zone before it ever gets close.\n\n**My fix: ABSL (Adaptive Bitshift Learning)**\n\nMy fix was to make the shift amount adaptive to the size of the error — so updates never round away to nothing:\n\n``` php\nfn update(&self, error: i32, input: i32) -> i32 {\n    let abs_error = error.abs();\n\n    let shift = match abs_error {\n        e if e > 15000 => 17,\n        e if e > 8000  => 16,\n        e if e > 4000  => 15,\n        e if e > 1500  => 14,\n        e if e > 500   => 12,\n        e if e > 200   => 10,\n        e if e > 80    => 8,\n        e if e > 20    => 7,\n        _ => 6,\n    };\n\n    (((error as i64) * (input as i64)) >> shift) as i32\n}\n```\n\nLarge errors get damped more (bigger shift, prevents overshoot), small errors get amplified more (smaller shift, prevents stalling).\n\nI also tried stochastic rounding as an alternative fix, which is closer to what's actually used in quantized-training resaerch. You probabilistically round the truncated fraction up or down instead of always down, so updates accumulate correctly on average.\n\n**Results**\n\nAlgorithm\n\nAvg time/run\n\nFinal weight error\n\nConverges after\n\nStandard (fixed shift)\n\n132.5 µs\n\n77\n\n1928 steps\n\nStochastic Rounding\n\n206.8 µs\n\n9\n\n846 steps\n\nAdaptiveShift (ABSL)\n\n143.6 µs\n\n5\n\n2230 steps\n\nABSL ends up the most accurate of the three, and noticeably faster per step than stochastic rounding. The trade-off: it takes more steps to fully settle, likely because it keeps making meaningful updates near the target instead of stalling out early — accuracy over speed-to-convergence.\n\n**What I'm still figuring out**\n\nThe shift thresholds (15000, 8000, 4000...) are hand-tuned magic numbers for this one neuron's weight scale. I don't yet know how to make them scale automatically to different layer sizes.\n\nEverything so far is tested on a single neuron, not a real multi-layer network with backprop.\n\nIs \"adaptive step size based on error magnitude\" a known technique outside of fixed-point contexts? It feels conceptually close to things like Rprop, but I got here from a completely different angle (avoiding integer truncation).\n\nRepo's here if you want to poke at the code: [https://github.com/Mojo0869/green-ai](https://github.com/Mojo0869/green-ai)\n\nFeedback very welcome, especially from anyone who's worked with quantized or fixed-point training before — I'd love to know what I'm missing.", "url": "https://wpnews.pro/news/i-taught-myself-integer-only-neural-net-training-in-rust-and-ran-into-a-real", "canonical_source": "https://dev.to/mojo0869/i-taught-myself-integer-only-neural-net-training-in-rust-and-ran-into-a-real-quantization-problem-bli", "published_at": "2026-07-28 03:16:28+00:00", "updated_at": "2026-07-28 03:32:01.810034+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research", "developer-tools"], "entities": ["Green-AI", "Rust", "ABSL"], "alternates": {"html": "https://wpnews.pro/news/i-taught-myself-integer-only-neural-net-training-in-rust-and-ran-into-a-real", "markdown": "https://wpnews.pro/news/i-taught-myself-integer-only-neural-net-training-in-rust-and-ran-into-a-real.md", "text": "https://wpnews.pro/news/i-taught-myself-integer-only-neural-net-training-in-rust-and-ran-into-a-real.txt", "jsonld": "https://wpnews.pro/news/i-taught-myself-integer-only-neural-net-training-in-rust-and-ran-into-a-real.jsonld"}}