Hi everyone,
This has been a really interesting discussion. I’ve been exploring the same topic over the past few months, so I thought these personal experiments might be relevant here.
The common theme is the distinction between integer storage, quantized kernels, and a model whose compute path remains entirely in integer arithmetic.
A deliberately wide Q16.48 reference implementation for a tiny character GPT and TinyLlama-1.1B. The tiny GPT trains and samples entirely in fixed-point integer arithmetic, while TinyLlama runs through an integer inference core. Conversion and reference checks remain boundary operations. The goal is a deterministic reference implementation for experimentation rather than a fast or production-ready runtime.
Explores how far stored weight precision can be reduced while using the integer reference implementation for regression checks. It uses signed-int16 F11/F12 weight representations, includes physical MCU measurements, and reports a mixed F11/F12 TinyLlama candidate that matched all 632 evaluated greedy decisions. The strongest results are storage and memory reduction rather than general speed improvements.
A related but dtype-independent experiment exploring reversible permutations of vocabulary rows, MLP neurons, GQA heads, and the residual hidden basis. The integer runtime served as an exact experimentation and validation environment before applying the same transformations directly to BF16/F16/F32 SafeTensors. Floating-point drift is reported separately.
The remaining blog post and all GitHub repositories are available from my Hugging Face profile.
These aren’t products or frameworks—just a few personal experiments that I thought might be relevant to this discussion.
Everything is open source, including the implementation, validation records, limitations, and negative results. If anyone finds them interesting, or has suggestions or pointers to related work, I’d be very happy to hear them.