BITNET – an autoregressive language model on the 6502 processor

Matt Beton trained a tiny Mamba-based autoregressive language model and built an inference engine to run it on the MOS 6502, an 8-bit processor from 1975 with 32KB RAM, which powered the BBC Micro and Apple II. The project demonstrates that modern language models can operate on extremely constrained hardware.