FreeToken : An LLM Engine to max the bandwidth of All-The-Things

FreeToken, an LLM engine designed to maximize bandwidth across all hardware, faces a fundamental challenge: bit-exact reproducibility is impossible across different CPU and GPU architectures due to differing floating-point arithmetic paths. The source notes that even identical software environments will produce different results on Intel versus AMD CPUs or on CUDA SM86, SM89, and SM120 GPUs, reflecting trade-offs between performance and precision. To validate cross-device consistency, the source suggests capturing logits as reference and testing them across hardware with different floating-point accumulators and matrix-multiplication extensions.

Bit-exact… to what reference? The arithmetic paths of a CPU and GPU are different. Intel or AMD CPU? SM86, SM89, SM120 CUDA? Even a perfectly pinned software environment will differ by underlying hardware. That is the nature of floating point compute, and the many trade offs made between performance and precision. See if you can capture the logits somehow for reference and we can run them against different devices with different hardware floating point accumulators and mma extensions.