Xilinx Kria KV260

A developer achieved 59,965 tokens per second running a 3.16M-parameter INT4 transformer entirely in the on-chip memory of a $250 AMD Xilinx Kria KV260 FPGA, with zero DRAM in the token loop. The bit-…

Mike Ayles' on-chip LLM project serves a large language model from a $250 FPGA (Xilinx Kria KV260) to the public internet via a Cloudflare Tunnel, achieving ~19,240 fabric tokens per second but only ~…