00:00
2026-08-09
mikeayles.com
artificial-intelligence
On-Chip LLM: Sampling Without Asking
An FPGA-based on-chip LLM decoded replies in 6.6ms, but browser delivery took 100ms; profiling revealed that 58% of the delay came from a host-side loop reading 193 logits per token over /dev/mem. Theβ¦