When a Zero-Parameter Cache Overtakes a Transformer
An engineer's experiment found that a zero-parameter count table over the current document overtakes a 1.43M-parameter transformer with a fixed 64-token window as documents grow longer. At L=1000 tokens the cache reached…