04:00
2026-07-07
arxiv.org
machine-learning
Induction Heads Interpolate N-Grams
Researchers found that induction heads in transformers implement two smoothing mechanisms—soft context-matching and BOS token pseudo-counts—that interpolate n-gram estimates, enabling in-context learn…