22:05
2026-07-10
lesswrong.com
machine-learning
An Induction Head in Disguise: Chasing Grammar in a Character-Level Transformer
A researcher trained a character-level transformer on Nietzsche texts and identified a copying head (B5H0) using eigenvalue analysis, but found its QK circuit insufficient for bracket completion. The …