A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt
A new theoretical paper asks how much an autoregressive transformer retains of its prompt after generating t tokens, and reports a conditional no-hair theorem: when two different prompts are followed by the same continua…