arXiv:2610.01054v2 Announce Type: replace Abstract: In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work compresses ICL into fixed activation vectors extracted from specific layers or positions, but these input-independent interventions fail on complex tasks where the output depends on fine-grained interactions with the input. By analyzing the ICL forward pass, we show that each attention head's output is an affine transformation of its context-masked counterpart, and that the parameters of this transformation are empirically stable across samples for a given task. Building on this, we introduce Task Operator (TO), which replays this transformation as an analytically derived update to the attention output projection. Across lexical, algorithmic, and reasoning tasks, TO achieves the best overall performance among prior methods and substantially narrows the gap between zero-shot inference and ICL. We further show that the extracted knowledge concentrates in a task-specific sparse circuit across layers and positions, and that averaging operators from disjoint demonstration batches enables effective many-shot scaling without expanding the context window. Our code is available at https://github.com/gzxiong/task_operator.
Capturing In-Context Learning Dynamics with Task Operators
A paper posted to arXiv (2610.01054v2) introduces Task Operator (TO), a method that captures in-context learning as an analytically derived update to the attention output projection, letting language models replay a task's transformation without reprocessing the full demonstration set. The authors show each attention head's output is an affine transformation of its context-masked counterpart whose parameters are empirically stable across samples for a given task, and that TO achieves the best overall performance among prior methods across lexical, algorithmic and reasoning tasks, substantially narrowing the gap between zero-shot inference and ICL. The work also finds the extracted knowledge concentrates in a task-specific sparse circuit across layers and positions, and that averaging operators from disjoint demonstration batches enables many-shot scaling without expanding the context window; code is available at github.com/gzxiong/task_operator.
Replace repeated full-demonstration ICL passes with a Task Operator update to the attention output projection, and average operators from disjoint demonstration batches to scale to many shots without growing the context window.
more in playbook: Inference and self-hostingRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.