cd /news/artificial-intelligence/high-dimensional-layer-state-traject… · home › topics › artificial-intelligence › article
[ARTICLE · art-154255] src=arxiv.org ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

High-dimensional layer-state trajectories reveal the internal dynamics of transformer inference

A study posted as arXiv:2610.26866v1 reports that layer order in transformer inference defines a measurable computation: across representative Qwen, Llama and Gemma checkpoints, realized layer order produced hidden-state paths 1.60–2.24 times as aligned with their endpoint chord as endpoint-preserving shuffles. The order effect transferred to a lexical task and was stronger in three random initializations of the same Qwen architecture, which the authors say identifies an architecture-compatible scaffold rather than a training-specific law. A confidence gate drove 17 of 87 held-out Qwen closure trajectories across the full-vocabulary top-1 candidate boundary while leaving all 203 non-closure top-1 outputs unchanged, though the authors state the work does not establish a complete theory of LLM dynamics or control of answer correctness.

by read1 min views1 publishedOct 21, 2026

arXiv:2610.26866v1 Announce Type: new Abstract: Mechanistic studies of large language models localize function in parameters, attention heads or neurons, but those parts do not specify how one inference unfolds. We treat the ordered layer-state trajectory as the empirical process and test whether layer order defines a measurable computation. Across representative Qwen, Llama and Gemma checkpoints, realized order produced paths 1.60--2.24 times as aligned with their endpoint chord as endpoint-preserving shuffles. The order effect transferred to a lexical task, but was stronger in three random initializations of the same Qwen architecture, identifying an architecture-compatible scaffold rather than a training-specific law. Direct hidden states provided stronger mechanism coordinates than vocabulary readbacks, and a task-defined coordinate made candidate competition computable. After additive interventions failed, local transition actuators selectively controlled internal trajectories across checkpoints. A confidence gate then drove 17 of 87 held-out Qwen closure trajectories across the full-vocabulary top-1 candidate boundary while leaving all 203 non-closure top-1 outputs unchanged. Safety-matched action reassignment performed similarly, locating the gain in selective gating and the bounded actuator family rather than fine-grained dose assignment. Layer order exposes an observable, computable and behaviourally consequential process of transformer inference without establishing a complete theory of LLM dynamics or control of answer correctness.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @qwen 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/high-dimensional-lay…] indexed:0 read:1min 2026-10-21 · —