{"slug": "high-dimensional-layer-state-trajectories-reveal-the-internal-dynamics-of", "title": "High-dimensional layer-state trajectories reveal the internal dynamics of transformer inference", "summary": "A study posted as arXiv:2610.26866v1 reports that layer order in transformer inference defines a measurable computation: across representative Qwen, Llama and Gemma checkpoints, realized layer order produced hidden-state paths 1.60–2.24 times as aligned with their endpoint chord as endpoint-preserving shuffles. The order effect transferred to a lexical task and was stronger in three random initializations of the same Qwen architecture, which the authors say identifies an architecture-compatible scaffold rather than a training-specific law. A confidence gate drove 17 of 87 held-out Qwen closure trajectories across the full-vocabulary top-1 candidate boundary while leaving all 203 non-closure top-1 outputs unchanged, though the authors state the work does not establish a complete theory of LLM dynamics or control of answer correctness.", "body_md": "arXiv:2610.26866v1 Announce Type: new \nAbstract: Mechanistic studies of large language models localize function in parameters, attention heads or neurons, but those parts do not specify how one inference unfolds. We treat the ordered layer-state trajectory as the empirical process and test whether layer order defines a measurable computation. Across representative Qwen, Llama and Gemma checkpoints, realized order produced paths 1.60--2.24 times as aligned with their endpoint chord as endpoint-preserving shuffles. The order effect transferred to a lexical task, but was stronger in three random initializations of the same Qwen architecture, identifying an architecture-compatible scaffold rather than a training-specific law. Direct hidden states provided stronger mechanism coordinates than vocabulary readbacks, and a task-defined coordinate made candidate competition computable. After additive interventions failed, local transition actuators selectively controlled internal trajectories across checkpoints. A confidence gate then drove 17 of 87 held-out Qwen closure trajectories across the full-vocabulary top-1 candidate boundary while leaving all 203 non-closure top-1 outputs unchanged. Safety-matched action reassignment performed similarly, locating the gain in selective gating and the bounded actuator family rather than fine-grained dose assignment. Layer order exposes an observable, computable and behaviourally consequential process of transformer inference without establishing a complete theory of LLM dynamics or control of answer correctness.", "url": "https://wpnews.pro/news/high-dimensional-layer-state-trajectories-reveal-the-internal-dynamics-of", "canonical_source": "https://arxiv.org/abs/2610.26866", "published_at": "2026-10-21 04:00:00+00:00", "updated_at": "2026-10-21 04:21:03.766939+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research", "neural-networks"], "entities": ["Qwen", "Llama", "Gemma", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/high-dimensional-layer-state-trajectories-reveal-the-internal-dynamics-of", "markdown": "https://wpnews.pro/news/high-dimensional-layer-state-trajectories-reveal-the-internal-dynamics-of.md", "text": "https://wpnews.pro/news/high-dimensional-layer-state-trajectories-reveal-the-internal-dynamics-of.txt", "jsonld": "https://wpnews.pro/news/high-dimensional-layer-state-trajectories-reveal-the-internal-dynamics-of.jsonld"}}