Mapping Hidden-State Attractors in TinyLlama: Building a Runtime Map of LLM Dynamics

A researcher has built an Attractor Map for TinyLlama that tracks the trajectory of hidden states during text generation, revealing that model trajectories repeatedly visit a limited number of dynamical regions rather than wandering randomly. The framework clusters runtime metrics such as hidden-state norm, cosine similarity, curvature, and output entropy to create a runtime map, but experiments with trajectory interventions (SRA-X) showed that changing the trajectory did not reliably improve reasoning, leading to the conclusion that runtime stability does not imply semantic correctness.

Over the past few months I’ve been working on an experimental framework The initial idea was simple: Instead of looking only at tokens, what if we observed the trajectory of hidden states during generation? This led me to build what I currently call an Attractor Map. The goal is not to explain semantics directly. The goal is to build a runtime map describing where the model is moving during inference. Why build an attractor map? Most interpretability work focuses on: I wanted to look at something different: the geometry of generation itself. Instead of asking: Which neuron represents X? I ask: What region of hidden-state space is the model currently visiting? Runtime features For every generated token I extract several measurements from the last hidden state. Current runtime features include: | Feature | Description | |---|---| | Hidden-state norm | Magnitude of the representation | | Cosine similarity | Local directional continuity | | Curvature | Change of trajectory between consecutive steps | | Output entropy | Decoder uncertainty | | Transition count | Dynamical regime changes | | Hidden-state vector | Complete latent representation | Each token therefore becomes a point in a multidimensional dynamical space. Building the attractor map Instead of clustering raw hidden states directly, the framework clusters runtime dynamical signatures. Conceptually: Hidden States │ ▼ Runtime Metrics norm, entropy, curvature, cosine similarity, … │ ▼ Feature Space │ ▼ Clustering │ ▼ Runtime Attractor Map The objective is to identify recurrent regions of the trajectory visited during generation. These regions are currently treated as dynamical clusters, not proven cognitive states. What the map revealed Across repeated generations, trajectories were not uniformly distributed. Instead they repeatedly visited a limited number of regions. A simplified view looks like this: Exploration ● ↗ ↘ Stable ●──────────────● Oscillation ↘ ↗ ● The exact geometry depends on the model and clustering parameters. The important observation is that trajectories repeatedly revisit similar regions rather than wandering randomly. Runtime transitions Generation can then be represented as a sequence of transitions. Example: Start ↓ Region A ↓ Region B ↓ Region B ↓ Region C ↓ Region B ↓ End Instead of analyzing isolated hidden states, the framework analyzes the trajectory itself. Segmentation Later versions introduced trajectory segmentation. Rather than assuming fixed reasoning stages, the framework searches for changes in runtime dynamics. Example output: Segments detected: 13 Segment 1 : tokens 0–5 Segment 2 : tokens 5–11 Segment 3 : tokens 11–16 Segment 13 : tokens 74–80 These segments appear automatically from trajectory statistics. Whether they correspond to reusable computational operations remains an open question. Runtime interventions SRA-X Once the attractor map existed, the obvious next question became: Can we modify the trajectory? Several experimental intervention strategies were explored: The interesting part is that the trajectory does change after intervention. However… Negative results probably the most important Changing the trajectory was not sufficient to reliably improve reasoning. Repeated experiments showed that: while the final answer can still be wrong. One of the strongest conclusions from this project became: Runtime stability does not imply semantic correctness. This completely changed the direction of the research. Today I view the attractor map as a runtime observability tool, not as a proof that the model contains literal “thinking states.” The map provides a way to describe: Control remains an open problem. Observation turned out to be much easier than intervention. LLM ↓ Hidden States ↓ Runtime Metrics ↓ Attractor Map ↓ Trajectory Analysis ↓ Segmentation ↓ Optional Runtime Intervention --- Why I think this is interesting Even if runtime control ultimately fails, I think there is value in having a reproducible way to observe hidden-state dynamics while a model is generating. The attractor map is my attempt to move from: "What token comes next?" towards "How is the model moving internally while deciding the next token?" --- I'm currently extending this work to additional architectures and larger models. I'd genuinely appreciate feedback from people working on: mechanistic interpretability dynamical systems representation learning hidden-state analysis runtime observability I'm especially interested in criticism of the methodology before scaling the experiments further.