{"slug": "mapping-hidden-state-attractors-in-tinyllama-building-a-runtime-map-of-llm", "title": "Mapping Hidden-State Attractors in TinyLlama: Building a Runtime Map of LLM Dynamics", "summary": "A researcher has built an Attractor Map for TinyLlama that tracks the trajectory of hidden states during text generation, revealing that model trajectories repeatedly visit a limited number of dynamical regions rather than wandering randomly. The framework clusters runtime metrics such as hidden-state norm, cosine similarity, curvature, and output entropy to create a runtime map, but experiments with trajectory interventions (SRA-X) showed that changing the trajectory did not reliably improve reasoning, leading to the conclusion that runtime stability does not imply semantic correctness.", "body_md": "Over the past few months I’ve been working on an experimental framework\n\nThe initial idea was simple:\n\nInstead of looking only at tokens, what if we observed the trajectory of hidden states during generation?\n\nThis led me to build what I currently call an Attractor Map.\n\nThe goal is not to explain semantics directly.\n\nThe goal is to build a runtime map describing where the model is moving during inference.\n\nWhy build an attractor map?\n\nMost interpretability work focuses on:\n\nI wanted to look at something different:\n\nthe geometry of generation itself.\n\nInstead of asking:\n\n*Which neuron represents X?*\n\nI ask:\n\nWhat region of hidden-state space is the model currently visiting?\n\nRuntime features\n\nFor every generated token I extract several measurements from the last hidden state.\n\nCurrent runtime features include:\n\n| Feature | Description |\n|---|---|\n| Hidden-state norm | Magnitude of the representation |\n| Cosine similarity | Local directional continuity |\n| Curvature | Change of trajectory between consecutive steps |\n| Output entropy | Decoder uncertainty |\n| Transition count | Dynamical regime changes |\n| Hidden-state vector | Complete latent representation |\n\nEach token therefore becomes a point in a multidimensional dynamical space.\n\nBuilding the attractor map\n\nInstead of clustering raw hidden states directly, the framework clusters runtime dynamical signatures.\n\nConceptually:\n\nHidden States\n\n│\n\n▼\n\nRuntime Metrics\n\n(norm, entropy, curvature,\n\ncosine similarity, …)\n\n│\n\n▼\n\nFeature Space\n\n│\n\n▼\n\nClustering\n\n│\n\n▼\n\nRuntime Attractor Map\n\nThe objective is to identify recurrent regions of the trajectory visited during generation.\n\nThese regions are currently treated as dynamical clusters, not proven cognitive states.\n\nWhat the map revealed\n\nAcross repeated generations, trajectories were not uniformly distributed.\n\nInstead they repeatedly visited a limited number of regions.\n\nA simplified view looks like this:\n\n```\n            Exploration\n                ●\n             ↗     ↘\n```\n\nStable ●──────────────● Oscillation\n\n```\n             ↘     ↗\n              ●\n```\n\nThe exact geometry depends on the model and clustering parameters.\n\nThe important observation is that trajectories repeatedly revisit similar regions rather than wandering randomly.\n\nRuntime transitions\n\nGeneration can then be represented as a sequence of transitions.\n\nExample:\n\nStart\n\n↓\n\nRegion A\n\n↓\n\nRegion B\n\n↓\n\nRegion B\n\n↓\n\nRegion C\n\n↓\n\nRegion B\n\n↓\n\nEnd\n\n```\nInstead of analyzing isolated hidden states, the framework analyzes the **trajectory** itself.\n\n Segmentation\n\nLater versions introduced trajectory segmentation.\n\nRather than assuming fixed reasoning stages, the framework searches for changes in runtime dynamics.\n\nExample output:\n\nSegments detected: 13\n\nSegment 1 : tokens 0–5\n\nSegment 2 : tokens 5–11\n\nSegment 3 : tokens 11–16\n\nSegment 13 : tokens 74–80\n```\n\nThese segments appear automatically from trajectory statistics.\n\nWhether they correspond to reusable computational operations remains an open question.\n\nRuntime interventions (SRA-X)\n\nOnce the attractor map existed, the obvious next question became:\n\nCan we modify the trajectory?\n\nSeveral experimental intervention strategies were explored:\n\nThe interesting part is that the trajectory does change after intervention.\n\nHowever…\n\nNegative results (probably the most important)\n\nChanging the trajectory was not sufficient to reliably improve reasoning.\n\nRepeated experiments showed that:\n\nwhile the final answer can still be wrong.\n\nOne of the strongest conclusions from this project became:\n\nRuntime stability does not imply semantic correctness.\n\nThis completely changed the direction of the research.\n\nToday I view the attractor map as a runtime observability tool, not as a proof that the model contains literal “thinking states.”\n\nThe map provides a way to describe:\n\nControl remains an open problem.\n\nObservation turned out to be much easier than intervention.\n\nLLM\n\n↓\n\nHidden States\n\n↓\n\nRuntime Metrics\n\n↓\n\nAttractor Map\n\n↓\n\nTrajectory Analysis\n\n↓\n\nSegmentation\n\n↓\n\n(Optional) Runtime Intervention\n\n```\n---\n\n# Why I think this is interesting\n\nEven if runtime control ultimately fails, I think there is value in having a reproducible way to observe hidden-state dynamics while a model is generating.\n\nThe attractor map is my attempt to move from:\n\n*\"What token comes next?\"*\n\ntowards\n\n*\"How is the model moving internally while deciding the next token?\"*\n\n---\n\nI'm currently extending this work to additional architectures and larger models.\n\nI'd genuinely appreciate feedback from people working on:\n\n* mechanistic interpretability\n* dynamical systems\n* representation learning\n* hidden-state analysis\n* runtime observability\n\nI'm especially interested in criticism of the methodology before scaling the experiments further.\n```\n\n", "url": "https://wpnews.pro/news/mapping-hidden-state-attractors-in-tinyllama-building-a-runtime-map-of-llm", "canonical_source": "https://discuss.huggingface.co/t/mapping-hidden-state-attractors-in-tinyllama-building-a-runtime-map-of-llm-dynamics/178161#post_1", "published_at": "2026-07-23 16:47:30+00:00", "updated_at": "2026-07-23 17:05:43.807318+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-tools", "ai-safety"], "entities": ["TinyLlama", "Attractor Map", "SRA-X"], "alternates": {"html": "https://wpnews.pro/news/mapping-hidden-state-attractors-in-tinyllama-building-a-runtime-map-of-llm", "markdown": "https://wpnews.pro/news/mapping-hidden-state-attractors-in-tinyllama-building-a-runtime-map-of-llm.md", "text": "https://wpnews.pro/news/mapping-hidden-state-attractors-in-tinyllama-building-a-runtime-map-of-llm.txt", "jsonld": "https://wpnews.pro/news/mapping-hidden-state-attractors-in-tinyllama-building-a-runtime-map-of-llm.jsonld"}}