{"slug": "research-from-functional-geometry-to-dynamic-grammar-new-limen-audits-v23-v24-7", "title": "[Research] From Functional Geometry to Dynamic Grammar: New LIMEN Audits (V23–V24) Across 7 Architectures", "summary": "Independent researcher LIMEN project finds that Transformer models follow a universal dynamic grammar of seven conserved state transition motifs, with ambiguity delaying decisional engagement in modern models like Phi-1.5 and Llama-3.2. The study across 7 open-source architectures reveals that hidden state trajectories are not random but follow a structured B→A→D funnel, suggesting auditability for hallucination detection and potential for dynamic steering beyond prompt engineering.", "body_md": "Hi everyone,\n\nI am sharing recent results from my independent research project, LIMEN (Liminal Internal Metric for Emergent Navigation), which aims to characterize the internal dynamics of Transformers through hidden state analysis.\n\nFollowing our previous findings that functional information is encoded in the relative geometry of representations rather than individual neurons (V22), this new phase focuses on the impact of context (ambiguity) and the temporal structure of state transitions (V23–V24).\n\nContext & Methodology\n\nModel Panel: 7 open-source models (GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B, TinyLlama-1.1B, Phi-1.5, Llama-3.2-1B).\n\nApproach: Layer-by-layer analysis of latent trajectories, linear probe decoding, and symbolic analysis of dynamic regimes.\n\nPhilosophy: Strict empiricism. Clear distinction between observation, interpretation, and speculation. Code and data are available upon request.\n\nV23: The Impact of Ambiguity on Internal Dynamics\n\nThe objective was to determine whether semantic ambiguity alters the model’s “cognitive trajectory.”\n\nKey Findings (V23.2b):\n\nAMBIGUITY_AFFECTS_TRAJECTORY = YES: Ambiguity significantly modifies trajectory geometry (curvature, cosine similarity).\n\nAMBIGUITY_INCREASES_INSTABILITY = NO: Counter-intuitively, ambiguity does not increase global chaos. Instead, the model becomes geometrically more “cautious.”\n\nAMBIGUITY_DELAYS_COMMITMENT = PARTIAL: Modern models (Phi-1.5, Llama-3.2) delay their decisional engagement when facing uncertainty, spending more time in exploration regimes.\n\nArchitectural Signature: Phi-1.5 shows unique sensitivity, increasing its occupancy of the bifurcation regime (D_STATE) under ambiguity, suggesting a distinct iterative reasoning mechanism compared to standard completion models.\n\nRelated Preprint: Conditional Dynamic Signatures in Large Language Models\n\nV24: Discovery of a “Universal Dynamic Grammar”\n\nBy shifting from continuous analysis to a symbolic analysis of state sequences, a striking structure emerged.\n\nKey Findings (V24.1):\n\nSTATE_GRAMMAR_EXISTS = YES: Trajectories are not random. They follow strict transitional patterns.\n\nUNIVERSAL_GRAMMAR = YES: Seven transition motifs are conserved across all tested architectures, notably:\n\nB→B (Initial Hesitation/Exploration)\n\nB→A (Convergence toward stable processing)\n\nA→A (Maintenance of the adaptive regime – the primary attractor)\n\nA→D (Transition to final decision)\n\nFunnel Structure: Typical dynamics follow an Exploration (B) → Stabilization/Processing (A) → Decision (D) schema. State A acts as a strong attractor (\n\n𝑃\n\n(\n\n𝐴\n\n→\n\n𝐴\n\n)\n\n≈\n\n0.91\n\nP(A→A)≈0.91).\n\nThe Phi-1.5 Exception: Unlike other models that quickly converge to A, Phi-1.5 maintains complex B↔A oscillations throughout the depth, confirming its nature as a “reasoning” model rather than a simple statistical completer.\n\nRelated Preprint: A Runtime Trajectory Dynamics Framework for Large Language Models (updated)\n\nImplications & Discussion\n\nThese results suggest that Transformer “intelligence” is not just a matter of static weights, but of constrained geometric navigation.\n\nAuditability: A violation of this universal grammar (e.g., a direct B→D jump without an A phase) could be an early indicator of hallucination or reasoning errors.\n\nControl: Understanding these attractors opens the door to more precise dynamic steering than prompt engineering alone.\n\nOpen Questions for the Community:\n\nHave you observed violations of this B→A→D grammar in cases of blatant hallucinations?\n\nHow do these motifs evolve in very large models (>70B) where depth is significantly greater?\n\nAre there recent publications on the “symbolic dynamics” of hidden states that align with these findings?\n\nI welcome any methodological criticism, suggestions for additional controls, or collaboration.\n\nBest regards,", "url": "https://wpnews.pro/news/research-from-functional-geometry-to-dynamic-grammar-new-limen-audits-v23-v24-7", "canonical_source": "https://discuss.huggingface.co/t/research-from-functional-geometry-to-dynamic-grammar-new-limen-audits-v23-v24-across-7-architectures/177260#post_1", "published_at": "2026-06-30 00:26:32+00:00", "updated_at": "2026-06-30 00:28:12.678314+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "machine-learning", "ai-research", "ai-safety"], "entities": ["GPT-2", "DistilGPT2", "OPT-125M", "Qwen2.5-0.5B", "TinyLlama-1.1B", "Phi-1.5", "Llama-3.2-1B"], "alternates": {"html": "https://wpnews.pro/news/research-from-functional-geometry-to-dynamic-grammar-new-limen-audits-v23-v24-7", "markdown": "https://wpnews.pro/news/research-from-functional-geometry-to-dynamic-grammar-new-limen-audits-v23-v24-7.md", "text": "https://wpnews.pro/news/research-from-functional-geometry-to-dynamic-grammar-new-limen-audits-v23-v24-7.txt", "jsonld": "https://wpnews.pro/news/research-from-functional-geometry-to-dynamic-grammar-new-limen-audits-v23-v24-7.jsonld"}}