[Research] From Functional Geometry to Dynamic Grammar: New LIMEN Audits (V23–V24) Across 7 Architectures

Independent researcher LIMEN project finds that Transformer models follow a universal dynamic grammar of seven conserved state transition motifs, with ambiguity delaying decisional engagement in modern models like Phi-1.5 and Llama-3.2. The study across 7 open-source architectures reveals that hidden state trajectories are not random but follow a structured B→A→D funnel, suggesting auditability for hallucination detection and potential for dynamic steering beyond prompt engineering.

Hi everyone, I am sharing recent results from my independent research project, LIMEN Liminal Internal Metric for Emergent Navigation , which aims to characterize the internal dynamics of Transformers through hidden state analysis. Following our previous findings that functional information is encoded in the relative geometry of representations rather than individual neurons V22 , this new phase focuses on the impact of context ambiguity and the temporal structure of state transitions V23–V24 . Context & Methodology Model Panel: 7 open-source models GPT-2, DistilGPT2, OPT-125M, Qwen2.5-0.5B, TinyLlama-1.1B, Phi-1.5, Llama-3.2-1B . Approach: Layer-by-layer analysis of latent trajectories, linear probe decoding, and symbolic analysis of dynamic regimes. Philosophy: Strict empiricism. Clear distinction between observation, interpretation, and speculation. Code and data are available upon request. V23: The Impact of Ambiguity on Internal Dynamics The objective was to determine whether semantic ambiguity alters the model’s “cognitive trajectory.” Key Findings V23.2b : AMBIGUITY AFFECTS TRAJECTORY = YES: Ambiguity significantly modifies trajectory geometry curvature, cosine similarity . AMBIGUITY INCREASES INSTABILITY = NO: Counter-intuitively, ambiguity does not increase global chaos. Instead, the model becomes geometrically more “cautious.” AMBIGUITY DELAYS COMMITMENT = PARTIAL: Modern models Phi-1.5, Llama-3.2 delay their decisional engagement when facing uncertainty, spending more time in exploration regimes. Architectural Signature: Phi-1.5 shows unique sensitivity, increasing its occupancy of the bifurcation regime D STATE under ambiguity, suggesting a distinct iterative reasoning mechanism compared to standard completion models. Related Preprint: Conditional Dynamic Signatures in Large Language Models V24: Discovery of a “Universal Dynamic Grammar” By shifting from continuous analysis to a symbolic analysis of state sequences, a striking structure emerged. Key Findings V24.1 : STATE GRAMMAR EXISTS = YES: Trajectories are not random. They follow strict transitional patterns. UNIVERSAL GRAMMAR = YES: Seven transition motifs are conserved across all tested architectures, notably: B→B Initial Hesitation/Exploration B→A Convergence toward stable processing A→A Maintenance of the adaptive regime – the primary attractor A→D Transition to final decision Funnel Structure: Typical dynamics follow an Exploration B → Stabilization/Processing A → Decision D schema. State A acts as a strong attractor 𝑃 𝐴 → 𝐴 ≈ 0.91 P A→A ≈0.91 . The Phi-1.5 Exception: Unlike other models that quickly converge to A, Phi-1.5 maintains complex B↔A oscillations throughout the depth, confirming its nature as a “reasoning” model rather than a simple statistical completer. Related Preprint: A Runtime Trajectory Dynamics Framework for Large Language Models updated Implications & Discussion These results suggest that Transformer “intelligence” is not just a matter of static weights, but of constrained geometric navigation. Auditability: A violation of this universal grammar e.g., a direct B→D jump without an A phase could be an early indicator of hallucination or reasoning errors. Control: Understanding these attractors opens the door to more precise dynamic steering than prompt engineering alone. Open Questions for the Community: Have you observed violations of this B→A→D grammar in cases of blatant hallucinations? How do these motifs evolve in very large models 70B where depth is significantly greater? Are there recent publications on the “symbolic dynamics” of hidden states that align with these findings? I welcome any methodological criticism, suggestions for additional controls, or collaboration. Best regards,