Prototype of LIMEN Runtime Audit — feedback welcome

A developer has released the first public prototype of LIMEN Runtime Audit, an open-source toolkit for inspecting layer-wise activation trajectories in open-weight language models. The toolkit measures metrics such as path length, displacement, tortuosity, and entropy to provide a descriptive observability layer for comparing internal runtime behavior. The project, hosted on GitHub under the Apache 2.0 license, is currently a descriptive audit tool and does not establish causal claims about reasoning or semantic content.

Hi everyone, I have just published the first public prototype of LIMEN Runtime Audit , a small open-source toolkit for inspecting layer-wise activation trajectories in open-weight language models. GitHub repository: This is my first public GitHub repository, so feedback on the code, methodology, terminology and documentation would be genuinely valuable. Most LLM evaluations focus on the final output: LIMEN adds another descriptive layer: it examines how the model’s hidden activation vector changes as information passes through successive layers. For each token, the toolkit treats the sequence of hidden states across layers as a measurable trajectory. The objective is not to claim that these trajectories directly represent reasoning, semantic concepts or cognitive states. The objective is to provide a reproducible observability layer for comparing internal runtime behaviour. Version 0.1 accepts exported NumPy arrays with: hidden states : tokens, layers, hidden dim logits : tokens, vocabulary It currently measures: | Metric | Plain-language interpretation | |---|---| | Path length | Total movement across layers | | Displacement | Direct distance between the first and final layer | | Tortuosity | How indirect the layer-wise route is | | Mean speed | Average change between adjacent layers | | Speed variability | How regularly or irregularly the representation changes | | Mean acceleration | How much the layer-to-layer movement itself changes | | Turning angle | How much the direction of movement changes | | Entropy | Uncertainty in the output distribution | | Top-1/top-2 margin | Separation between the two leading token probabilities | The toolkit also: With proper validation and controls, this kind of instrumentation could potentially help with: These are possible applications, not validated product claims. This prototype is currently a descriptive audit and observability tool . Its measurements do not, by themselves, establish: A geometrical difference is an observation. Explaining what that difference means requires separate experiments, controls and held-out validation. The next important milestone is therefore predictive validation: testing whether LIMEN trajectory features detect or predict held-out behavioural regressions beyond conventional baselines and shuffled controls. The project grew from an independent empirical investigation of runtime activation dynamics. Earlier experiments used broader dynamical-systems terminology. As controls accumulated, I decided that the public engineering implementation should use narrower and more cautious language. Instead of waiting for every research question to be resolved, I am releasing the minimal audit layer so that other people can inspect it, challenge it and potentially reproduce or improve it. The repository is released under the Apache 2.0 licence. GitHub: Thanks in advance for any critical feedback, failed reproductions, issues or pull requests. At this stage, a rigorous objection is more useful to me than enthusiastic agreement.