# Prototype of LIMEN Runtime Audit — feedback welcome

> Source: <https://discuss.huggingface.co/t/prototype-of-limen-runtime-audit-feedback-welcome/178185#post_1>
> Published: 2026-07-24 23:06:44+00:00

Hi everyone,

I have just published the first public prototype of **LIMEN Runtime Audit**, a small open-source toolkit for inspecting layer-wise activation trajectories in open-weight language models.

GitHub repository:

This is my first public GitHub repository, so feedback on the code, methodology, terminology and documentation would be genuinely valuable.

Most LLM evaluations focus on the final output:

LIMEN adds another descriptive layer: it examines how the model’s hidden activation vector changes as information passes through successive layers.

For each token, the toolkit treats the sequence of hidden states across layers as a measurable trajectory.

The objective is not to claim that these trajectories directly represent reasoning, semantic concepts or cognitive states. The objective is to provide a reproducible observability layer for comparing internal runtime behaviour.

Version 0.1 accepts exported NumPy arrays with:

`hidden_states`

: `[tokens, layers, hidden_dim]`

`logits`

: `[tokens, vocabulary]`

It currently measures:

| Metric | Plain-language interpretation |
|---|---|
| Path length | Total movement across layers |
| Displacement | Direct distance between the first and final layer |
| Tortuosity | How indirect the layer-wise route is |
| Mean speed | Average change between adjacent layers |
| Speed variability | How regularly or irregularly the representation changes |
| Mean acceleration | How much the layer-to-layer movement itself changes |
| Turning angle | How much the direction of movement changes |
| Entropy | Uncertainty in the output distribution |
| Top-1/top-2 margin | Separation between the two leading token probabilities |

The toolkit also:

With proper validation and controls, this kind of instrumentation could potentially help with:

These are possible applications, not validated product claims.

This prototype is currently a **descriptive audit and observability tool**.

Its measurements do not, by themselves, establish:

A geometrical difference is an observation. Explaining what that difference means requires separate experiments, controls and held-out validation.

The next important milestone is therefore predictive validation: testing whether LIMEN trajectory features detect or predict held-out behavioural regressions beyond conventional baselines and shuffled controls.

The project grew from an independent empirical investigation of runtime activation dynamics. Earlier experiments used broader dynamical-systems terminology. As controls accumulated, I decided that the public engineering implementation should use narrower and more cautious language.

Instead of waiting for every research question to be resolved, I am releasing the minimal audit layer so that other people can inspect it, challenge it and potentially reproduce or improve it.

The repository is released under the Apache 2.0 licence.

GitHub:

Thanks in advance for any critical feedback, failed reproductions, issues or pull requests. At this stage, a rigorous objection is more useful to me than enthusiastic agreement.