# From SkillCheck to TraceMantle: Tracking Agent Skill Changes and Test Evidence

> Source: <https://dev.to/moonrunnerkc/from-skillcheck-to-tracemantle-tracking-agent-skill-changes-and-test-evidence-3enh>
> Published: 2026-09-13 03:22:03+00:00

I’ve renamed SkillCheck to TraceMantle and expanded its scope. The original project focused mainly on validating AI agent skill files. TraceMantle retains that functionality and adds support for tracking the files in a skill package, comparing changes, and checking whether previous test evidence still applies.

The rename resolves a conflict with another project using the SkillCheck name. It also better reflects the tool’s broader purpose: helping developers assess both the contents of an agent skill and the evidence used to approve a particular version. The project is available on [GitHub](https://github.com/moonrunnerkc/tracemantle).

An agent skill can include more than its main `SKILL.md` instruction file. Supporting scripts, reference documents, templates, and schemas can all affect how it behaves.

Consider a skill that uses a Python helper to process a document. The instructions remain unchanged, but someone modifies the helper. A passing test report from the previous version may no longer cover the current behavior. Checking only the instruction file would miss an important part of that change.

TraceMantle addresses this by connecting package changes with the evidence supplied for its checks. The aim is to make it easier to identify which earlier results remain relevant and which checks need to run again.

Skill validation remains part of the tool. TraceMantle checks frontmatter, file references, size limits, and compatibility advice. These checks help identify structural problems before a skill is committed or released.

The expanded functionality adds three related capabilities:

`SKILL.md` stays the same.
The commands support text, JSON, and GitHub annotation output, allowing the same information to be used during local development or within a CI workflow.

An imported report does not become trusted simply because it contains a passing result. The project owner or trusted automation must approve the evidence through a policy that identifies the required checks and their inputs.

TraceMantle reads that policy from a specific trusted Git revision. Changes in the candidate package cannot replace the selected policy or quietly make a required check optional.

The comparison also distinguishes a failed check from missing or unsuitable evidence. An `unknown` result means the available evidence does not establish a pass. That distinction helps avoid treating an incomplete evaluation as either a successful release check or a confirmed failure of the skill itself.

The [evidence workflow documentation](https://github.com/moonrunnerkc/tracemantle/blob/main/docs/evidence-workflow.md) explains the supported format, approval process, and comparison states.

The work also included improvements to parsing, configuration, reporting, and history storage. Analysis stages share a parsed document model to reduce repeated work, while file-size and parser limits help control resource use when processing external input.

New history records are stored individually outside the evaluated bundle. Migration from the older history format is explicit and preserves the original files. The release workflow also checks built packages through clean installations, alongside tests, linting, and strict type checking.

These changes support the broader workflow while keeping TraceMantle a local Python CLI and library.

TraceMantle analyzes files and supplied evidence. It does not run agent skills or evaluators itself, and a structurally valid skill is not proof that an agent will perform a task correctly.

Its dependency analysis also has defined limits. It cannot infer every runtime dependency or arbitrary programming-language import. Trusted checks still need accurate input declarations, and incomplete coverage should require further evaluation.

Live agent performance remains unverified in the project’s current evaluation record. Static checks, imported model judgments, and synthetic test fixtures have separate meanings and should remain distinguishable.

TraceMantle 1.6.0 is published, with subsequent changes recorded under [Unreleased](https://github.com/moonrunnerkc/tracemantle/blob/main/CHANGELOG.md#unreleased). A recent audit identified two remaining defects involving Markdown dependency detection and numeric JSON input handling. One can cause a required rerun to be missed; the other can crash an import. Those corrections remain necessary before calling the implementation complete.

Existing SkillCheck users should follow the [migration guide](https://github.com/moonrunnerkc/tracemantle/blob/main/docs/migration.md). TraceMantle is a separate Python distribution, so installing it is not an automatic upgrade of the old package.

The [repository](https://github.com/moonrunnerkc/tracemantle) contains the source, examples, tests, and implementation records for anyone interested in reviewing or contributing to the project.
