# Build a Tested Agent Skill with SKILL.md and Python Scripts

> Source: <https://dev.to/paladini/build-a-tested-agent-skill-with-skillmd-and-python-scripts-3777>
> Published: 2026-08-30 12:34:17+00:00

AI agents are good at interpreting goals, but prose instructions are a weak place to enforce exact rules. If a skill says "keep the commit subject short" or "never commit without approval," an agent can still misunderstand the boundary.

The open-source [how-to-create-a-skill-tutorial](https://github.com/paladini/how-to-create-a-skill-tutorial) shows a practical split: let the agent make judgments, and let small local scripts validate repeatable rules. This tutorial builds the smallest useful version of that pattern: a `commit-crafter`

skill with a `SKILL.md`

file, a Python validator, and tests that run with the Python standard library.

An Agent Skill is a directory containing at least `SKILL.md`

. Put the workflow and safety boundaries in that file. Put exact validation in a script. Keep the script deterministic, return meaningful exit codes, and run it before presenting the result to a user.

The finished repository's example skill validates Conventional Commit messages. You can copy the same structure for release notes, config generation, research reports, or any other workflow with rules that can be checked mechanically.

You need:

The project has no stable release tag at the time of writing. The examples and commands below are checked against the current `main`

branch. Read the [Agent Skills specification](https://agentskills.io/specification) if your client uses a different discovery directory.

The repository documents two useful scopes. A personal skill belongs in your user skills directory. A project skill belongs in the repository so a team can review and install it with the project.

```
mkdir -p .agents/skills/commit-crafter/scripts
mkdir -p .agents/skills/commit-crafter/references
```

The required layout is simple:

```
commit-crafter/
|-- SKILL.md
|-- scripts/
|   `-- check_message.py
`-- references/
    |-- conventional-commits.md
    `-- examples.md
```

Only `SKILL.md`

is required. The `scripts/`

, `references/`

, and `assets/`

directories are conventions for executable code, on-demand documentation, and reusable resources.

The frontmatter is the discovery contract. The `name`

must match the directory name and use lowercase letters, numbers, and hyphens. The `description`

should explain both the capability and the user vocabulary that should trigger it.

```
---
name: commit-crafter
description: Write git commit messages from staged changes following Conventional Commits. Use when the user asks to commit, asks for a commit message, wants to fix or improve a commit message, or mentions conventional commits.
---

# Commit Crafter

Write a commit message for the staged changes. The user reviews and approves
the message before anything is committed.

## Workflow

1. Run `git status` and `git diff --staged`.
2. If nothing is staged, report the changed files and ask what to include.
3. Classify the change and draft an imperative Conventional Commit message.
4. Run `python scripts/check_message.py --file message.txt`.
5. Fix violations and run the validator again.
6. Show the message. Commit only after explicit approval.

## Rules

- Never stage files yourself.
- Never commit without explicit approval.
- Never add an AI attribution footer unless asked.
- Read the linked reference files when the type or format is unclear.
```

Two boundaries are important here. First, the skill may inspect staged work but may not stage files. Second, drafting a message and executing `git commit`

are separate actions. A good skill describes that separation directly.

The tutorial's design principle is: let the model decide, let the script verify, and let the script calculate. A model can choose whether a change is a feature or a fix. A script can consistently check the subject format, maximum length, blank-line separator, and breaking-change footer.

The example validator exposes a pure `validate(message)`

function and a small CLI. It uses only the standard library, so there is no package installation step for the validation path.

``` php
def validate(message: str) -> list[Violation]:
    """Return deterministic violations for one commit message."""
    lines = message.splitlines()
    if not lines or not lines[0].strip():
        return [Violation("error", "empty", "Message has no subject line.")]

    subject, body = lines[0], lines[1:]
    violations = []
    match = SUBJECT_RE.match(subject)
    if not match:
        return [Violation("error", "format", "Invalid Conventional Commit subject.")]

    if len(match.group("subject")) > 72:
        violations.append(Violation("error", "subject-length", "Subject is too long."))
    if body and body[0].strip():
        violations.append(Violation("error", "body-separator", "Body needs a blank line."))
    return violations
```

The full implementation in [ check_message.py](https://github.com/paladini/how-to-create-a-skill-tutorial/blob/main/examples/commit-crafter/scripts/check_message.py) also checks allowed types, imperative mood warnings, breaking-change footers, and long body lines. The abbreviated code above illustrates the shape; use the repository file when you want the complete behavior.

Exit codes are part of the interface with the agent:

`0`

means the input passed.`1`

means a validation or business-rule violation needs attention.`2`

means there was no input.Non-zero output should say what to fix. "Subject is 79 characters; maximum is 72" gives the agent a useful next action. "Validation failed" does not.

The repository includes unit tests for the validator, including valid messages, malformed subjects, missing blank lines, breaking changes, warnings, and CLI exit codes.

```
python examples/commit-crafter/tests/test_check_message.py -v
```

On the current `main`

checkout, this command completed with 16 tests passing. The repository's other deterministic examples also have standard-library test files for changelog generation and memory recording.

```
python examples/changelog-forge/tests/test_generate_changelog.py -v
python examples/lessons-keeper/tests/test_lessons.py -v
```

These tests are intentionally close to the scripts. A skill's instructions can drift, but a focused test keeps the mechanical contract visible in CI.

Copy the skill directory into the discovery path used by your agent. For a personal installation, the repository documents this example:

```
cp -r examples/commit-crafter ~/.agents/skills/
```

Then try prompts that resemble real requests, not only the ideal wording:

``` php
commit this                         -> should trigger
write a good commit message         -> should trigger
reword my last commit               -> should trigger
what does the diff do?              -> should not trigger
create a git branch                 -> should not trigger
```

The description is the main trigger signal. If "reword my last commit" does not activate the skill, improve the description with the user's vocabulary before adding more body instructions.

`SKILL.md`

is loaded as instructions, so it is the right place for intent, sequencing, judgment, and safety boundaries. Reference files can hold detail that is only needed for a particular decision. Scripts are better for parsing, counting, rendering, and rejecting invalid output.

This is also a token-budget decision. A model does not need to reconstruct a line-count rule every time. It can run a small script and use the result. Deterministic scripts also make retries safer when they are idempotent and avoid network access unless the workflow truly requires it.

A skill is executable code with the permissions of the agent. Before installing a third-party skill, read its scripts. Treat `curl | bash`

, unexpected credential-file reads, telemetry, and undeclared network calls as reasons to stop and investigate.

Common authoring failures include vague descriptions, a `SKILL.md`

that is too large, scripts that silently succeed on invalid input, and instructions that blur drafting with side effects. Keep the main file focused, use actionable stderr messages, and require explicit approval before operations such as commits or publication.

The examples here are local and standard-library-first. They do not provide a security guarantee for the agent that executes them. Review compatibility requirements and permissions for your own client.

No. Use a script when a stable rule is easier to enforce mechanically. A small skill that only provides judgment or reference material may need just `SKILL.md`

.

`SKILL.md`

?
You can, but the specification recommends progressive disclosure. Keep the activated instructions concise and move detailed references into separate files that the agent reads when needed.

No package installation is required for the example skills. They are folders that you copy into a supported skills directory. The tutorial site itself uses MkDocs Material, but that dependency is separate from running the example validators.

Start an Agent Skill with a narrow promise and an explicit boundary. Put the workflow in `SKILL.md`

, put repeatable checks in deterministic scripts, and make tests exercise the same commands the agent will run. That gives you an installable skill that is easier to review, debug, and evolve.

What is one agent workflow in your project where a short validator would be more reliable than another paragraph of instructions?

AI assistance disclosure: This article was prepared with AI assistance using the repository's public documentation and source examples. The commands, project status, and test result were checked against the current public

`main`

branch before publication.
