cd /news/ai-agents/show-hn-trace2rules-mine-and-audit-c… · home › topics › ai-agents › article
[ARTICLE · art-173900] src=github.com ↗ pub= topic=ai-agents verified=true sentiment=· neutral

Show HN: trace2rules – Mine and audit coding-agent logs into AGENTS.md rules

A new local-first CLI called trace2rules ingests coding-agent session trajectories, mines recurring friction, and audits existing AGENTS.md and CLAUDE.md rules against real sessions, labeling each rule observed, compliant, violated, unobserved, or already enforced by CI/pre-commit. On the Open-SWE swc-project/swc split of 188 sessions, the tool's audit of the current AGENTS.md found 65 rules: 10 measured, 0 partial, and 55 unmeasured, with rules such as "Commit frequently" violated in 52 of 188 sessions (28%) and "Submodule update before tests" violated in 32 of 32 (100%). The tool outputs proposals rather than applied rules, and its author recommends at least 10 local sessions or a public dataset for useful results.

key takeaway

trace2rules audits AGENTS.md rules against real agent sessions and found 55 of 65 rules in swc-project/swc's AGENTS.md unmeasured, with the submodule rule violated in 32 of 32 sessions.

read9 min views5 publishedNov 21, 2026
Show HN: trace2rules – Mine and audit coding-agent logs into AGENTS.md rules
Image: Michielbdejong (auto-discovered)

Mine and audit coding-agent sessions into evidence-backed AGENTS.md rules.

trace2rules is a local-first CLI. It ingests agent trajectories, mines recurring friction, audits existing AGENTS.md/ CLAUDE.md rules against real sessions, and identifies whether each rule is observed, compliant, violated, unobserved, or already enforced by CI/pre-commit.

The output is a proposal, not an automatically applied AGENTS.md.

Small local corpus? Results are most useful with about 10+ local sessions. With fewer, ingest a public dataset first; see Public datasets.

Paste this into your coding agent to audit the current checkout:

Install trace2rules if missing (`uv tool install trace2rules`). Run `trace2rules audit --prompt > audit.txt`, read audit.txt, and list unobserved, violated, and unenforced rules. Propose concrete enforcement without duplicating existing CI/pre-commit checks.

Or run the deterministic audit manually:

trace2rules audit

Paste this into your coding agent to generate rules:

Install trace2rules if missing (`uv tool install trace2rules`). Run `trace2rules mine --prompt --out-dir prompts/` for my local agent sessions. Read the prompt files and synthesize repo-specific AGENTS.md rules yourself. Keep only rules supported by the evidence.

Or generate the evidence prompts manually:

trace2rules mine --prompt --out-dir prompts/

See Auditing AGENTS.md and Generating rules for the full flows.

trace2rules audit is deterministic: it prints counts, coverage, verdicts, enforcement status, and deterministic suggestions.

For a fresh-agent review, emit the evidence prompt:

trace2rules audit --prompt > audit.txt

Give audit.txt to your agent. The LLM receives the audit table but not the deterministic suggestions.

After the agent writes proposed rules, validate them:

trace2rules audit --proposal AGENTS.proposed.md

--proposal parses the proposed Markdown, audits it against sessions, and prints verdicts before you apply it.

On the Open-SWE swc-project/swc split (188 sessions), the current AGENTS.md audit finds 65 rules: 10 measured, 0 partial, and 55 unmeasured.

Rule Followed Violations Confidence Enforcement
Commit frequently 136 52/188 (28%) high missing
No nightly-only rustc features 172 16/188 (9%) low missing
Use gh for GitHub data 18 6/24 (25%) medium .github/workflows/CI.yml
Snapshot update with UPDATE=1 cargo test 23 2/25 (8%) medium missing
--no-verify guard 13 1/14 (7%) low missing
Read the codebase before editing 123 62/185 (34%) high guidance only
Verify assumptions against source/tests 123 62/185 (34%) high guidance only
Submodule update before tests 0 32/32 (100%) medium guidance only
git commit only aftergit add 8 6/14 (43%) low guidance only
cargo test -p <crate> 14 0/14 (0%) low missing

An audit --prompt pass over the same table suggests:

  • Delete the four stale binding commands and add a workflow that checks backticked AGENTS.md paths atHEAD .
  • Reword commit cadence: commit one logical change after cargo fmt --all -- --check and the narrowest affectedcargo test -p <crate> .
  • Enforce the nightly rule with cargo +stable check and a guard rejecting#![feature(...)] ,RUSTC_BOOTSTRAP , and nightlyrust-toolchain.toml .
  • Replace the GitHub access rule with: use gh ; CI fails oncurl /wget togithub.com .
  • Run snapshot tests with UPDATE unset; keepUPDATE=1 local-only for regeneration.
  • Make hooks and CI apply the same checks so --no-verify cannot bypass them.
  • Consolidate the two read rules into a PR Source: /Test: requirement.
  • Initialize submodules before every test job.
  • Stage explicit paths with git add ; rejectgit commit -a .

The review proposed three new rules: no package-scoped test commands in root AGENTS.md, every AGENTS path must exist at HEAD, and CI must run with UPDATE unset.

MSRV is not measured yet, so the review does not propose a separate MSRV enforcement rule.

audit prints the audited/total session count and audits all sessions by default. Pass --latest-commit to use the latest AGENTS.md commit as a cutoff; use --since/--until for explicit dates. By default, audit output shows measured and partial rules only; pass --all-rules to include unmeasured rules.

Mining produces evidence, not rules. trace2rules mine --prompt --out-dir prompts/ writes one prompt per detected repository; give those prompts to your agent. The agent proposes rules, then validate them with audit --proposal. Use --root when the checkout is not the directory containing AGENTS.md, so path and enforcement checks inspect the real repository.

swc-project/swc

  • [guidance] Before changing implementation code, run an existing fixture/test or add a focused failing test and execute it on the unmodified tree; do not use a standalone /tmp mock as the only reproduction for changes undercrates/ .
  • [guidance] Create one-off reproductions under /tmp or as tests/fixtures in the affected crate; do not add a temporary workspace crate or edit workspaceCargo.toml /Cargo.lock for a reproduction.
  • [guidance] When the default toolchain rejects repository rustflags, run cargo with RUSTUP_TOOLCHAIN=<stable> and setRUSTFLAGS at invocation; do not edit.cargo/config.toml to work around the toolchain.
  • [guidance] For large source files, read the relevant ranges with sed -n '<start>,<end>p' <file> or locate symbols withrg -n ; do not page whole files until output is clipped.

rkkautsar/trace2rules

  • [guidance] Keep src/trace2rules/cli.py limited to Typer wiring and option parsing; put non-CLI logic in a dedicated module undersrc/trace2rules/ .
  • [guidance] When adding, renaming, or removing a CLI command or option in src/trace2rules/cli.py , update the matching row in theREADME.md command table in the same change.

The latest synthesis produced these four rules after the stale-path filter was corrected. Rules are marked guidance when no concrete CI, pre-commit, or lint check exists; enforceable rules require a specific check. Validate proposals with audit --proposal.

trace2rules proposes durable, normative AGENTS.md rules, not memories.

It targets two destinations:

  • Enforceable rules : a CI check, pre-commit hook, lint rule, or test can observe and enforce them.
  • Stable AGENTS rules : durable repo or global policy that future agents should follow even though it is not mechanically checkable.

It deliberately does not mine memories, historical decisions, ownership, dates, task state, or "what happened" summaries. Those are contextual and belong in a memory system, not in AGENTS.md. If evidence is descriptive rather than prescriptive, the tool should omit it.

  1. Ingest normalizes tool calls into DuckDB (sessions ,tool_events ,feedback_events ,test_results ).
  2. Mining finds hotspots, recovery loops, validation gaps, edit rejections, blind edits, environment thrash, destructive commands, generated-path edits, dependency drift, and correction loops.
  3. Sequence mining normalizes commands and paths, then mines repo-local sequential rules (antecedent -> specific error ) with support, confidence, lift, and a representative evidence window. Exact sequences use lift and paired-rework evidence; windows never cross a compaction boundary.
  4. Prompt selects bounded findings and writes one readable Markdown evidence prompt per repo whenmine --prompt is used.
  5. Audit checks existingAGENTS.md rules against sessions.holdout splits sessions by time and audits the same rules on an earlier train slice and later holdout.

Generated verifier scripts and low-support task-specific sequences are filtered before prompting.

Command Purpose
ingest Load sessions and tool events into DuckDB
mine Mine friction findings, JSON, or evidence prompts; auto-ingests local sessions
doctor Check install, Node, source stores, and database schema
patterns Mine repo-local sequential rules (antecedent → specific error)
exact Show recurring normalized sequences with evidence
audit Audit existing AGENTS.md rules against sessions and suggest enforcement
holdout Audit rules on an earlier train split and a later time holdout

Useful options:

trace2rules ingest                         # ~, all detected local stores
trace2rules ingest --format swe-rebench    # opt into the public dataset
trace2rules ingest --format open-swe --repos owner/name
trace2rules ingest --source PATH --limit 200
trace2rules exact --length 4 --per-repo 2
trace2rules patterns --top 10
trace2rules audit --agents AGENTS.md
trace2rules audit --agents AGENTS.md --commit HEAD --json
trace2rules audit --agents AGENTS.md --root . --prompt
trace2rules audit --agents AGENTS.md --all-rules
trace2rules holdout --agents AGENTS.md --fraction 0.3
trace2rules mine --prompt --min-support 5
trace2rules mine --prompt --max-rules 5
trace2rules mine --prompt --out-dir prompts/ --max-rules 10
trace2rules mine --json

--source is repeatable and defaults to your home directory. It scans:

  • ~/.claude/projects
  • ~/.codex/sessions
  • ~/.pi/agent/sessions
  • ~/.omp/agent/sessions
  • ~/.local/share/opencode

The default cap is 200 sessions per source. --limit N selects the newest N files per invocation; re-runs can select a different set when files are added or touched. Use --limit 0 for the full deterministic set. OpenCode is read directly from its SQLite database; JSON exports are still accepted:

uv run trace2rules ingest --format agent-trajectory --source ~/.local/share/opencode

The repository label is inferred from the git remote (owner/name) when available, falling back to the working-directory basename. mine --prompt emits one prompt per detected repo:

trace2rules mine --prompt --out-dir prompts/

Local ingest treats each compaction segment as its own session by default; use --no-split-compactions to keep the original boundaries. This applies to OMP, Pi, Codex, Claude Code, and OpenCode.

mine and audit auto-ingest local sessions. They refresh the default database on each run; explicit --source or non-default --format triggers ingestion into a custom --db. ingest writes to $XDG_DATA_HOME/trace2rules/trace2rules.duckdb by default (~/.local/share/...). Set TRACE2RULES_DB to override it.

Use a public dataset when the local corpus is small. Both datasets are public and require no Hugging Face token. The first run downloads the parquet into ~/.cache/trace2rules; set TRACE2RULES_CACHE_DIR to move it, or pass --refresh-cache to re-download.

nebius/SWE-rebench-openhands-trajectories

trace2rules ingest --format swe-rebench
trace2rules ingest --format swe-rebench --repos django__django --limit 50

The default repositories are django__django and pytest-dev__pytest. For swe-rebench, --repos takes instance prefixes such as django__django.

trace2rules ingest --format open-swe --repos owner/name

--repos is required for open-swe and takes repository labels in owner/name form. This downloads six public train-*.parquet files.

Local Claude Code, Codex, Pi, OMP, and OpenCode SQLite/JSON stores. See Local sessions.

uv tool install trace2rules
pip install trace2rules

For development from a clone:

uv sync

agent-trajectory calls its bundled JS bridge through the system Node runtime for Claude Code, Codex, Pi, and OMP. Install Node.js 20 or newer; OpenCode SQLite ingestion is pure Python.

Variables are read from the process environment. trace2rules does not load .env files.

Variable Purpose
TRACE2RULES_DB Override the default DuckDB path
TRACE2RULES_CACHE_DIR Local parquet cache; defaults to ~/.cache/trace2rules
TRACE2RULES_WINDOWS_STORES Auto-detects WSL and scans /mnt/c/Users ; set to0 to disable or1 to force
TRACE2RULES_WORKERS Mining worker count; defaults to the memory budget
TRACE2RULES_MEMORY_LIMIT DuckDB memory cap; defaults to 25% of available memory
  • Session data stays in your local DuckDB database.

  • mine performs no LLM calls. Findings can still contain commands, paths, and repository names; review before sharing a prompt externally.

  • data/ is gitignored.

  • Detectors are heuristics. Review every proposed rule.

  • Local-session formats change frequently. agent-trajectory is the compatibility layer; pin its version.

uv sync
uv run pytest
uv build

MIT. See LICENSE.

── more in #ai-agents 4 stories · sorted by recency
── more on @trace2rules 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/show-hn-trace2rules-…] indexed:0 read:9min 2026-11-21 · —