In a previous post, I extracted skill usage stats from my conversation logs. This time I'm going the other way: a weekly loop that reads the transcripts to detect whether my Claude Code environment is degrading, and when a threshold is crossed, lets claude -p
fix things itself.
~/.claude/scripts/cc-self-audit.sh
(177 lines) runs every Sunday at 8:30 via launchd. It collects five numbers, compares them against thresholds, and if everything is green it sends a one-line ✅; if anything is red, it self-repairs, independently re-measures, and reports to Discord. About 70% of the implementation effort went into "why a naive grep doesn't work here."
A performance audit on 2026-07-11 uncovered a triple whammy: agents that were supposedly moved out of the way were still being injected — 99 of them; CLAUDE.md had silently bloated to 228KB; and a Stop hook was firing dozens of times per week.
Each of those is fixable on its own. The real problem was that there was no mechanism to notice the "fixed it, but it broke again" cycle. A human doing a weekly visual check doesn't last, so I made a script do it.
launchd（日曜 8:30）
→ cc-self-audit.sh
① collect() → 5数値（静的2 + 動的3）
② breaches() → 閾値超過を列挙
③ 緑 → ✅通知・終了
④ 赤 → claude -p（sonnet・MAX枠）が ~/.claude 内を修正
⑤ collect() 独立再実行 → 数値改善を確認（自己申告は信じない）
⑥ Discord #01_alerts へ報告
(The flow: launchd on Sunday 8:30 → ① collect()
gathers 5 numbers (2 static + 3 dynamic) → ② breaches()
lists threshold violations → ③ green: notify ✅ and exit → ④ red: claude -p
(sonnet, on my MAX plan) fixes things inside ~/.claude
→ ⑤ collect()
re-runs independently to confirm the numbers improved (self-reports are not trusted) → ⑥ report to Discord #01_alerts
.)
Thresholds (overridable via env vars):
| Metric | Threshold | Variable |
|---|---|---|
| inject_bytes | 40,000 B | SELF_AUDIT_TH_INJECT |
| agents_loaded | 60 files | SELF_AUDIT_TH_AGENTS |
| stopspam | 15 /week | SELF_AUDIT_TH_STOPSPAM |
| frustration | 8 /week | SELF_AUDIT_TH_FRUST |
| toolerr | 400 /week | SELF_AUDIT_TH_TOOLERR |
inject_bytes=$(( \
$(find "$HOME/.claude/rules" -name '*.md' -print0 2>/dev/null | xargs -0 cat 2>/dev/null | wc -c) + \
$(cat "$HOME/.claude/CLAUDE.md" 2>/dev/null | wc -c) + \
$(cat "$HOME/CLAUDE.md" 2>/dev/null | wc -c) + \
$(cat "$HOME/.claude/projects/-Users-matsubara/memory/MEMORY.md" 2>/dev/null | wc -c) ))
agents_loaded=$(find "$HOME/.claude/agents" -name '*.md' 2>/dev/null | wc -l | tr -d ' ')
The find ~/.claude/agents -name '*.md'
is deliberately recursive so it re-detects a failure mode I've already been burned by: agents that were "archived" into dot-prefixed subdirectories (like .backup/
) getting loaded again. If agents_loaded
exceeds 60, I treat it as "the archiving has leaked."
This is the core of the design. Targeting "up to 200 JSONL files over 100KB that were updated since the last run," it counts stopspam (actual firings of the audit hook) and frustration (the user's frustration words).
STOP_MARKER = 'Stop hook feedback:\n[~/.claude/hooks/self_audit_stop.sh]: '
FRUST_RE = re.compile(r'何回も言|いい加減にし|嘘つ|舐めんな|なんで治らん|最悪やろ|頭悪い')
for line in fh:
if STOP_MARKER in line:
stopspam += 1
try:
o = json.loads(line)
except Exception:
continue
if o.get('type') != 'user' or o.get('isMeta'):
continue # ← ここが肝
content = (o.get('message') or {}).get('content')
texts = [content] if isinstance(content, str) else \
[b.get('text', '') for b in content if isinstance(b, dict) and b.get('type') == 'text'] \
if isinstance(content, list) else []
if any(FRUST_RE.search(t) for t in texts):
frustration += 1
(The regex matches Japanese phrases like "how many times do I have to say this," "give me a break," "liar," "why isn't this fixed yet," and so on; the # ← ここが肝
comment marks the crucial line.)
Only blocks with type=user
that are not isMeta
are examined — in other words, only messages a human actually typed. Why that matters is the next section.
The first implementation looked like this.
stopspam=$(echo "$files" | xargs grep -h -c 'self_audit_stop' 2>/dev/null | awk '{s+=$1} END{print s+0}')
frustration=$(echo "$files" | xargs grep -hE -c 'なんで治らん|いい加減' 2>/dev/null | awk '{s+=$1} END{print s+0}')
history.jsonl
still holds the measured values from back then.
{"ts":"2026-07-11 22:37:28","inject_bytes":21798,"agents_loaded":48,"stopspam":6,"frustration":4,"toolerr":2}
{"ts":"2026-07-12 08:30:06","inject_bytes":21966,"agents_loaded":48,"stopspam":40,"frustration":18,"toolerr":67}
{"ts":"2026-07-12 08:40:03","inject_bytes":21966,"agents_loaded":48,"stopspam":58,"frustration":32,"toolerr":68}
The evening of 07-11 was normal (GREEN). The morning of 07-12, stopspam jumped from 6 to 40 and frustration from 4 to 18 — RED. Then claude -p
attempted a fix, the independent re-measurement ran, and the numbers had climbed further, to 58/32.
The cause: grep was doing a string search over the entire JSONL. The transcripts contained:
old_string
/new_string
)All of it was matching. Of the 40 stopspam hits, actual hook firings were near zero; of the 18 frustration hits, only a handful were frustration words a human had actually typed.
A naive string grep cannot be used against transcripts.Unless you first determine "which role, which block" within the JSONL, quotations, tool output, and echoes of past logs all get counted. The resolution of your measurement design determines the quality of your audit.
The fix was switching to a JSONL parser. Only text blocks with type=user
and isMeta: false
(messages the user actually sent) are examined, and stopspam counts only lines matching the exact fixed format the harness actually injects. That excluded quotations, tool output, and echoes of past logs entirely.
The core of the prompt passed to claude -p
when things go red:
PROMPT="あなたはClaude Code環境の自己監査・自己修復エージェント。
## 検知した違反
$BREACH
## 指示
1. ~/.claude 内だけを調査・修正する（プロジェクトコード・secret・plist削除は禁止）
2. 変更は1件ずつ ${CHANGELOG} に「日時/対象/理由/戻し方」を追記
3. 修正後に必ず 'bash ~/.claude/scripts/cc-self-audit.sh --collect-only' を実行し改善を数値で確認
4. 自分で安全に直せない項目は無理せず、最後に『要人間: 理由』と1行で出力
5. 最終出力は3行以内の要約のみ"
(In English, the prompt says: "You are a self-audit / self-repair agent for the Claude Code environment. ## Detected violations: $BREACH
. ## Instructions: 1. Investigate and fix only inside ~/.claude
(project code, secrets, and plist deletion are forbidden). 2. For each change, append date/target/reason/how-to-revert to ${CHANGELOG}
. 3. After fixing, always run bash ~/.claude/scripts/cc-self-audit.sh --collect-only
and confirm improvement numerically. 4. Don't force items you can't safely fix yourself — end with a one-line 'Needs human: reason.' 5. Final output is a summary of at most 3 lines.")
The important part is instruction 3 — or rather, what happens outside it. Instead of only having claude -p
re-measure itself, the harness also runs collect()
independently and checks the numbers (from line 165):
AFTER=$(collect)
log "after: $AFTER"
echo "$AFTER" >> "$HISTORY"
STILL=$(echo "$AFTER" | breaches)
If the AI says "fixed it" but the numbers haven't improved, a 🚨 goes out. When you use a Claude-built tool to repair Claude's own environment, trusting only the self-report is how you get hurt.
What the current history.jsonl
tells us:
The first two entries are the before/after of the manual --collect-only
on 07-11; entries 3–4 are the 07-12 grep misdetection and the subsequent re-measurement. Production operation as a weekly job starts from 08-30.
<key>StartCalendarInterval</key><dict>
<key>Weekday</key><integer>0</integer> <!-- 日曜 -->
<key>Hour</key><integer>8</integer>
<key>Minute</key><integer>30</integer>
</dict>
(Weekday 0
with the 日曜
comment means Sunday.)
RunAtLoad
is false
. If the job ran immediately on registration, the first measurement would happen in the "empty" state right after launchd starts. Instead, the first baseline measurement is done by manually running --collect-only
to put one line into history.jsonl
before registering the job.
type=user && !isMeta
self_audit_stop
match for stopspam also hit quotations of the hook script itselfcollect()
independently of the claude -p
output, append to HISTORY
, and only then judgeclaude
not found under launchd's minimal PATHPATH
explicitly in the plist's EnvironmentVariables
(/opt/homebrew/bin
~/.local/bin
are required)frustration
pinned at zero can look like "nothing is happening"claude -p
self-repairstype=user && !isMeta
) is riddled with false positives — Next time, I'll walk through what claude -p
actually changed when this audit fired, using real entries from the change log (self-audit-changes.log
).
*Written by Lily — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*