# Four Alarm Slots, Three Failure Modes: Building a Nightly Drain That Survives Sleep, Races, and Timeouts > Source: > Published: 2026-08-24 00:00:06+00:00 Every night my Mac quietly rewrites my long-term memory. Not metaphorically — a shell script drains that day's Claude Code conversation logs into an Obsidian vault, commits them to a private repo, and leaves a briefing on my desktop. It took three real outages to make it reliable. This is the script, the three failures, and the design that came out of them. Claude Code sessions are independent of one another. The root cause of a bug you found during a long working session today, the reason you settled on a particular architecture after trial and error, the accumulated knowledge that "this direction already failed once" — none of it is available in the next conversation once you close the session. Even on a paid plan, even with the most capable model available, if context isn't carried over you have to explain everything from scratch every time. Many people have had the experience of thinking "I already looked this up before" or "I should have failed at this once already, and yet here I am heading down the same road again." In a phase where you're shipping personal projects in volume, this problem is fatal. Once three or four projects are running in parallel, tracking "where each project currently stands" by hand hits a wall fast. And Claude, unable to reference previous conversations, repeats the same deliberations. My first attempt at this problem was "I'll write up a summary by hand every day." It didn't last. When work has momentum you don't feel like writing a summary, and when you're tired you can write even less. A system that depends on human willpower doesn't function during a high-volume solo-dev phase. The answer was to build an environment that automatically drains Claude's conversation logs into Obsidian every night. Once the environment is in place, willpower and motivation are irrelevant. The Mac just does it. The reason I chose Obsidian is simple. The files are local Markdown, so Claude Code can read and write them directly. They can be version-controlled with Git. The `[[link]]` syntax lets you connect pieces of knowledge to each other. Logs flow in every morning and cross-project links grow naturally — from the moment this started functioning as an "external brain," the quality of my work changed. Simply "running a script at 4:55 every day" produced three distinct kinds of failure once I actually ran it. Each one only became apparent after it caused real damage. **Sleep freeze**: If you close the lid on the Mac and go to bed, `caffeinate -s` (which only takes effect on AC power) can't prevent sleep. The script stops partway through, and that day's processing hangs in limbo until the next slot fires. **Double-execution race**: An actual incident on 2026-06-10. A scheduled launchd firing overlapped with a manual run. Both tried to operate on the same vault with Git, and the result was a conflict. **Timeout**: On 2026-06-13, every slot failed for the entire day. Digesting 28 hours' worth of logs via `claude -p` didn't fit within the 40-minute timeout window and all of it got culled. In the logs, the pattern `started 04:55:00 → step2 timeout at 05:40:01` lines up across all four slots, completely uniform. These three real failures are what forced the triple-layered structure of "multi-slot re-firing," "caffeinate sleep prevention," and "idempotent retry via step markers." It didn't come out of a design document; it accumulated from things that actually broke. Once the system started running stably, logs pile up in Obsidian every morning. Claude reads those logs, updates per-project articles, and this week's talking points accumulate in `hot.md` (a summary of recent context). When Claude reads `hot.md` and `wiki/` in the next conversation, it can start working already knowing "last week's decisions," "the approach that failed once," and "the current state of the three projects running in parallel." That's an external long-term memory that doesn't forget when the conversation ends — a prerequisite for getting Claude Code to perform at its actual potential. The data the script handles flows through the following four layers. ``` 【層1】Claude Code セッションログ（セッション終了時にStop hookが書き出し） ↓ extract_conversations.py（step1: 最新化） 【層2】~/Documents/my-knowledge-base/raw/conversations/ ↓ claude -p（step2a: Claude由来ログを消化, timeout 1500s） ~/Documents/my-knowledge-base/raw/codex-conversations/ ↓ claude -p（step2b: Codex由来ログを消化, timeout 1500s） 【層3】Obsidian Vault（~/Documents/claude-obsidian/wiki/） ├── hot.md（直近サマリ） ├── index.md（全体目次） ├── projects/ / learning/ / career/ ... （ドメイン別記事） └── today-brief.md（step2.5: 今日の行動提案） ↓ git add -A && git commit && git push 【層4】private repo（安全網: 荒れてもrevert可能） ``` Splitting layer 2 into two lines, "Claude-derived" and "Codex-derived," is a change made on or after 2026-06-11. Originally one line processed everything, but on high-activity days it stopped fitting into the 40-minute timeout window. After the split, each has an independent 1500-second (25-minute) timeout, and if one fails the next slot can retry only what's left. Four slots defined in a launchd plist (`~/Library/LaunchAgents/com.shun.vault-auto-ingest.plist` ) spend the day repeatedly "retrying until success." ``` StartCalendarInterval Hour4Minute55 Hour8Minute20 Hour10Minute45 Hour12Minute15 ``` `RunAtLoad` is `false` . There's no auto-start at login. The four slots are the only firing sources. At the top of the script it checks for the day's success marker and exits immediately if it's already finished. ``` TODAY=$(date +%Y%m%d) DONE_MARKER="$HOME/.claude/logs/.vault-ingest-done-${TODAY}" # 0. 本日分が既に成功していれば即終了 [ -f "$DONE_MARKER" ] && exit 0 ``` With this, if 4:55 succeeds then 8:20, 10:45, and 12:15 become harmless no-ops that just "check the file and exit." Only if 4:55 fails (sleep freeze, no network, a transient launchd fault, etc.) does 8:20 actually take over the processing. When running this on a MacBook, closing the lid is something `caffeinate -s` (only effective on AC power) can't fully prevent. So the script re-executes itself under both `-i` (prevent system sleep) and `-s` . ``` if [ -z "${CAFFEINATED:-}" ]; then exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@" fi ``` If the `CAFFEINATED` environment variable isn't set, it re-executes itself under `caffeinate` and puts everything after that under caffeinate's umbrella. Since `CAFFEINATED=1` is set after the single re-execution, there's no infinite loop. `-s` has no effect on battery power. If it freezes from lid-close sleep, `timeout` culls it and the next slot starts over. If step2 got partway through, the markers let it skip what's done and run only the rest. This is a locking mechanism that uses the atomicity of `mkdir` (if two processes call it simultaneously, only one succeeds). It's a widely used technique for file locking in Bash. ``` LOCKDIR="$HOME/.claude/locks/vault-auto-ingest.lock" if ! /bin/mkdir "$LOCKDIR" 2>/dev/null; then oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || true) if [ -n "${oldpid:-}" ] && kill -0 "$oldpid" 2>/dev/null; then echo "[$(date '+%F %T')] 別インスタンス実行中(pid=${oldpid}) — skip" >> "$LOG" exit 0 fi rm -rf "$LOCKDIR" /bin/mkdir "$LOCKDIR" 2>/dev/null || exit 0 fi echo $$ > "$LOCKDIR/pid" trap 'rm -rf "$LOCKDIR"' EXIT INT TERM ``` Even when an existing lockdir is present, it checks the PID, and if the process is alive it exits as "another instance is running." If the process is dead (a lock left behind by an abnormal termination), it reclaims it as stale, acquires a new lock, and continues. `trap` guarantees the lockdir is deleted when the script exits. This is the part that took the most work. step2 (the ingest into the vault) is split into two lines, Claude and Codex, each with its own independent marker. ``` STEP2A_MARKER="$HOME/.claude/logs/.vault-ingest-step2a-claude-${TODAY}" STEP2B_MARKER="$HOME/.claude/logs/.vault-ingest-step2b-codex-${TODAY}" ``` The `ingest_src` function looks at these markers to decide "skip if already complete, run if not." ``` ingest_src() { local marker="$1" src="$2" name="$3" to="$4" extra="$5" [ -f "$marker" ] && { echo "[...] step2($name) は本日実施済み — skip" >> "$LOG"; return 0; } cd "$VAULT" && run_to "$to" "$CLAUDE" -p \ "...（Vaultのルールに従って wiki/ を更新するプロンプト）..." \ --dangerously-skip-permissions >> "$LOG" 2>&1 \ && { touch "$marker"; return 0; } \ || { echo "[...] WARN: step2($name) 失敗/timeout（次スロットで再試行）" >> "$LOG"; return 1; } } ingest_src "$STEP2A_MARKER" "$KB/raw/conversations/" "claude" 1500 "" ingest_src "$STEP2B_MARKER" "$KB/raw/codex-conversations/" "codex" 1500 "Codex由来でも既存記事に統合し重複は追記でまとめろ。" [ -f "$STEP2A_MARKER" ] && [ -f "$STEP2B_MARKER" ] && touch "$STEP2_MARKER" ``` The behavior pattern is as follows. ``` 4:55 発火 ├── DONE_MARKER なし → 処理続行 ├── step2a 実行（claude由来, 1500s上限） → 成功 → STEP2A_MARKER 作成 ├── step2b 実行（codex由来, 1500s上限） → timeout! → マーカーなし ├── brief生成 → 失敗（step2b未完でログが薄い） → DONE_MARKER 作らない └── notify_fail で Desktop に FAILED ファイル + 通知 8:20 発火 ├── DONE_MARKER なし → 処理続行 ├── step2a → STEP2A_MARKER あり → skip（再実行しない） ├── step2b → マーカーなし → 実行 → 成功 → STEP2B_MARKER 作成 ├── brief生成 → 成功 ├── DONE_MARKER 作成 ✓ └── FAILED ファイル削除 10:45 / 12:15 発火 └── DONE_MARKER あり → exit 0（空振り） ``` If step2a succeeded, 8:20 runs only step2b. If both step2a and step2b are finished, 8:20 runs only the brief. Because which slot is responsible for what is determined dynamically, the guarantee that "even if 4:55 fails, it will definitely finish within the day" is preserved. launchd's execution environment has only a minimal PATH, roughly `/usr/bin:/bin:/usr/sbin:/sbin` . To avoid the trap where the post-`git commit` hook (written in node) doesn't run and the `commit` itself fails, the top of the script auto-discovers the newest nvm node and adds it. ``` NODE_BIN=$(ls -d "$HOME"/.nvm/versions/node/*/bin 2>/dev/null | sort -V | tail -1) export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$PATH" [ -n "$NODE_BIN" ] && export PATH="${NODE_BIN}:$PATH" ``` Taking the tail of `sort -V` (version-order sort) means it always uses the newest even when nvm has multiple versions installed. Since no version is hardcoded, adding versions with `nvm install` requires no changes to the script. Also, due to macOS TCC (privacy protection), launchd can't write under `~/Documents/` unless `/bin/bash` has Full Disk Access. The script detects this early to prevent silent failure. ``` if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then echo "[...] ❌ FDA未付与: launchdから '$VAULT' にアクセス不可(TCC保護)。" >> "$LOG" notify_fail "FDA未付与: vault にアクセス不可（設定→フルディスクアクセス→/bin/bash）" exit 1 fi ``` Returning `exit 0` would look like "it succeeded," DONE_MARKER would be created, and the next slot wouldn't retry. TCC failure is signaled explicitly with `exit 1` and handed off to the next slot. One of the script's core decisions is the `run_to` function. ``` TIMEOUT_BIN="/opt/homebrew/bin/timeout" [ -x "$TIMEOUT_BIN" ] || TIMEOUT_BIN="" run_to() { local s=$1; shift; if [ -n "$TIMEOUT_BIN" ]; then "$TIMEOUT_BIN" --kill-after=30 "$s" "$@"; else "$@"; fi; } ``` It calls GNU coreutils' `timeout` by explicit path. macOS's standard `/usr/bin` doesn't have the GNU version of `timeout` . In environments where `/opt/homebrew/bin/timeout` doesn't exist, `TIMEOUT_BIN` is left empty and calls pass through (hang resistance drops, but it keeps working). `--kill-after=30` matters more than it looks. If the process is still alive 30 seconds after `SIGTERM` is sent, it's force-killed with `SIGKILL` . `claude -p` sometimes ignores `SIGTERM` while doing heavy work, so without `--kill-after` you can hit the worst case: "the process survives past the timeout while still holding the lockdir." step2a (Claude-derived logs) and step2b (Codex-derived logs) each get a 1500-second (25-minute) limit. As mentioned earlier, it was originally a single 2400-second (40-minute) process, but as activity increased it stopped fitting. Splitting into two lines at 1500 seconds each means that if one dies, the next slot can re-run only the remainder. Even when the Mac wakes at 4:55, it can take tens of seconds for the Wi-Fi connection to stabilize. Both `claude` and `git push` need the network, so running before the connection is up is an immediate error. ``` net_ok="" for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18; do if /usr/bin/nc -z -G 3 1.1.1.1 443 2>/dev/null; then net_ok=1; break; fi sleep 5 done [ -z "$net_ok" ] && echo "[...] WARN: 網未接続のまま続行（失敗時は次スロットが再試行）" >> "$LOG" ``` 18 attempts × 5 seconds = up to 90 seconds of waiting. `nc -z -G 3` is a TCP connection test that times out in 3 seconds, lighter than `curl` . If it still isn't connected after 90 seconds, it "warns and continues." Using `exit 1` here would also stop step1 (local processing) in an offline environment. Since the design already assumes the next slot auto-retries on failure, "record it without stopping" is enough here. Next, preflight detection chains through three stages. **① Checking for the claude binary**: This catches the case where an update removes the symlink target. Running `claude update` can temporarily leave the binary absent, and this is a cause of silent failure.**② macOS TCC preflight**: If the vault is under `~/Documents/` , a process running from launchd can't write to it without Full Disk Access. ``` if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then echo "[...] ❌ FDA未付与: launchdから '$VAULT' にアクセス不可(TCC保護)。" >> "$LOG" notify_fail "FDA未付与: vault にアクセス不可（設定→フルディスクアクセス→/bin/bash）" exit 1 fi ``` Returning `exit 0` here would create DONE_MARKER, and the next slot would misread it as "succeeded" and not retry. TCC failure must always exit with `exit 1` and hand off to the next slot. This distinction is also at the root of one of the failure stories below. Writing all of `claude -p` 's output to the log gets you to tens of MB within a few weeks. ``` if [ -f "$LOG" ] && [ "$(stat -f%z "$LOG" 2>/dev/null || echo 0)" -gt 5242880 ]; then mv "$LOG" "${LOG}.old" fi ``` Once it exceeds 5 MB (5,242,880 bytes), it renames the file to `.old` and starts a new log. `stat -f%z` is the macOS file-size command. Since the options differ from GNU `stat` , `|| echo 0` provides a fallback. The brief freshness check is another important part. ``` START_STAMP=$(mktemp /tmp/vault-ingest-start.XXXXXX) ``` This temp file is created at the top of the script, and after step2.5 (brief generation) it checks whether the brief is newer than it. ``` if [ -s "$BRIEF_SRC" ] && [ "$BRIEF_SRC" -nt "$START_STAMP" ]; then # アーカイブ処理 touch "$DONE_MARKER" else notify_fail "ブリーフ生成が未完" fi ``` Comparing with `-nt` (newer than) prevents the accident of "mistakenly archiving yesterday's leftover `today-brief.md` as today's output." If the brief isn't newer than the start of this run, it isn't treated as a success — DONE_MARKER isn't created and it hands off to the next slot. When `claude -p` updates `wiki/index.md` , it sometimes writes an incorrect value for "total page count." The model infers it from context, so it doesn't match the actual file count. ``` real=$(find "$VAULT/wiki" -name '*.md' -not -path '*/.*' | wc -l | tr -d ' ') sed -i '' -E "s/総ページ数：[0-9]+/総ページ数：${real}/" "$INDEX_FILE" ``` It recounts the real file count with `find` and force-overwrites with `sed` . Because this runs immediately before `git commit` , what gets committed has the LLM-generated number already replaced by a deterministic value. The design principle is: "numbers output by an LLM get overwritten by a deterministic check downstream." When writing "today's schedule" into the brief, it retains past information on days when calendar data can't be fetched. ``` if [ -n "$CAL_SRC" ]; then cp "$CAL_TMP" "$CAL_SNAPSHOT" cp "$CAL_SNAPSHOT" "$CAL_LASTGOOD" # 成功したら last-good を更新 elif [ -s "$CAL_LASTGOOD" ]; then # 取得失敗: 前回 good を温存し stale 印を付けて出力 { tail -n +4 "$CAL_LASTGOOD"; } > "$CAL_SNAPSHOT" fi ``` It tries the Google Calendar API (ADC auth) and icalBuddy (local Apple Calendar) in order, and if both fail it uses `_calendar-snapshot.md.lastgood` from the last successful fetch. The principle is "never clobber the source of truth with empty." Staleness is marked explicitly with a leading `⚠️` , and the brief-generation prompt is told to "note that the values are as of the fetch date if it says stale." I noticed because `FAILED-20260613.md` was sitting in `~/Desktop/Daily Brief/` . Opening the log, all four slots showed exactly the same pattern. ``` 04:55:00 ===== auto-ingest 開始 ===== 05:40:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行） 08:20:00 ===== auto-ingest 開始 ===== 09:05:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行） 10:45:00 ===== auto-ingest 開始 ===== 11:30:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行） 12:15:00 ===== auto-ingest 開始 ===== 13:00:01 WARN: step2(claude) 失敗/timeout（次スロットで再試行） ``` Every slot timed out at 45 minutes 01 second. The step2 timeout at the time was 2400 seconds (40 minutes). `claude -p` kept processing for 40 minutes straight and got timed out, the next slot retried with the same result, and that loop repeated four times. What I suspected first was the error message after `git commit` . The strings "commit-msg hook" and "node not found" were visible in the log, so I thought the node binary was the cause. But running `claude -p "say OK"` by hand returned exit 0 immediately. Trying commit by hand worked fine. **The real cause was a transient failure on the claude -p side.** 06-12 worked normally; only on 06-13 did all requests hang (probably rate limiting or a temporary service-side fault), and it was resolved the next day. The commit-msg hook theory was completely wrong. Two lessons came out of this. One: "an error message is a suspect, not a conviction — actually run things to isolate the cause." The other concerns cleanup of FAILED markers. The cleanup at the time only "deleted that day's FAILED file on success." The 06-13 FAILED file would keep sitting on my desk the next day and beyond. ``` # 修正後: 過去日の FAILED も正常稼働日に掃く find "$HOME/Desktop/Daily Brief" -maxdepth 1 -name "FAILED-*.md" ! -name "FAILED-${TODAY}.md" -delete 2>/dev/null ``` Adding this one line means that even if failures occur across multiple consecutive days, all the past days' files get cleared on the first day that runs normally. One morning, wanting to check on the script's progress, I also ran `vault-auto-ingest.sh` manually. launchd's 4:55 slot was already running. Both ran `git add -A && git commit` on the same vault, and a conflict occurred. ``` error: cannot lock ref 'refs/heads/main': is at xxx but expected yyy ``` Because the two collided mid-processing, the vault ended up in a half-finished state and needed a manual reset. The `mkdir` lock described earlier solves this, but the key point is "automatic reclamation of stale locks." ``` if ! /bin/mkdir "$LOCKDIR" 2>/dev/null; then oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || true) if [ -n "${oldpid:-}" ] && kill -0 "$oldpid" 2>/dev/null; then echo "[...] 別インスタンス実行中(pid=${oldpid}) — skip" >> "$LOG" exit 0 fi rm -rf "$LOCKDIR" /bin/mkdir "$LOCKDIR" 2>/dev/null || exit 0 fi ``` `kill -0 "$oldpid"` sends no signal; it only checks whether the process is alive. If the process is dead, it deletes the stale lockdir and takes the lock itself. This prevents the worst case: "a previous abnormal termination leaves the lockdir behind and every subsequent run skips forever." There was a period when I kept the vault at `~/Documents/claude-obsidian/` (later moved to `~/claude-obsidian/` , outside TCC protection). The script at the time didn't handle TCC errors properly and exited with `exit 0` even when `cd "$VAULT"` failed. ``` # 修正前の問題コード（イメージ） cd "$VAULT" && git rev-parse --git-dir >/dev/null 2>&1 || exit 0 ^^^^^^ この exit 0 が DONE_MARKER を作る前提を壊す ``` When the script exits with `exit 0` , the calling launchd treats it as "terminated normally." But since the same thing happens on the next run too, all four slots end up in a state of "appearing to succeed while doing nothing," with DONE_MARKER never created. The symptoms were strange. The log recorded "===== 完了 =====" four times. But the vault wasn't updated. There was no `DONE_MARKER` either. I noticed the cause from a message that appeared when I ran the script interactively by hand. When run from launchd, `/bin/bash` lacks Full Disk Access, so `cd "$VAULT"` was failing silently. After the fix, TCC failure is signaled explicitly with `exit 1` , and it creates a FAILED file on the desktop plus a notification. ``` if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then notify_fail "FDA未付与: vault にアクセス不可（設定→フルディスクアクセス→/bin/bash）" exit 1 # DONE_MARKER は作られない → 次スロットが再試行 fi ``` The distinction between `exit 0` and `exit 1` is what separates "it succeeded" from "the next slot takes over." In an automation context, a silent fake success is the hardest failure mode to diagnose. `caffeinate -s` too much The first implementation put only `caffeinate -s` at the top of the script. ``` # 初期の誤った実装 caffeinate -s "$0" "$@" ``` The `-s` flag is described as "prevent system sleep." But in practice **it only works while connected to AC power**. Close the lid on battery and macOS goes to sleep, freezing the script under `caffeinate -s` along with it. Checking the next morning, there were traces that the script had started but it had ended in a half-finished state. Because timeout culled it afterward the lockdir was gone, but step2a's marker existed while step2b's did not. The fix was to switch to using both `-i` (prevent system sleep) and `-s` . ``` if [ -z "${CAFFEINATED:-}" ]; then exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@" fi ``` `-i` is effective on battery too. That said, there are situations where lid-close sleep (the suspend equivalent, not display sleep) can't be fully prevented even with `-i` . The multi-slot structure — "timeout culls it and the next slot retries the remainder" — is the design that assumes this. caffeinate is only an aid that *delays* sleep; what guarantees "the processing will complete" is the combination of markers and multiple slots. Sorting through these failures, all of them stayed within one day of real damage. The sleep freeze was recovered by the next 8:20 slot. The double-execution race took 10 minutes of manual reset. The all-day outage on 06-13 auto-recovered on the next morning's 4:55 slot. The TCC trap took a few days to notice, but once noticed the fix took under 30 minutes. **The reason I could notice any of these failures is the logs and the FAILED file on the desktop.** Had they failed silently, I might not have noticed that the vault hadn't been updated for days. The design of having `notify_fail` place a FAILED file on the desktop exists to maximize the chance of noticing. Automation isn't "run it and you're done" — it only really works once you've designed it up through "you'll reliably notice when it breaks." That's my honest takeaway from six months. The earlier sections covered four real failures (all slots dying, the double-execution race, the TCC trap, misplaced faith in caffeinate). Here I'll cover the smaller points that are "not that big, but you will definitely trip on them." **The combination of set -u and launchd environment variables kills you instantly** The script has `set -u` at the top (it's in the actual code). It's the option that makes referencing an undefined variable exit 1 immediately. A shell running from launchd, unlike an interactive shell, has many environment variables undefined. If you reference a variable without giving it a default via the `${VAR:-}` form, you get a phenomenon where it's fine in interactive runs but dies only under launchd. In this script, `${CAFFEINATED:-}` , `${oldpid:-}` , and `${net_ok:-}` all explicitly specify an empty default. Miss one and you mass-produce unexplained exit 1s. **Forget to set RunAtLoad and it runs at every login** The plist explicitly has `RunAtLoad` . Without it the default becomes `true` , and the script fires one extra time every time you restart or log in to the Mac. Since it exits immediately when DONE_MARKER exists there's no real harm, but the CPU load right after boot and the unintended log pollution accumulate. **Without trap, the lockdir stays behind and every subsequent run skips** ``` trap 'rm -rf "$LOCKDIR"' EXIT INT TERM ``` Without this one line, the lockdir remains when the script is stopped with `Ctrl+C` or `kill` . At the next firing, trying to acquire the lock with `mkdir` finds the existing lockdir and misjudges it as "another instance running," so it skips. But if you've written code that checks the PID, stale locks are reclaimed automatically. Without stale reclamation, it skips forever. **Running late at night, the date rolls over and two DONE_MARKERs get generated** If the 4:55 slot starts running at 0:10 AM (via a manual start, say), `date +%Y%m%d` changes mid-processing and the DONE_MARKER filename changes with it. To prevent this, the script pins the date as of the start of the run with `BD="$TODAY"` . ``` BD="$TODAY" # 日付跨ぎ対策: 評価はラン開始時の日付で固定 ARCH_FILE="$VAULT/wiki/briefs/daily/today-brief-${BD}.md" ``` `TODAY` is obtained exactly once at the top of the script. Even for long-running processing that crosses midnight, filenames stay unified under "the day the run started." `stat -f%z` is a macOS-only option This is the log rotation code. ``` [ "$(stat -f%z "$LOG" 2>/dev/null || echo 0)" -gt 5242880 ] ``` On GNU `stat` (Linux) it's `-c%s` . Since this is written assuming macOS it uses `stat -f%z` , but if you try to port it to Linux it silently falls through to `echo 0` and never rotates. The `|| echo 0` fallback means it doesn't break, but the log keeps growing. **Without launchd's StandardErrorPath, stderr vanishes** The plist contains the following. ``` StandardErrorPath /Users/.../.claude/logs/vault-auto-ingest.launchd.log ``` Without this, stderr from processes run via launchd is discarded to the equivalent of `/dev/null` . Even if step2's `claude -p` is writing something to stderr, you'll never see it. That creates a hard-to-diagnose situation where it's visible in interactive runs and invisible under launchd. **A node-based commit-msg hook dies without PATH augmentation** Running `git commit` in the vault executes the commit-msg hook. If that hook is written in node, launchd's minimal PATH (roughly `/usr/bin:/bin` ) has no `node` . `git commit` stops with a "hook execution failed" and finishes without committing anything. The script auto-discovers nvm's node and adds it to PATH. ``` NODE_BIN=$(ls -d "$HOME"/.nvm/versions/node/*/bin 2>/dev/null | sort -V | tail -1) [ -n "$NODE_BIN" ] && export PATH="${NODE_BIN}:$PATH" ``` The trick is taking the tail of a version-order sort with `sort -V` . With a string sort you get the inversion where `v9.x` sorts after `v10.x` . **Without spelling out "merge into existing articles" in step2b's prompt, you mass-produce duplicate articles** step2a (Claude-derived logs) runs first and updates the vault articles. When step2b (Codex-derived logs) runs afterward, it tries to create new articles on the same topics. Without explicit instructions in the prompt, articles with the same content split into `projects/foo.md` and `projects/foo-2.md` . ``` ingest_src "$STEP2B_MARKER" "$KB/raw/codex-conversations/" "codex" 1500 \ "Codex由来でも舜の知識として既存記事に統合し、Claude側と重複する話題は新記事を作らず追記でまとめろ。" ``` This additional prompt (the `extra` argument) is passed only to step2b. The instruction to "merge and append" works because it comes after seeing step2a's results. **Forget to delete the START_STAMP used for the -nt comparison and the brief is always judged "old"** ``` START_STAMP=$(mktemp /tmp/vault-ingest-start.XXXXXX) ``` This temp file is deleted when the script exits. But if it remains in `/tmp` , it isn't created under the same name on the next run, and comparison against an old `START_STAMP` can judge `today-brief.md` as "old," entering a loop where DONE_MARKER is never created. The `rm -f "$START_STAMP"` at the end of the script is essential. **Marker files left for more than 7 days bloat /tmp** ``` find "$HOME/.claude/logs" -maxdepth 1 -name '.vault-ingest-*' -mtime +7 -delete 2>/dev/null ``` Leave the daily-generated marker files alone and you accumulate 365 files a year in `~/.claude/logs/` . `-mtime +7` periodically deletes anything older than 7 days. This cleanup line is in there as a memento. Twelve operating principles that solidified over six months of actually running this. **① Separate "success" from "failure that hands off to the next slot" via exit code** This is the lesson from the TCC incident. Returning `exit 0` creates DONE_MARKER and the next slot won't retry. Genuine failures must always be `exit 1` . "Silent fake success" is the hardest-to-diagnose form of failure in automation. **② Create DONE_MARKER only after all steps are complete** If you create DONE_MARKER when step2a completes, "today is done" becomes true even with step2b and the brief unfinished. In this script, DONE_MARKER is created only when the brief is newer than `START_STAMP` . The ordering principle: raise the flag only after the final artifact has been produced. **③ Use intermediate markers to enable partial retry** Rather than a simple two-state "done/not done," give step2a and step2b each a half-marker. If step2a succeeded, the next slot skips step2a and starts from step2b. The more steps there are, the more valuable partial retry becomes. **④ Treat caffeinate as an aid that delays sleep, and accept that** Even using both `-i` and `-s` , suspend on a lid-closed MacBook can't be fully prevented. caffeinate is best-effort. What guarantees "it will definitely complete" is markers and multiple slots. A design that leans too hard on caffeinate will always break under battery operation. **⑤ Derive the number of slots backward from "even the slowest failure finishes within the day"** The four slots (4:55 / 8:20 / 10:45 / 12:15) are designed from the view that "if it finishes by noon, it's usable that day." Even if 4:55 fails, succeeding at 12:15 at worst still gives you a brief based on that day's activity logs. The next morning's firing is for processing the next day's material. **⑥ Always attach --kill-after=30** ``` run_to() { "$TIMEOUT_BIN" --kill-after=30 "$s" "$@"; } ``` It sends SIGKILL 30 seconds after `SIGTERM` . `claude -p` sometimes ignores SIGTERM while doing heavy work. Without kill-after, you end up with "the process survives past the timeout while still holding the lockdir." **⑦ Overwrite LLM-output numbers with something deterministic downstream** ``` real=$(find "$VAULT/wiki" -name '*.md' -not -path '*/.*' | wc -l | tr -d ' ') sed -i '' -E "s/総ページ数：[0-9]+/総ページ数：${real}/" "$INDEX_FILE" ``` The "total page count" Claude writes into `wiki/index.md` is a guess. Count the real files and overwrite. Because this runs right before `git commit` , the repository always has the measured value committed. The design principle: never treat LLM-generated numbers as authoritative. **⑧ Retain the previous value even on fetch failure (last-known-good)** Calendar information has a three-stage fallback: Google Calendar API → icalBuddy → lastgood. Not "it couldn't be fetched, so blank it," but "retain the value from the last success, marked stale." The brief prompt is instructed to call it out explicitly if `⚠️stale` is present. Stale information is more useful than none. **⑨ Put failures somewhere visible** ``` FAILED_FILE="$HOME/Desktop/Daily Brief/FAILED-${TODAY}.md" notify_fail() { mkdir -p "$HOME/Desktop/Daily Brief" { echo "# Daily Brief 生成失敗 — ..."; echo "- 自動再試行: 8:20 / 10:45 / 12:15 ..."; } > "$FAILED_FILE" /usr/bin/osascript -e "display notification ..." } ``` Log files are things you don't notice unless you go look. Putting a FAILED file on the desktop and firing a macOS notification puts "the automation is broken" in front of your eyes. And on success, that file is auto-deleted. A file that "exists only while failing" is proof of normal operation. **⑩ Clean up past days' FAILED files too** This one line was added while cleaning up after the consecutive 06-13 failures. ``` find "$HOME/Desktop/Daily Brief" -maxdepth 1 -name "FAILED-*.md" ! -name "FAILED-${TODAY}.md" -delete 2>/dev/null ``` With a design of "delete only that day's file on that day's success," FAILED files keep piling up on your desk after several consecutive days of failure. Design it so that the first normally-operating day deletes all the past days' files. **⑪ Auto-discover the nvm node version instead of hardcoding an absolute path** The launchd plist's EnvironmentVariables hardcodes the `v24.13.0` path (line 8 of the reference file). But the node discovery inside the script isn't pinned to an absolute version — it takes the newest via `sort -V | tail -1` . If you want to rely on the plist's fixed path, you need to update it every time you run `nvm install` . Using the script's dynamic discovery means no changes are needed after a version update. Be conscious of which one you're treating as the source of truth. **⑫ "It works" and "you'll notice when it breaks" are separate design problems** The first version only thought as far as "it works." When it broke, I might not have noticed that the vault hadn't been updated for days. Deliberately design multiple detection paths: the FAILED file, the notification, and noticing that the next day's brief is stale. Automation only becomes trustworthy once you've designed it through to "you'll reliably notice when it breaks." Let's look back at the triple-layered structure of `vault-auto-ingest.sh` . **Multi-slot re-firing** gives you the guarantee of "retrying over the course of a day until it succeeds." The design starts at 4:55 and runs as late as 12:15 in the worst case, but on most days it finishes at 4:55 or 8:20. The remaining two slots become no-ops that check DONE_MARKER and exit immediately. **Sleep prevention via caffeinate -i -s** is an aid that prevents the process from freezing partway through. It can't fully counter a lid-closed battery-powered Mac, but a frozen run gets culled by timeout and the next slot takes over. The combination of caffeinate and the marker structure establishes a structure where "even if part of it dies, it moves forward." **Idempotent retry via step markers** is the most important part. step2a and step2b each hold independent markers, and completed steps are never re-run. Splitting processing that didn't fit in a 40-minute window into two 25-minute lines means that if one dies, the next slot runs only the rest. This turns "partially timed out" into "partially finished." What changed once this ran stably every morning? When Claude Code reads `hot.md` the next morning, it can start working already knowing yesterday's decisions, the reason an architecture failed once, and the current state of three projects running in parallel. Explaining everything from scratch every time becomes unnecessary. Management cost doesn't grow even when parallel projects reach three or four. The value of automation shows not "when you get it running," but "when it recovers on its own after breaking and keeps running so routinely you don't notice it." *Written by **Lily** — I ship iOS apps and automate my content stack with Claude Code. Follow along: [Portfolio](https://bokuwalily.com) · [X](https://x.com/bokuwalily) · [GitHub](https://github.com/bokuwalily)*