cd /news/developer-tools/it-can-die-in-its-sleep-self-healing… · home topics developer-tools article
[ARTICLE · art-64928] src=dev.to ↗ pub= topic=developer-tools verified=true sentiment=· neutral

It Can Die in Its Sleep — Self-Healing launchd Jobs with Multi-Slot Firing and a Done-Marker

A developer created a self-healing launchd job pattern for macOS that uses multiple scheduled firing slots and done-markers to ensure a daily Obsidian Vault update completes despite failures from sleep, network loss, or API timeouts. The job fires at 4:55, 8:20, 10:45, and 12:15 each day, with each slot checking a date-stamped marker file; if the day's run already succeeded, the slot exits immediately, otherwise it retries from the last completed step. The approach uses half-markers per sub-step to avoid redoing work, so a failure in step2b (Codex logs) only retries that step on the next slot.

read7 min views1 publishedJul 19, 2026

My previous piece, "Making a launchd Job Unload Itself," built a job that runs exactly once and then unloads itself. This time it's the mirror image: a pattern designed around the assumption that the job will die mid-run — it fires several slots a day and delivers "retry until it succeeds, then quit immediately on success."

Every morning I hand Claude Code the task of updating my Obsidian Vault, and it kept dying partway through — killed by macOS sleep, no network on wake, or a claude timeout. Instead of trying to prevent every failure perfectly, I decided "it can die overnight as long as it's done by the time I wake up" was the more realistic goal, and I redesigned around that.

There are three ways a launchd job fails to run to completion.

caffeinate -s

only works on AC power. On battery, the job freezes the instant you close the lid, and gets reaped by timeout after wake.git push

and claude's API calls time out.All three can look like "the job started, exit code 0," so you notice late.

The fix is simple: stuff multiple StartCalendarInterval entries into the plist, and at the top of the script check "if today's run already succeeded, exit 0 immediately."

<!-- com.shun.vault-auto-ingest.plist (StartCalendarInterval excerpt) -->
<key>StartCalendarInterval</key>
<array>
    <dict>
        <key>Hour</key><integer>4</integer><key>Minute</key><integer>55</integer>
    </dict>
    <dict>
        <key>Hour</key><integer>8</integer><key>Minute</key><integer>20</integer>
    </dict>
    <dict>
        <key>Hour</key><integer>10</integer><key>Minute</key><integer>45</integer>
    </dict>
    <dict>
        <key>Hour</key><integer>12</integer><key>Minute</key><integer>15</integer>
    </dict>
</array>

4:55 is the ideal start time. If that one succeeds, the later three slots (8:20 / 10:45 / 12:15) all see the done-marker and finish in three seconds.

Here's the check at the top of the script.

TODAY=$(date +%Y%m%d)
DONE_MARKER="$HOME/.claude/logs/.vault-ingest-done-${TODAY}"

[ -f "$DONE_MARKER" ] && exit 0

The done-marker is only touch

ed when the final step succeeds. A run that dies partway never sets the marker, so the next slot redoes the whole thing.

A single claude ingest can take 25+ minutes. When you retry on the next slot, repeating already-successful steps is wasteful. So I placed a "half-marker" per step.

STEP2A_MARKER="$HOME/.claude/logs/.vault-ingest-step2a-claude-${TODAY}"
STEP2B_MARKER="$HOME/.claude/logs/.vault-ingest-step2b-codex-${TODAY}"
STEP2_MARKER="$HOME/.claude/logs/.vault-ingest-step2-done-${TODAY}"

The function that calls the ingest is written like this.

ingest_src() {
  local marker="$1" src="$2" name="$3" to="$4" extra="$5"
  [ -f "$marker" ] && {
    echo "[$(date '+%F %T')] step2($name) は本日実施済み — skip" >> "$LOG"
    return 0
  }
  cd "$VAULT" && run_to "$to" "$CLAUDE" -p "..." \
    --dangerously-skip-permissions >> "$LOG" 2>&1 \
    && { touch "$marker"; echo "[$(date '+%F %T')] step2($name) 完了" >> "$LOG"; return 0; } \
    || { echo "[$(date '+%F %T')] WARN: step2($name) 失敗/timeout" >> "$LOG"; return 1; }
}

ingest_src "$STEP2A_MARKER" "$KB/raw/conversations/"       "claude" 1500 ""
ingest_src "$STEP2B_MARKER" "$KB/raw/codex-conversations/" "codex"  1500 "..."

[ -f "$STEP2A_MARKER" ] && [ -f "$STEP2B_MARKER" ] && touch "$STEP2_MARKER"

step2a (Claude logs) succeeds → step2b (Codex logs) times out → next slot fires → step2a is skipped → only step2b is retried. Once both are done, STEP2_MARKER

goes up and the pipeline moves to the next stage. The half-markers reset naturally the next day thanks to the ${TODAY}

suffix.

Note

The step split came out of a real incident. Trying to process 28h of Claude logs + Codex logs in one shot overran the 40-minute window and timed out three days in a row → hot.md froze (2026-06-11 to 13). Splitting into two per-source runs, each capped at a 25-minute (1500-second) timeout, made it stable.

The pattern where the script re-executes itself under caffeinate.

if [ -z "${CAFFEINATED:-}" ]; then
  exec /usr/bin/caffeinate -i -s env CAFFEINATED=1 /bin/bash "$0" "$@"
fi

Because exec

replaces the process with itself, caffeinate becomes the process's own parent rather than a child. Putting CAFFEINATED=1

in the environment prevents an infinite re-invocation.

Warning

-s

(prevent system sleep on AC power) isinactive on battery. Close the lid on battery and it freezes. That can't be fully prevented, so the design is "a frozen run gets reaped by timeout and the next slot redoes it."-i

(prevent idle sleep) works regardless of power source, but doesn't stop lid-close.

Move right after wake and WiFi isn't up yet. git push

and claude's API calls fail immediately.

net_ok=""
for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18; do
  if /usr/bin/nc -z -G 3 1.1.1.1 443 2>/dev/null; then net_ok=1; break; fi
  sleep 5
done
[ -z "$net_ok" ] && echo "[$(date '+%F %T')] WARN: 網未接続のまま続行(失敗時は次スロットが再試行)" >> "$LOG"

18 iterations × 5 seconds = up to 90 seconds of waiting. It breaks as soon as it confirms a connection. If it still isn't connected after 90 seconds, it just emits a warning and continues. It doesn't exit here because in an offline environment step1 (local log conversion) alone can still run. If the later git push

or claude fails, the next slot retries.

The -G

in nc -z -G 3

specifies macOS's connect timeout (in seconds); note that GNU nc uses different flags.

Under launchd, ~/Documents/

is protected by TCC (Full Disk Access): unless you grant /bin/bash

full disk access from the GUI, you can't read or write it. If it silently ends with exit 0, you won't notice for a week. Detect it explicitly at the top.

if ! ( cd "$VAULT" 2>/dev/null && git rev-parse --git-dir >/dev/null 2>&1 ); then
  echo "[$(date '+%F %T')] ❌ FDA未付与: launchdから '$VAULT' にアクセス不可(TCC保護)。" >> "$LOG"
  echo "    解決: システム設定 > プライバシーとセキュリティ > フルディスクアクセス で /bin/bash を許可。" >> "$LOG"
  notify_fail "FDA未付与: vault にアクセス不可(設定→フルディスクアクセス→/bin/bash)"
  exit 1
fi

Whether git rev-parse --git-dir

succeeds tells you whether you have access to the Vault. On failure it exit 1

s (without setting the done-marker) → the next slot retries → once FDA is granted it resolves naturally.

Note

iCloud-synced folders (under~/Documents/

) are especially heavily protected. Escaping to~/

via a symlink is a no-go, because iCloud's conflict handling breaks the symlink (verified 2026-06-01). Keep the Vault directly under~/Documents/

.

A failure you only notice if you go read the log gets ignored. On failure, drop a FAILED-${TODAY}.md

on the desktop, and auto-delete it on success.

notify_fail() {
  local reason="$1"
  mkdir -p "$HOME/Desktop/Daily Brief"
  { echo "# Daily Brief 生成失敗 — $(date '+%F %T')"
    echo "- 理由: ${reason}"
    echo "- 詳細ログ: ~/.claude/logs/vault-auto-ingest.log"
    echo "- 自動再試行: 8:20 / 10:45 / 12:15(成功したらこのファイルは自動で消える)"
  } > "$FAILED_FILE"
  /usr/bin/osascript -e "display notification \"${reason}\" with title \"Daily Brief 生成失敗\"" >/dev/null 2>&1
}

Only when step2.6 (archiving the brief) succeeds does it rm -f "$FAILED_FILE"

. "The file disappears only on a day that succeeded" = "if the file is still there, it's still failing."

A race where the launchd version and a manual version ran at the same time actually happened (2026-06-10). Exclude it with an mkdir

-based lock.

LOCKDIR="$HOME/.claude/locks/vault-auto-ingest.lock"
if ! /bin/mkdir "$LOCKDIR" 2>/dev/null; then
  oldpid=$(cat "$LOCKDIR/pid" 2>/dev/null || true)
  if [ -n "${oldpid:-}" ] && kill -0 "$oldpid" 2>/dev/null; then
    echo "[$(date '+%F %T')] 別インスタンス実行中(pid=${oldpid}) — skip" >> "$LOG"
    exit 0
  fi
  rm -rf "$LOCKDIR"
  /bin/mkdir "$LOCKDIR" 2>/dev/null || exit 0
fi
echo $$ > "$LOCKDIR/pid"
trap 'rm -rf "$LOCKDIR"' EXIT INT TERM

A stale lock (a lock dir left behind by a dead process) is auto-reclaimed via a pid liveness check (kill -0

). Because mkdir

is atomic, even if multiple processes hit it simultaneously, only one succeeds.

-i

alone doesn't stop lid-close.git rev-parse

preflight. A silent failure went unnoticed for a week.stat -f%z

and rotate to .old

when it's over (dumping claude's entire output into the log makes it bloat surprisingly fast).TIMEOUT_BIN=/opt/homebrew/bin/timeout

first, and pass through if absent (brew install coreutils

is a prerequisite).StartCalendarInterval

entries in the plist, and caffeinate -i -s

isn't a silver bullet. It can't prevent a battery lid-close freeze, so design it so the next slot takes over a frozen run.~/Documents/

. Confirm access from launchd with a git rev-parse

preflight to prevent silent failures.Combined with the previous piece, "Making a launchd Job Unload Itself," you now have two patterns: "a job you want to run to completion exactly once" and "a job you want to reliably complete once a day." Next time I plan to write about how to efficiently inject the Vault this job keeps building up into Claude Code's context.

*Written by Lily — I ship iOS apps and automate my content stack with Claude Code.

Follow along: Portfolio · X · GitHub*

── more in #developer-tools 4 stories · sorted by recency
── more on @claude code 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/it-can-die-in-its-sl…] indexed:0 read:7min 2026-07-19 ·