For 30 straight days as a college student earning ¥100k/month, I posted to Instagram by hand, and then I burned out and stopped. Today the same job runs on a Claude Code autonomous environment, I touch nothing, and it holds up ¥1.2M/month in revenue. Except for the two hours when it quietly stopped: three consecutive launchd runs, zero pieces of content generated, last exit=0
every single time, and not one alert. The cause was a process that had already been killed, holding a lock file nobody would take away from it.
The problem with updating social media by hand is that it burns willpower. No matter how motivated you are, sleep, health, and mood all fluctuate. During the period when I was laid off and my income went to zero, I had no mental slack for posting at all.
The autonomous environment I spent six months building with Claude Code runs regardless of my emotional state. launchd calls a script, the script generates content with claude -p
(MAX plan quota; paid APIs are off-limits), the output is queued for auto-posting, and it goes out to Instagram every day at 19:30. As long as this machinery keeps working, ¥1.2M/month in sales holds up without me lifting a finger.
A lot of people think "automation = writing scripts," and that's only half right. A script is correct at the moment you write it. Given time, external dependencies break, processes die for reasons you didn't anticipate, and lock files turn into debris that blocks every future run.
An autonomous environment that actually works is one that assumes breakage and carries a layer that repairs it. The lock story here is a textbook case.
~/dev/brand-404/sns/gen_feature.py
is a script launched on a schedule by launchd that auto-generates Instagram feature articles. A single run takes a long time (up to three claude -p
calls, plus image generation, adding up to tens of minutes), so it has a lock mechanism to prevent the next run from overlapping with one that hasn't finished.
Mishandling that lock file (~/dev/brand-404/sns/gen_work/.lock
) meant that pid 94799 held the lock without releasing it even though it had already been killed, every subsequent run was skipped with "lock held — 終了", and generation stopped for about two hours.
The lock check has a constant LOCK_STALE_SEC = 2 * 3600
(2 hours) (line 57 of gen_feature.py). It's a safety valve: "even if the lock exists, steal it if the mtime is older than 2 hours."
But the original implementation only looked at mtime, without checking whether the pid was alive. Even with a killed pid sitting in .lock
, as long as the mtime was within 2 hours it kept deciding "still running" and skipping. launchd could fire a third and a fourth time — all of them "lock held — 終了" until the two hours elapsed.
The worst possible state — automation that is "running" but produces nothing — continued silently for two hours.
With manual work, you notice: "huh, nothing got generated today," and you run it by hand. But building an autonomous environment means taking on responsibility for it continuing to work correctly while nobody is watching.
A pid liveness check looks like belt-and-suspenders, but in practice "the script gets killed and the lock stays behind" happens routinely. If you kill it instantly with SIGKILL, the release_lock()
in the finally
block never runs at all. Same thing when you stop it by hand during development. If you're scheduling a long-running script with launchd, a pid liveness check is mandatory.
launchd (毎日定時 + 毎日19:30)
│
├─ gen_feature.py（毎日定時）
│ acquire_lock() ← 今回の話
│ ↓
│ キュー残数チェック
│ QUEUE_TARGET=3 に不足があれば
│ ↓
│ brand-catalog.json からブランド選定
│ ↓
│ Shopify /products.json 取得
│ ↓
│ claude -p #1: コピー生成 (copy.md)
│ ↓
│ claude -p #2: 画像役割選定 (images.md)
│ ↓
│ スライド生成 (build-post-from-json.mjs)
│ ↓
│ claude -p #3: セルフQA (qa.md)
│ ↓ QAを通過
│ content/sns/feature-XX-{slug}/ に出力
│ release_lock()
│
└─ ig_autopost.py（毎日19:30）
content/sns/feature-* のキューから1本取り出し
↓
Instagram Graph API で投稿
↓
state/ig_posted.jsonl に記録
The queue targets a standing inventory of 3 items (gen_feature.py line 60, QUEUE_TARGET = 3
; line 61, MAX_GEN_PER_RUN = 2
). With 3 in stock, posting doesn't break even if generation fails for one or two days in a row.
When you manage a script with a long single run (up to CLAUDE_TIMEOUT = 600
seconds × 3 calls, plus image downloads) via scheduled launchd starts, these problems appear:
brand-catalog.json
corrupt itgen_work/.lock
is what prevents that. The process writes its own PID into the lock file, and on the next start the script checks whether that PID is alive before deciding whether to run.
Lines 225–249 of gen_feature.py
are the whole thing.
LOCK_STALE_SEC = 2 * 3600 # 57行目
def acquire_lock() -> bool:
GEN_WORK.mkdir(parents=True, exist_ok=True)
if LOCK_FILE.exists():
age = time.time() - LOCK_FILE.stat().st_mtime
try:
pid = int(LOCK_FILE.read_text(encoding="utf-8").strip())
if pid <= 0:
raise ValueError
except (OSError, ValueError):
log("stale lock 奪取: PIDが空または非数値")
else:
try:
os.kill(pid, 0) # ← プロセス生死チェック
except ProcessLookupError:
log(f"stale lock 奪取: pid={pid} は不在")
except PermissionError:
if age <= LOCK_STALE_SEC:
return False
log("stale lock 奪取: mtime 2h超")
else:
if age <= LOCK_STALE_SEC:
return False
log("stale lock 奪取: mtime 2h超")
LOCK_FILE.write_text(str(os.getpid()), encoding="utf-8")
return True
The decision logic before the fix (the broken code) only looked at mtime.
def acquire_lock() -> bool:
GEN_WORK.mkdir(parents=True, exist_ok=True)
if LOCK_FILE.exists():
age = time.time() - LOCK_FILE.stat().st_mtime
if age <= LOCK_STALE_SEC:
return False # ← kill済みpidでも2h以内は全部ここで返る
log("stale lock 奪取: mtime 2h超")
LOCK_FILE.write_text(str(os.getpid()), encoding="utf-8")
return True
os.kill(pid, 0)
sends signal 0 (the null signal). Signal 0 doesn't actually send anything. The kernel just checks the pid and returns: success if the process exists and you have permission to signal it, ProcessLookupError if the process doesn't exist, and PermissionError if the process exists but belongs to another user.
Using this behavior, you can safely check whether a process is alive.
| Result of os.kill(pid, 0) | Meaning | acquire_lock's decision |
|---|---|---|
| No exception | The process for that pid exists and is running | If mtime is within 2h, treat as "running" and skip |
| ProcessLookupError | The process for that pid is gone (killed, etc.) | Stale lock — steal it immediately |
| PermissionError | The pid exists but is owned by another user | mtime fallback (steal if older than 2h) |
In this incident, where pid 94799 had been killed, the fixed code would raise ProcessLookupError
, immediately log "stale lock 奪取: pid=94799 は不在"
, and let the next runner take the lock.
The same pattern is used in the multi-start guard of ~/.claude/scripts/automation-health.sh
(lines 20–30).
_ah_lock="${TMPDIR:-/tmp}/automation-health.lock"
if ! mkdir "$_ah_lock" 2>/dev/null; then
if kill -0 "$(cat "$_ah_lock/pid" 2>/dev/null)" 2>/dev/null; then
echo "automation-health: 別インスタンス稼働中のためスキップ" >&2
exit 0
fi
rm -rf "$_ah_lock"
mkdir "$_ah_lock" 2>/dev/null || { echo "lock取得失敗・スキップ" >&2; exit 0; }
fi
echo $$ > "$_ah_lock/pid"
trap 'rm -rf "$_ah_lock"' EXIT
The comments (lines 17–20) spell out the reasoning: "--deep
scans everything, so I/O piles up and load average spikes if multiple instances run. An atomic mkdir
lock narrows it to one. The PID liveness check auto-steals stale locks, and trap EXIT
guarantees release on both normal and abnormal termination."
The shell's kill -0 <pid>
has exactly the same semantics as Python's os.kill(pid, 0)
. 2>/dev/null
discards stderr so the "no such process" error message isn't shown to the user. If the process exists, the exit code is 0; if not, it's 1 — so the success or failure of if kill -0 ...
is the liveness check.
On top of that, trap 'rm -rf "$_ah_lock"' EXIT
guarantees lock release on normal exit, error exit, or signal receipt. It's the same idea as calling release_lock()
from Python's try/finally
.
Python (gen_feature.py) Shell (automation-health.sh)
───────────────────── ─────────────────────────────
os.kill(pid, 0) kill -0 <pid>
ProcessLookupError → 奪取 終了コード1 → 残骸 → 奪取
PermissionError → mtime fb 終了コード≠0→ 同上
例外なし → 実行中 終了コード0 → 実行中
try/finally release_lock() trap 'rm -rf lock' EXIT
Different languages, same three-step pattern for correct lock acquisition.
kill -0
/ os.kill(pid, 0)
mtime is strictly the last resort for when the pid check isn't usable — it must never be the primary check.
I've talked about acquire_lock()
, but the correct lock pattern has its crux on the release side too.
release_lock()
at lines 252–256 of gen_feature.py
is simple.
def release_lock() -> None:
try:
LOCK_FILE.unlink()
except FileNotFoundError:
pass
Swallowing FileNotFoundError
is deliberate. It's a guard against crashing when the finally
block and a manual call overlap, and it prevents the absurdity of "the main work raises an exception because releasing the lock failed."
What matters is the structure of the caller, main()
(lines 995–1020).
if not acquire_lock():
log("lock held（他プロセスが実行中 or 2h以内）— 終了")
return 0
try:
made = 0
attempts = 0
max_attempts = need + 3
attempted_brands: set[str] = set()
while made < need and attempts < max_attempts:
result = run_pipeline(attempted_brands)
...
log(f"生成完了: {made}/{need}本 試行{attempts}回 (queue残 {count_queue_remaining()}本)")
finally:
release_lock()
Everything after acquiring the lock is wrapped in try/finally
. Even if an exception is raised in the middle of run_pipeline()
, the finally
block always runs, so the lock is reliably released.
There's one exception where this doesn't work: SIGKILL. If you force-terminate the process with kill -9 <pid>
, the Python runtime itself dies instantly, so the finally
block never runs. That was the direct cause of this incident.
The same goes for trap EXIT
(automation-health.sh
line 30).
trap 'rm -rf "$_ah_lock"' EXIT
The EXIT
trap runs on SIGTERM
(a normal kill), normal exit, and error exit — but not on SIGKILL. In either language, lock debris from a forced termination can't be prevented by try/finally
and trap EXIT
alone. That's exactly why you need the liveness check via os.kill(pid, 0)
.
The os.kill(pid, 0)
call in acquire_lock()
has three branches (lines 235–247).
try:
os.kill(pid, 0)
except ProcessLookupError:
log(f"stale lock 奪取: pid={pid} は不在")
except PermissionError:
if age <= LOCK_STALE_SEC:
return False
log("stale lock 奪取: mtime 2h超")
else:
if age <= LOCK_STALE_SEC:
return False
log("stale lock 奪取: mtime 2h超")
** ProcessLookupError** is the case where the process doesn't exist. This incident (a killed pid) falls here. Steal immediately. There's no need to look at mtime.
** PermissionError** is the case where the pid exists but belongs to another user's process. Since you lack permission to send a signal, you can confirm existence but not whether it's "a previous instance of my own script." In that situation you have no choice but to fall back on mtime, so it leans toward "treat as running" if it's within two hours.
No exception (the else block) means the process exists and you do have permission to signal it — that is, it is definitely still running. Here too, if mtime is within two hours, it's treated as a legitimate in-progress run.
LOCK_STALE_SEC = 2 * 3600
is purely a last-resort safety valve. It only kicks in for the PermissionError
case where the pid check doesn't apply, and for the else
case where the process is running but is taking unexpectedly long for some reason. This connects back to what I said earlier about never making mtime the primary check. Whether two hours is the right threshold isn't the essence of the problem; doing the pid check first is.
The case where the lock file exists but its contents are corrupt is handled too (lines 229–234).
try:
pid = int(LOCK_FILE.read_text(encoding="utf-8").strip())
if pid <= 0:
raise ValueError
except (OSError, ValueError):
log("stale lock 奪取: PIDが空または非数値")
OSError
is when the file couldn't be read (permission issues, etc.), and ValueError
is when the int conversion fails or the pid is 0 or below. Both are treated as "undecidable = debris" and the lock is stolen.
The pid <= 0
check is validation that leans on the POSIX guarantee that a PID is always a positive integer. int("0")
and int("-1")
convert successfully but aren't valid values for a real process PID. If strip()
on an empty file yields an empty string, int("")
raises ValueError
, which falls into the same path.
automation-health.sh
implements its lock with mkdir
(line 22) because that's more atomic than writing a file.
if ! mkdir "$_ah_lock" 2>/dev/null; then
if kill -0 "$(cat "$_ah_lock/pid" 2>/dev/null)" 2>/dev/null; then
echo "automation-health: 別インスタンス稼働中のためスキップ" >&2
exit 0
fi
rm -rf "$_ah_lock"
mkdir "$_ah_lock" 2>/dev/null || { echo "lock取得失敗・スキップ" >&2; exit 0; }
fi
echo $$ > "$_ah_lock/pid"
mkdir
is atomic at the kernel level. If the directory doesn't exist it's created and succeeds; if it exists it fails. Even if two processes call mkdir
simultaneously, the kernel lets only one of them succeed.
Python's write_text()
carries no such guarantee. There is a theoretically possible race where A writes first and B overwrites immediately after. In the case of gen_feature.py
, launchd starts it on an interval so the probability of concurrent execution is extremely low — but not zero. If you want the same atomicity in Python, one approach is os.mkdir()
. The reason gen_feature.py
deliberately doesn't use that approach is that the high-frequency problem for this script wasn't "simultaneous start races" but "killed pids left behind." The simple implementation was chosen to match the priority of the problem that actually needed solving.
The first sign of trouble was in my Discord #brand-404
channel. The "🆕 o81 IG特集生成: ..." notification that had arrived daily up to the previous day didn't come.
Nothing came into the Discord alert channel either. The symptom wasn't "an error occurred" — it was "nothing happened."
Thinking the launchd job had failed, I ran automation-health.sh
, and com.lily.gen-feature
showed "loaded / last exit=0". exit 0 is a normal exit. But no generation was coming.
Digging through the logs, I found three hours' worth of these lines.
[gen] lock held（他プロセスが実行中 or 2h以内）— 終了
All three launchd starts ended right there. Someone was holding the lock.
cat
-ing ~/dev/brand-404/sns/gen_work/.lock
gave me:
94799
pid 94799 was written in it. ps aux | grep 94799
returned nothing. Not a live process.
Checking the mtime with ls -la ~/dev/brand-404/sns/gen_work/.lock
showed a timestamp about 1.5 hours old. Against LOCK_STALE_SEC = 2 * 3600
(2 hours), that was still 0.5 hours short. That's why it kept deciding "within 2h = running" and skipping.
pid 94799 had died because, during debugging the night before, I ran kill -9 $(cat ~/dev/brand-404/sns/gen_work/.lock)
in another terminal. Normally release_lock()
would run and .lock
would disappear. But -9
(SIGKILL) skips the finally
block. The lock stayed behind and swallowed every launchd start the next morning.
The first thing I thought of was "2 hours is too long, let's make it 30 minutes."
I actually changed it to LOCK_STALE_SEC = 30 * 60
and committed. That was wrong.
The problem wasn't that 2 hours is long — it was that a killed pid was being judged alive. Even at 30 minutes, the same problem persists for 30 minutes after the kill. Worse, it introduces the risk of stealing the lock in the middle of a legitimate long run (three claude -p
calls = up to CLAUDE_TIMEOUT = 600
seconds × 3, plus image downloads).
The correct fix is "check whether the pid is alive before looking at mtime," not changing the mtime threshold. Tuning the threshold isn't a cure; it just softens the symptom. I reverted that commit and switched to the current approach of checking os.kill(pid, 0)
first.
After putting the first fixed version — which only caught ProcessLookupError
— into production, I got stuck again during testing.
I ran sudo python3 ~/dev/brand-404/sns/gen_feature.py --dry-run
once for verification purposes, and when I then ran it as a normal user, the leftover-lock symptom appeared again.
A process run under sudo
is owned by root. Calling os.kill(root_pid, 0)
as a normal user returns PermissionError
. My first fix only looked at ProcessLookupError
, so PermissionError
propagated out and produced a stack trace.
try:
os.kill(pid, 0)
except ProcessLookupError:
log(f"stale lock 奪取: pid={pid} は不在")
else:
if age <= LOCK_STALE_SEC:
return False
In Python's try/except/else
structure, else
runs when no exception at all was raised in the try
block. Since PermissionError
wasn't caught, that case entered neither except ProcessLookupError
nor else
, and the exception propagated straight to the caller. In the end I caught PermissionError
explicitly and routed it to the mtime fallback, arriving at the current form (lines 241–243).
except PermissionError:
if age <= LOCK_STALE_SEC:
return False
log("stale lock 奪取: mtime 2h超")
What this burned into me hardest is that exit 0 and "generation completed" are different things.
log("lock held — 終了"); return 0
is exit 0. The launchd job history records it as "success." automation-health.sh
returns green too. No Discord alert arrives. From the outside everything is fine — and yet the actual goal, content generation, didn't happen at all.
This kind of silent failure can only be detected if you design monitoring around "did the intended side effect occur?" rather than "did the process die?"
I now have separate monitoring that fires a Discord alert if the remaining queue count (count_queue_remaining()
) stays at zero for a certain period. Process health checks and output health checks need to be designed separately. Making an autonomous environment "designed on the assumption of breakage" includes monitoring, in a separate layer, not only whether the process is alive but whether the environment keeps producing the output it's supposed to.
The earlier sections covered "the design flaw of only looking at mtime," "the fix via os.kill(pid, 0)," and "missing PermissionError." Here I'll add the points I actually got stuck on while running scripts with this same structure.
PID reuse causes a false "alive" verdict
PIDs on Linux/macOS are finite and wrap around. On macOS they cycle around a maximum near 99999. In a long-running environment, it's possible that the PID of a process you kill -9
'd yesterday is being used by an unrelated process today. Call os.kill(pid, 0)
in that state and you get "the process is alive." gen_feature.py
currently writes only the PID into .lock
(line 248, LOCK_FILE.write_text(str(os.getpid()), ...)
), but the combination of launchd running it once a day and LOCK_STALE_SEC = 2 * 3600
(line 57) means there's virtually no real harm. For scripts invoked at high frequency, consider writing the PID paired with a start timestamp and treating it as "running" only when both match.
Forget GEN_WORK.mkdir(parents=True, exist_ok=True) and the first run dies instantly
Line 226 of acquire_lock()
starts with GEN_WORK.mkdir(parents=True, exist_ok=True)
. Without that one line, on a first start where the gen_work/
directory doesn't exist, an operation before LOCK_FILE.exists()
raises FileNotFoundError
. launchd starts the script, it dies immediately, and you're left with last exit=1
and nothing running. Keep the order: create before releasing.
write_text()
is not atomic (a theoretical race exists)
With Python's Path.write_text()
, other processes can potentially read the mid-write state. If the next instance calls read_text()
before the LOCK_FILE.write_text(str(os.getpid()), ...)
write completes, it reads an empty string, int("")
raises ValueError
→ the lock is stolen as debris → two instances run simultaneously. That race is theoretically valid. Since gen_feature.py
is started intermittently by launchd, simultaneous starts essentially don't happen. But for a high-frequency script called every minute, use an atomic write with os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)
. This is exactly why automation-health.sh
implements its lock with mkdir
(line 22, if ! mkdir "$_ah_lock" 2>/dev/null;
).
SIGKILL isn't the only thing that skips finally
The middle section covered how SIGKILL (kill -9
) skips try/finally
. One more thing to watch for is a C extension module calling os._exit()
internally. sys.exit()
raises SystemExit
, so finally
runs; os._exit()
terminates the process along with the runtime immediately, so finally
does not run. The more you depend on external libraries, the higher the chance that finally
won't run. Designing the pid liveness check on the assumption that finally
won't run is the only fundamental countermeasure.
Building lock file paths from relative paths makes them environment-dependent
If the launchd plist has a WorkingDirectory
key set, the script's starting directory changes to it. Build the lock file path from relative paths and the lock file ends up in a different place depending on the plist's WorkingDirectory
. gen_feature.py
builds every path as an absolute path from ROOT = Path(__file__).resolve().parent.parent
(line 36). Scripts that run as launchd jobs should use __file__
-based absolute paths.
_draft-feature-XX-{slug}
directories accumulate
On QA failure or slide-generation failure, run_pipeline()
leaves the _draft-feature-XX-{slug}
directory in place instead of deleting it. That's an intentional design decision so failed content can be inspected later (lines 954–959). count_queue_remaining()
excludes anything with the _draft-
prefix from its count (lines 203–218), so inventory calculation isn't affected. But if they keep piling up, content/sns/
fills with junk. A monthly cleanup with find content/sns -name '_draft-*' -mtime +30 -type d
is needed.
Interpreting exit 0 as "normal" lets silent failures slip through
The main()
function also ends with return 0
when lock acquisition fails (lines 996–997). launchd records that as a normal exit, and the launchd check in automation-health.sh
displays last exit=0
as ✓
. "exit 0 because the queue was stocked and no generation was needed" and "exit 0 because lock acquisition failed and nothing was done" look identical to launchd. That's why the actual two-hour stoppage was invisible.
The trap of the "no error means everything's fine" design philosophy
Discord alerts only arrive for failure cases where discord_alert()
is called (lines 140–142). A failed lock acquisition is a skip, not an error, so no alert arrives. If you design automation monitoring purely around "notify me when something bad happens," silent skips will never be detected. What made me notice this incident in the first place was the absence of the success notification "🆕 o81 IG特集生成: ..." (discord_post_review()
, line 973) in the #brand-404
channel.
Discord's default User-Agent gets blocked by Cloudflare
As the comment in _discord_post()
notes (lines 93–94), Cloudflare rejects the Python-urllib/x.y
User-Agent that Python's urllib
sends by default with 403/1010. It's worked around by explicitly setting ua_header = "lily-o81-gen/1.0"
. If an automation script hits an external service with the default settings of urllib
or the requests
library, it can get blocked without warning.
Misjudging the balance between curl timeouts and retries
fetch_products()
retries Shopify's products.json
with attempts=3
(lines 503–522). The design of waiting 15 seconds and retrying on failure comes from real damage: "some stores temporarily return empty results under consecutive access" (Beyond The Vines, 2026-07-26). Make the timeout extremely short and you get "a brand that actually has products is judged to have 0 and demoted to skip." Make it too long and a single failure stalls things for minutes. Scripts that hit external APIs should design three things as a set: the per-attempt timeout, how many times to retry, and the wait between retries.
Building on the earlier sections and the gotchas, here are the practices for lock management in long-running launchd scripts, extracted from real code.
1. Fix the pid liveness check as the primary decision and mtime as the secondary one
try:
os.kill(pid, 0)
except ProcessLookupError:
pass # 即奪取
except PermissionError:
if age <= LOCK_STALE_SEC:
return False # mtime fallback
else:
if age <= LOCK_STALE_SEC:
return False # mtime fallback
The shell version has the same semantics with kill -0 "$(cat pid_file)"
(automation-health.sh
line 23).
2. Always implement the three exception branches as a set
If you catch only ProcessLookupError
, PermissionError
propagates out and crashes the script. Write all three as a set. Miss even one and you get "a bug specific to that case."
3. Always wrap lock release in try/finally
if not acquire_lock():
return 0
try:
finally:
release_lock()
In shell, trap 'rm -rf "$lock"' EXIT
is the equivalent. It releases on normal exit, on an exception, and on sys.exit()
.
4. Have release_lock() swallow FileNotFoundError
It's a guard so a second unlink()
doesn't crash when finally
and a manual call overlap. It prevents the absurdity of "the main work raises an exception because releasing the lock failed" (the pattern at lines 252–256).
5. Derive LOCK_STALE_SEC from the maximum run time
The longest run of gen_feature.py
is CLAUDE_TIMEOUT = 600
seconds (line 66) × up to 3 calls + image downloads, roughly 35–40 minutes. LOCK_STALE_SEC = 2 * 3600
(line 57) gives plenty of margin against that. Shortening the threshold does not solve the "a killed pid releases the lock within 2h" problem. Threshold tuning is symptomatic treatment; the pid liveness check is the cure.
6. Build the lock file path as an absolute path based on Path(file).resolve()
An implementation that doesn't depend on launchd's WorkingDirectory
is mandatory. The pattern of building every path from ROOT = Path(__file__).resolve().parent.parent
(line 36) in gen_feature.py
works as a template as-is.
7. Always run GEN_WORK.mkdir(parents=True, exist_ok=True) before acquiring the lock
It's required so the first start doesn't crash when the directory doesn't exist. Put it at the top of acquire_lock()
(line 226).
8. In shell, ensure atomicity with a mkdir lock
mkdir
is atomic at the kernel level. Even if two processes call it simultaneously, only one succeeds (automation-health.sh
lines 22–27). If you need atomicity in Python too, use os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)
.
9. Design monitoring around "the intended output," not the process
The launchd check in automation-health.sh
treats last exit=0
as normal, but a lock-acquisition skip is also exit 0. The right answer is positive monitoring: alert if the success notification "content generation completed" (discord_post_review()
, line 973) doesn't arrive within a certain window.
10. Have success notifications (don't rely on error notifications alone)
A design that only notifies on errors misses silent skips. The design where the absence of the "🆕 o81 IG特集生成: ..." success notification signals a problem was the only signal that made me notice this failure. Having a "something good happened" notification is what makes anomaly detection effective.
11. Keep several days' worth of queue inventory
The combination of QUEUE_TARGET = 3
(line 60) and MAX_GEN_PER_RUN = 2
(line 61) is an inventory design where "posting doesn't break even if generation fails one or two days in a row." Even if a lock failure stops things for a day, 3 items in stock keep the posting going. Having a buffer eliminates the single point of failure in automation.
12. Make it possible to check state without taking the lock, via a --dry-run option
gen_feature.py --dry-run
(lines 981–993) shows only the remaining queue count without taking the lock. It's safe to call even while production is running. During incident investigation you can check state without worrying about "going to grab the lock and causing contention."
13. Make it possible to run E2E tests against production logic with a --force option
Normally it only generates until QUEUE_TARGET
is met, but --force
generates one item regardless of the remaining queue count (lines 980–984). You can test the whole pipeline without changing production logic, and use it to verify a fix.
14. Always set an explicit User-Agent on urllib calls to external APIs
Python's default Python-urllib/x.y
gets blocked by Cloudflare and some CDNs. Specify an identifiable string like "lily-o81-gen/1.0"
. It's mandatory when hitting Discord, Shopify, or any other API behind Cloudflare.
pid 94799 occupied the lock for two hours because of one missing line. The check that confirms whether a process is alive with os.kill(pid, 0)
was absent, so a killed pid kept being misjudged as "still running." The fix itself is a few lines.
But what really needs to be understood is the "why." SIGKILL skips finally
. launchd's exit 0 doesn't mean "it worked correctly," only "the process terminated." Monitoring should be designed around "is the intended output occurring?" rather than process liveness. Those three points are the substance of what it means to make an autonomous environment "designed on the assumption of breakage."
The first, wrong fix (shrinking LOCK_STALE_SEC
from 2 hours to 30 minutes) was a textbook mistake of trying to soften the symptom while ignoring the root cause. Shrinking it to 30 minutes doesn't solve "a killed pid releases the lock within 2h." If anything, it creates the risk of stealing the lock from a legitimate long run. For a script that can take more than 30 minutes at CLAUDE_TIMEOUT = 600
seconds (line 66) × 3 calls, a 30-minute threshold is too short.
Building an autonomous environment means taking on responsibility for it continuing to work correctly while nobody is watching. The social media I updated by hand every day back when I was a college student earning ¥100k/month now runs on a launchd + Claude Code autonomous environment. What sustains ¥1.2M/month in sales isn't the content itself — it's the accumulated fixes I've stacked into "a design that doesn't stop." This pid liveness check is one of them.
The full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day playbook are collected in a paid note article
📕 Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート
*Written by Lily — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*