{"slug": "a-dead-pid-held-my-lock-for-2-hours-one-missing-line-zero-output-exit-0-every", "title": "A Dead PID Held My Lock for 2 Hours: One Missing Line, Zero Output, exit 0 Every Time", "summary": "A developer's autonomous Instagram posting system, built with Claude Code, silently failed for two hours when a dead process held a lock file, causing zero content generation despite exit code 0. The issue was traced to a lock check that only examined file modification time without verifying the process ID was alive. The developer fixed it by adding a PID liveness check, ensuring the system that supports ¥1.2M/month in revenue runs reliably.", "body_md": "For 30 straight days as a college student earning ¥100k/month, I posted to Instagram by hand, and then I burned out and stopped. Today the same job runs on a Claude Code autonomous environment, I touch nothing, and it holds up ¥1.2M/month in revenue. Except for the two hours when it quietly stopped: three consecutive launchd runs, zero pieces of content generated, `last exit=0`\n\nevery single time, and not one alert. The cause was a process that had already been killed, holding a lock file nobody would take away from it.\n\nThe problem with updating social media by hand is that it burns willpower. No matter how motivated you are, sleep, health, and mood all fluctuate. During the period when I was laid off and my income went to zero, I had no mental slack for posting at all.\n\nThe autonomous environment I spent six months building with Claude Code runs regardless of my emotional state. launchd calls a script, the script generates content with `claude -p`\n\n(MAX plan quota; paid APIs are off-limits), the output is queued for auto-posting, and it goes out to Instagram every day at 19:30. As long as this machinery keeps working, ¥1.2M/month in sales holds up without me lifting a finger.\n\nA lot of people think \"automation = writing scripts,\" and that's only half right. A script is correct at the moment you write it. Given time, external dependencies break, processes die for reasons you didn't anticipate, and lock files turn into debris that blocks every future run.\n\nAn autonomous environment that actually works is one that *assumes* breakage and carries a layer that repairs it. The lock story here is a textbook case.\n\n`~/dev/brand-404/sns/gen_feature.py`\n\nis a script launched on a schedule by launchd that auto-generates Instagram feature articles. A single run takes a long time (up to three `claude -p`\n\ncalls, plus image generation, adding up to tens of minutes), so it has a lock mechanism to prevent the next run from overlapping with one that hasn't finished.\n\nMishandling that lock file (`~/dev/brand-404/sns/gen_work/.lock`\n\n) meant that **pid 94799 held the lock without releasing it even though it had already been killed**, every subsequent run was skipped with \"lock held — 終了\", and generation stopped for about two hours.\n\nThe lock check has a constant `LOCK_STALE_SEC = 2 * 3600`\n\n(2 hours) (line 57 of gen_feature.py). It's a safety valve: \"even if the lock exists, steal it if the mtime is older than 2 hours.\"\n\nBut the original implementation only looked at mtime, without checking whether the pid was alive. Even with a killed pid sitting in `.lock`\n\n, as long as the mtime was within 2 hours it kept deciding \"still running\" and skipping. launchd could fire a third and a fourth time — all of them \"lock held — 終了\" until the two hours elapsed.\n\nThe worst possible state — automation that is \"running\" but produces nothing — continued silently for two hours.\n\nWith manual work, you notice: \"huh, nothing got generated today,\" and you run it by hand. But building an autonomous environment means taking on responsibility for it continuing to work correctly *while nobody is watching*.\n\nA pid liveness check looks like belt-and-suspenders, but in practice \"the script gets killed and the lock stays behind\" happens routinely. If you kill it instantly with SIGKILL, the `release_lock()`\n\nin the `finally`\n\nblock never runs at all. Same thing when you stop it by hand during development. If you're scheduling a long-running script with launchd, a pid liveness check is mandatory.\n\n```\nlaunchd (毎日定時 + 毎日19:30)\n  │\n  ├─ gen_feature.py（毎日定時）\n  │     acquire_lock()         ← 今回の話\n  │     ↓\n  │     キュー残数チェック\n  │     QUEUE_TARGET=3 に不足があれば\n  │     ↓\n  │     brand-catalog.json からブランド選定\n  │     ↓\n  │     Shopify /products.json 取得\n  │     ↓\n  │     claude -p #1: コピー生成 (copy.md)\n  │     ↓\n  │     claude -p #2: 画像役割選定 (images.md)\n  │     ↓\n  │     スライド生成 (build-post-from-json.mjs)\n  │     ↓\n  │     claude -p #3: セルフQA (qa.md)\n  │     ↓ QAを通過\n  │     content/sns/feature-XX-{slug}/ に出力\n  │     release_lock()\n  │\n  └─ ig_autopost.py（毎日19:30）\n        content/sns/feature-* のキューから1本取り出し\n        ↓\n        Instagram Graph API で投稿\n        ↓\n        state/ig_posted.jsonl に記録\n```\n\nThe queue targets a standing inventory of 3 items (gen_feature.py line 60, `QUEUE_TARGET = 3`\n\n; line 61, `MAX_GEN_PER_RUN = 2`\n\n). With 3 in stock, posting doesn't break even if generation fails for one or two days in a row.\n\nWhen you manage a script with a long single run (up to `CLAUDE_TIMEOUT = 600`\n\nseconds × 3 calls, plus image downloads) via scheduled launchd starts, these problems appear:\n\n`brand-catalog.json`\n\ncorrupt it`gen_work/.lock`\n\nis what prevents that. The process writes its own PID into the lock file, and on the next start the script checks whether that PID is alive before deciding whether to run.\n\nLines 225–249 of `gen_feature.py`\n\nare the whole thing.\n\n``` php\nLOCK_STALE_SEC = 2 * 3600       # 57行目\n\ndef acquire_lock() -> bool:\n    GEN_WORK.mkdir(parents=True, exist_ok=True)\n    if LOCK_FILE.exists():\n        age = time.time() - LOCK_FILE.stat().st_mtime\n        try:\n            pid = int(LOCK_FILE.read_text(encoding=\"utf-8\").strip())\n            if pid <= 0:\n                raise ValueError\n        except (OSError, ValueError):\n            log(\"stale lock 奪取: PIDが空または非数値\")\n        else:\n            try:\n                os.kill(pid, 0)          # ← プロセス生死チェック\n            except ProcessLookupError:\n                log(f\"stale lock 奪取: pid={pid} は不在\")\n            except PermissionError:\n                if age <= LOCK_STALE_SEC:\n                    return False\n                log(\"stale lock 奪取: mtime 2h超\")\n            else:\n                if age <= LOCK_STALE_SEC:\n                    return False\n                log(\"stale lock 奪取: mtime 2h超\")\n    LOCK_FILE.write_text(str(os.getpid()), encoding=\"utf-8\")\n    return True\n```\n\n**The decision logic before the fix (the broken code)** only looked at mtime.\n\n``` php\n# ❌ 修正前: pid の生死を見ない\ndef acquire_lock() -> bool:\n    GEN_WORK.mkdir(parents=True, exist_ok=True)\n    if LOCK_FILE.exists():\n        age = time.time() - LOCK_FILE.stat().st_mtime\n        if age <= LOCK_STALE_SEC:\n            return False          # ← kill済みpidでも2h以内は全部ここで返る\n        log(\"stale lock 奪取: mtime 2h超\")\n    LOCK_FILE.write_text(str(os.getpid()), encoding=\"utf-8\")\n    return True\n```\n\n`os.kill(pid, 0)`\n\nsends signal 0 (the null signal). Signal 0 doesn't actually send anything. The kernel just checks the pid and returns: **success if the process exists and you have permission to signal it, ProcessLookupError if the process doesn't exist, and PermissionError if the process exists but belongs to another user.**\n\nUsing this behavior, you can safely check whether a process is alive.\n\n| Result of os.kill(pid, 0) | Meaning | acquire_lock's decision |\n|---|---|---|\n| No exception | The process for that pid exists and is running | If mtime is within 2h, treat as \"running\" and skip |\n| ProcessLookupError | The process for that pid is gone (killed, etc.) | Stale lock — steal it immediately |\n| PermissionError | The pid exists but is owned by another user | mtime fallback (steal if older than 2h) |\n\nIn this incident, where pid 94799 had been killed, the fixed code would raise `ProcessLookupError`\n\n, immediately log `\"stale lock 奪取: pid=94799 は不在\"`\n\n, and let the next runner take the lock.\n\nThe same pattern is used in the multi-start guard of `~/.claude/scripts/automation-health.sh`\n\n(lines 20–30).\n\n```\n_ah_lock=\"${TMPDIR:-/tmp}/automation-health.lock\"\nif ! mkdir \"$_ah_lock\" 2>/dev/null; then\n  if kill -0 \"$(cat \"$_ah_lock/pid\" 2>/dev/null)\" 2>/dev/null; then\n    echo \"automation-health: 別インスタンス稼働中のためスキップ\" >&2\n    exit 0\n  fi\n  rm -rf \"$_ah_lock\"\n  mkdir \"$_ah_lock\" 2>/dev/null || { echo \"lock取得失敗・スキップ\" >&2; exit 0; }\nfi\necho $$ > \"$_ah_lock/pid\"\ntrap 'rm -rf \"$_ah_lock\"' EXIT\n```\n\nThe comments (lines 17–20) spell out the reasoning: \"`--deep`\n\nscans everything, so I/O piles up and load average spikes if multiple instances run. An atomic `mkdir`\n\nlock narrows it to one. **The PID liveness check auto-steals stale locks**, and `trap EXIT`\n\nguarantees release on both normal and abnormal termination.\"\n\nThe shell's `kill -0 <pid>`\n\nhas exactly the same semantics as Python's `os.kill(pid, 0)`\n\n. `2>/dev/null`\n\ndiscards stderr so the \"no such process\" error message isn't shown to the user. If the process exists, the exit code is 0; if not, it's 1 — so the success or failure of `if kill -0 ...`\n\nis the liveness check.\n\nOn top of that, `trap 'rm -rf \"$_ah_lock\"' EXIT`\n\nguarantees lock release on normal exit, error exit, or signal receipt. It's the same idea as calling `release_lock()`\n\nfrom Python's `try/finally`\n\n.\n\n```\nPython (gen_feature.py)          Shell (automation-health.sh)\n─────────────────────            ─────────────────────────────\nos.kill(pid, 0)                  kill -0 <pid>\n  ProcessLookupError → 奪取        終了コード1 → 残骸 → 奪取\n  PermissionError    → mtime fb    終了コード≠0→ 同上\n  例外なし           → 実行中       終了コード0 → 実行中\n\ntry/finally release_lock()       trap 'rm -rf lock' EXIT\n```\n\nDifferent languages, same three-step pattern for correct lock acquisition.\n\n`kill -0`\n\n/ `os.kill(pid, 0)`\n\nmtime is strictly the last resort for when the pid check isn't usable — it must never be the primary check.\n\nI've talked about `acquire_lock()`\n\n, but the correct lock pattern has its crux on the release side too.\n\n`release_lock()`\n\nat lines 252–256 of `gen_feature.py`\n\nis simple.\n\n``` php\ndef release_lock() -> None:\n    try:\n        LOCK_FILE.unlink()\n    except FileNotFoundError:\n        pass\n```\n\nSwallowing `FileNotFoundError`\n\nis deliberate. It's a guard against crashing when the `finally`\n\nblock and a manual call overlap, and it prevents the absurdity of \"the main work raises an exception because releasing the lock failed.\"\n\nWhat matters is the structure of the caller, `main()`\n\n(lines 995–1020).\n\n```\nif not acquire_lock():\n    log(\"lock held（他プロセスが実行中 or 2h以内）— 終了\")\n    return 0\n\ntry:\n    made = 0\n    attempts = 0\n    max_attempts = need + 3\n    attempted_brands: set[str] = set()\n    while made < need and attempts < max_attempts:\n        result = run_pipeline(attempted_brands)\n        ...\n    log(f\"生成完了: {made}/{need}本 試行{attempts}回 (queue残 {count_queue_remaining()}本)\")\nfinally:\n    release_lock()\n```\n\nEverything after acquiring the lock is wrapped in `try/finally`\n\n. Even if an exception is raised in the middle of `run_pipeline()`\n\n, the `finally`\n\nblock always runs, so the lock is reliably released.\n\nThere's one exception where this doesn't work: **SIGKILL**. If you force-terminate the process with `kill -9 <pid>`\n\n, the Python runtime itself dies instantly, so the `finally`\n\nblock never runs. That was the direct cause of this incident.\n\nThe same goes for `trap EXIT`\n\n(`automation-health.sh`\n\nline 30).\n\n```\ntrap 'rm -rf \"$_ah_lock\"' EXIT\n```\n\nThe `EXIT`\n\ntrap runs on `SIGTERM`\n\n(a normal kill), normal exit, and error exit — but not on SIGKILL. In either language, lock debris from a forced termination can't be prevented by `try/finally`\n\nand `trap EXIT`\n\nalone. That's exactly why you need the liveness check via `os.kill(pid, 0)`\n\n.\n\nThe `os.kill(pid, 0)`\n\ncall in `acquire_lock()`\n\nhas three branches (lines 235–247).\n\n```\ntry:\n    os.kill(pid, 0)\nexcept ProcessLookupError:\n    log(f\"stale lock 奪取: pid={pid} は不在\")\nexcept PermissionError:\n    if age <= LOCK_STALE_SEC:\n        return False\n    log(\"stale lock 奪取: mtime 2h超\")\nelse:\n    if age <= LOCK_STALE_SEC:\n        return False\n    log(\"stale lock 奪取: mtime 2h超\")\n```\n\n** ProcessLookupError** is the case where the process doesn't exist. This incident (a killed pid) falls here. Steal immediately. There's no need to look at mtime.\n\n** PermissionError** is the case where the pid exists but belongs to another user's process. Since you lack permission to send a signal, you can confirm existence but not whether it's \"a previous instance of my own script.\" In that situation you have no choice but to fall back on mtime, so it leans toward \"treat as running\" if it's within two hours.\n\n**No exception (the else block)** means the process exists and you do have permission to signal it — that is, it is definitely still running. Here too, if mtime is within two hours, it's treated as a legitimate in-progress run.\n\n`LOCK_STALE_SEC = 2 * 3600`\n\nis purely a last-resort safety valve. It only kicks in for the `PermissionError`\n\ncase where the pid check doesn't apply, and for the `else`\n\ncase where the process is running but is taking unexpectedly long for some reason. This connects back to what I said earlier about never making mtime the primary check. Whether two hours is the right threshold isn't the essence of the problem; doing the pid check first is.\n\nThe case where the lock file exists but its contents are corrupt is handled too (lines 229–234).\n\n```\ntry:\n    pid = int(LOCK_FILE.read_text(encoding=\"utf-8\").strip())\n    if pid <= 0:\n        raise ValueError\nexcept (OSError, ValueError):\n    log(\"stale lock 奪取: PIDが空または非数値\")\n```\n\n`OSError`\n\nis when the file couldn't be read (permission issues, etc.), and `ValueError`\n\nis when the int conversion fails or the pid is 0 or below. Both are treated as \"undecidable = debris\" and the lock is stolen.\n\nThe `pid <= 0`\n\ncheck is validation that leans on the POSIX guarantee that a PID is always a positive integer. `int(\"0\")`\n\nand `int(\"-1\")`\n\nconvert successfully but aren't valid values for a real process PID. If `strip()`\n\non an empty file yields an empty string, `int(\"\")`\n\nraises `ValueError`\n\n, which falls into the same path.\n\n`automation-health.sh`\n\nimplements its lock with `mkdir`\n\n(line 22) because that's more atomic than writing a file.\n\n```\nif ! mkdir \"$_ah_lock\" 2>/dev/null; then\n  if kill -0 \"$(cat \"$_ah_lock/pid\" 2>/dev/null)\" 2>/dev/null; then\n    echo \"automation-health: 別インスタンス稼働中のためスキップ\" >&2\n    exit 0\n  fi\n  rm -rf \"$_ah_lock\"\n  mkdir \"$_ah_lock\" 2>/dev/null || { echo \"lock取得失敗・スキップ\" >&2; exit 0; }\nfi\necho $$ > \"$_ah_lock/pid\"\n```\n\n`mkdir`\n\nis atomic at the kernel level. If the directory doesn't exist it's created and succeeds; if it exists it fails. Even if two processes call `mkdir`\n\nsimultaneously, the kernel lets only one of them succeed.\n\nPython's `write_text()`\n\ncarries no such guarantee. There is a theoretically possible race where A writes first and B overwrites immediately after. In the case of `gen_feature.py`\n\n, launchd starts it on an interval so the probability of concurrent execution is extremely low — but not zero. If you want the same atomicity in Python, one approach is `os.mkdir()`\n\n. The reason `gen_feature.py`\n\ndeliberately doesn't use that approach is that the high-frequency problem for this script wasn't \"simultaneous start races\" but \"killed pids left behind.\" The simple implementation was chosen to match the priority of the problem that actually needed solving.\n\nThe first sign of trouble was in my Discord `#brand-404`\n\nchannel. The \"🆕 o81 IG特集生成: ...\" notification that had arrived daily up to the previous day didn't come.\n\nNothing came into the Discord alert channel either. The symptom wasn't \"an error occurred\" — it was \"nothing happened.\"\n\nThinking the launchd job had failed, I ran `automation-health.sh`\n\n, and `com.lily.gen-feature`\n\nshowed \"loaded / last exit=0\". exit 0 is a normal exit. But no generation was coming.\n\nDigging through the logs, I found three hours' worth of these lines.\n\n```\n[gen] lock held（他プロセスが実行中 or 2h以内）— 終了\n```\n\nAll three launchd starts ended right there. Someone was holding the lock.\n\n`cat`\n\n-ing `~/dev/brand-404/sns/gen_work/.lock`\n\ngave me:\n\n```\n94799\n```\n\npid 94799 was written in it. `ps aux | grep 94799`\n\nreturned nothing. Not a live process.\n\nChecking the mtime with `ls -la ~/dev/brand-404/sns/gen_work/.lock`\n\nshowed a timestamp about 1.5 hours old. Against `LOCK_STALE_SEC = 2 * 3600`\n\n(2 hours), that was still 0.5 hours short. That's why it kept deciding \"within 2h = running\" and skipping.\n\npid 94799 had died because, during debugging the night before, I ran `kill -9 $(cat ~/dev/brand-404/sns/gen_work/.lock)`\n\nin another terminal. Normally `release_lock()`\n\nwould run and `.lock`\n\nwould disappear. But `-9`\n\n(SIGKILL) skips the `finally`\n\nblock. The lock stayed behind and swallowed every launchd start the next morning.\n\nThe first thing I thought of was \"2 hours is too long, let's make it 30 minutes.\"\n\nI actually changed it to `LOCK_STALE_SEC = 30 * 60`\n\nand committed. That was **wrong**.\n\nThe problem wasn't that 2 hours is long — it was that a killed pid was being judged alive. Even at 30 minutes, the same problem persists for 30 minutes after the kill. Worse, it introduces the risk of stealing the lock in the middle of a legitimate long run (three `claude -p`\n\ncalls = up to `CLAUDE_TIMEOUT = 600`\n\nseconds × 3, plus image downloads).\n\nThe correct fix is \"check whether the pid is alive *before* looking at mtime,\" not changing the mtime threshold. Tuning the threshold isn't a cure; it just softens the symptom. I reverted that commit and switched to the current approach of checking `os.kill(pid, 0)`\n\nfirst.\n\nAfter putting the first fixed version — which only caught `ProcessLookupError`\n\n— into production, I got stuck again during testing.\n\nI ran `sudo python3 ~/dev/brand-404/sns/gen_feature.py --dry-run`\n\nonce for verification purposes, and when I then ran it as a normal user, the leftover-lock symptom appeared again.\n\nA process run under `sudo`\n\nis owned by root. Calling `os.kill(root_pid, 0)`\n\nas a normal user returns `PermissionError`\n\n. My first fix only looked at `ProcessLookupError`\n\n, so `PermissionError`\n\npropagated out and produced a stack trace.\n\n```\n# ❌ PermissionError を見落とした中間バージョン\ntry:\n    os.kill(pid, 0)\nexcept ProcessLookupError:\n    log(f\"stale lock 奪取: pid={pid} は不在\")\n# PermissionError は捕まえておらず、外へ伝播する\nelse:\n    if age <= LOCK_STALE_SEC:\n        return False\n```\n\nIn Python's `try/except/else`\n\nstructure, `else`\n\nruns when no exception at all was raised in the `try`\n\nblock. Since `PermissionError`\n\nwasn't caught, that case entered neither `except ProcessLookupError`\n\nnor `else`\n\n, and the exception propagated straight to the caller. In the end I caught `PermissionError`\n\nexplicitly and routed it to the mtime fallback, arriving at the current form (lines 241–243).\n\n```\nexcept PermissionError:\n    if age <= LOCK_STALE_SEC:\n        return False\n    log(\"stale lock 奪取: mtime 2h超\")\n```\n\nWhat this burned into me hardest is that exit 0 and \"generation completed\" are different things.\n\n`log(\"lock held — 終了\"); return 0`\n\nis exit 0. The launchd job history records it as \"success.\" `automation-health.sh`\n\nreturns green too. No Discord alert arrives. From the outside everything is fine — and yet the actual goal, content generation, didn't happen at all.\n\nThis kind of silent failure can only be detected if you design monitoring around \"did the intended side effect occur?\" rather than \"did the process die?\"\n\nI now have separate monitoring that fires a Discord alert if the remaining queue count (`count_queue_remaining()`\n\n) stays at zero for a certain period. Process health checks and output health checks need to be designed separately. Making an autonomous environment \"designed on the assumption of breakage\" includes monitoring, in a separate layer, not only whether the process is alive but whether the environment keeps producing the output it's supposed to.\n\nThe earlier sections covered \"the design flaw of only looking at mtime,\" \"the fix via os.kill(pid, 0),\" and \"missing PermissionError.\" Here I'll add the points I actually got stuck on while running scripts with this same structure.\n\n**PID reuse causes a false \"alive\" verdict**\n\nPIDs on Linux/macOS are finite and wrap around. On macOS they cycle around a maximum near 99999. In a long-running environment, it's possible that the PID of a process you `kill -9`\n\n'd yesterday is being used by an unrelated process today. Call `os.kill(pid, 0)`\n\nin that state and you get \"the process is alive.\" `gen_feature.py`\n\ncurrently writes only the PID into `.lock`\n\n(line 248, `LOCK_FILE.write_text(str(os.getpid()), ...)`\n\n), but the combination of launchd running it once a day and `LOCK_STALE_SEC = 2 * 3600`\n\n(line 57) means there's virtually no real harm. For scripts invoked at high frequency, consider writing the PID paired with a start timestamp and treating it as \"running\" only when both match.\n\n**Forget GEN_WORK.mkdir(parents=True, exist_ok=True) and the first run dies instantly**\n\nLine 226 of `acquire_lock()`\n\nstarts with `GEN_WORK.mkdir(parents=True, exist_ok=True)`\n\n. Without that one line, on a first start where the `gen_work/`\n\ndirectory doesn't exist, an operation before `LOCK_FILE.exists()`\n\nraises `FileNotFoundError`\n\n. launchd starts the script, it dies immediately, and you're left with `last exit=1`\n\nand nothing running. Keep the order: create before releasing.\n\n`write_text()`\n\nis not atomic (a theoretical race exists)\n\nWith Python's `Path.write_text()`\n\n, other processes can potentially read the mid-write state. If the next instance calls `read_text()`\n\nbefore the `LOCK_FILE.write_text(str(os.getpid()), ...)`\n\nwrite completes, it reads an empty string, `int(\"\")`\n\nraises `ValueError`\n\n→ the lock is stolen as debris → two instances run simultaneously. That race is theoretically valid. Since `gen_feature.py`\n\nis started intermittently by launchd, simultaneous starts essentially don't happen. But for a high-frequency script called every minute, use an atomic write with `os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)`\n\n. This is exactly why `automation-health.sh`\n\nimplements its lock with `mkdir`\n\n(line 22, `if ! mkdir \"$_ah_lock\" 2>/dev/null;`\n\n).\n\n**SIGKILL isn't the only thing that skips finally**\n\nThe middle section covered how SIGKILL (`kill -9`\n\n) skips `try/finally`\n\n. One more thing to watch for is a C extension module calling `os._exit()`\n\ninternally. `sys.exit()`\n\nraises `SystemExit`\n\n, so `finally`\n\nruns; `os._exit()`\n\nterminates the process along with the runtime immediately, so `finally`\n\ndoes not run. The more you depend on external libraries, the higher the chance that `finally`\n\nwon't run. Designing the pid liveness check on the assumption that `finally`\n\nwon't run is the only fundamental countermeasure.\n\n**Building lock file paths from relative paths makes them environment-dependent**\n\nIf the launchd plist has a `WorkingDirectory`\n\nkey set, the script's starting directory changes to it. Build the lock file path from relative paths and the lock file ends up in a different place depending on the plist's `WorkingDirectory`\n\n. `gen_feature.py`\n\nbuilds every path as an absolute path from `ROOT = Path(__file__).resolve().parent.parent`\n\n(line 36). Scripts that run as launchd jobs should use `__file__`\n\n-based absolute paths.\n\n`_draft-feature-XX-{slug}`\n\ndirectories accumulate\n\nOn QA failure or slide-generation failure, `run_pipeline()`\n\nleaves the `_draft-feature-XX-{slug}`\n\ndirectory in place instead of deleting it. That's an intentional design decision so failed content can be inspected later (lines 954–959). `count_queue_remaining()`\n\nexcludes anything with the `_draft-`\n\nprefix from its count (lines 203–218), so inventory calculation isn't affected. But if they keep piling up, `content/sns/`\n\nfills with junk. A monthly cleanup with `find content/sns -name '_draft-*' -mtime +30 -type d`\n\nis needed.\n\n**Interpreting exit 0 as \"normal\" lets silent failures slip through**\n\nThe `main()`\n\nfunction also ends with `return 0`\n\nwhen lock acquisition fails (lines 996–997). launchd records that as a normal exit, and the launchd check in `automation-health.sh`\n\ndisplays `last exit=0`\n\nas `✓`\n\n. \"exit 0 because the queue was stocked and no generation was needed\" and \"exit 0 because lock acquisition failed and nothing was done\" look identical to launchd. That's why the actual two-hour stoppage was invisible.\n\n**The trap of the \"no error means everything's fine\" design philosophy**\n\nDiscord alerts only arrive for failure cases where `discord_alert()`\n\nis called (lines 140–142). A failed lock acquisition is a skip, not an error, so no alert arrives. If you design automation monitoring purely around \"notify me when something bad happens,\" silent skips will never be detected. What made me notice this incident in the first place was the *absence* of the success notification \"🆕 o81 IG特集生成: ...\" (`discord_post_review()`\n\n, line 973) in the `#brand-404`\n\nchannel.\n\n**Discord's default User-Agent gets blocked by Cloudflare**\n\nAs the comment in `_discord_post()`\n\nnotes (lines 93–94), Cloudflare rejects the `Python-urllib/x.y`\n\nUser-Agent that Python's `urllib`\n\nsends by default with 403/1010. It's worked around by explicitly setting `ua_header = \"lily-o81-gen/1.0\"`\n\n. If an automation script hits an external service with the default settings of `urllib`\n\nor the `requests`\n\nlibrary, it can get blocked without warning.\n\n**Misjudging the balance between curl timeouts and retries**\n\n`fetch_products()`\n\nretries Shopify's `products.json`\n\nwith `attempts=3`\n\n(lines 503–522). The design of waiting 15 seconds and retrying on failure comes from real damage: \"some stores temporarily return empty results under consecutive access\" (Beyond The Vines, 2026-07-26). Make the timeout extremely short and you get \"a brand that actually has products is judged to have 0 and demoted to skip.\" Make it too long and a single failure stalls things for minutes. Scripts that hit external APIs should design three things as a set: the per-attempt timeout, how many times to retry, and the wait between retries.\n\nBuilding on the earlier sections and the gotchas, here are the practices for lock management in long-running launchd scripts, extracted from real code.\n\n**1. Fix the pid liveness check as the primary decision and mtime as the secondary one**\n\n```\ntry:\n    os.kill(pid, 0)\nexcept ProcessLookupError:\n    pass  # 即奪取\nexcept PermissionError:\n    if age <= LOCK_STALE_SEC:\n        return False  # mtime fallback\nelse:\n    if age <= LOCK_STALE_SEC:\n        return False  # mtime fallback\n```\n\nThe shell version has the same semantics with `kill -0 \"$(cat pid_file)\"`\n\n(`automation-health.sh`\n\nline 23).\n\n**2. Always implement the three exception branches as a set**\n\nIf you catch only `ProcessLookupError`\n\n, `PermissionError`\n\npropagates out and crashes the script. Write all three as a set. Miss even one and you get \"a bug specific to that case.\"\n\n**3. Always wrap lock release in try/finally**\n\n```\nif not acquire_lock():\n    return 0\n\ntry:\n    # メイン処理\nfinally:\n    release_lock()\n```\n\nIn shell, `trap 'rm -rf \"$lock\"' EXIT`\n\nis the equivalent. It releases on normal exit, on an exception, and on `sys.exit()`\n\n.\n\n**4. Have release_lock() swallow FileNotFoundError**\n\nIt's a guard so a second `unlink()`\n\ndoesn't crash when `finally`\n\nand a manual call overlap. It prevents the absurdity of \"the main work raises an exception because releasing the lock failed\" (the pattern at lines 252–256).\n\n**5. Derive LOCK_STALE_SEC from the maximum run time**\n\nThe longest run of `gen_feature.py`\n\nis `CLAUDE_TIMEOUT = 600`\n\nseconds (line 66) × up to 3 calls + image downloads, roughly 35–40 minutes. `LOCK_STALE_SEC = 2 * 3600`\n\n(line 57) gives plenty of margin against that. Shortening the threshold does not solve the \"a killed pid releases the lock within 2h\" problem. Threshold tuning is symptomatic treatment; the pid liveness check is the cure.\n\n**6. Build the lock file path as an absolute path based on Path(__file__).resolve()**\n\nAn implementation that doesn't depend on launchd's `WorkingDirectory`\n\nis mandatory. The pattern of building every path from `ROOT = Path(__file__).resolve().parent.parent`\n\n(line 36) in `gen_feature.py`\n\nworks as a template as-is.\n\n**7. Always run GEN_WORK.mkdir(parents=True, exist_ok=True) before acquiring the lock**\n\nIt's required so the first start doesn't crash when the directory doesn't exist. Put it at the top of `acquire_lock()`\n\n(line 226).\n\n**8. In shell, ensure atomicity with a mkdir lock**\n\n`mkdir`\n\nis atomic at the kernel level. Even if two processes call it simultaneously, only one succeeds (`automation-health.sh`\n\nlines 22–27). If you need atomicity in Python too, use `os.open(path, os.O_CREAT | os.O_EXCL | os.O_WRONLY)`\n\n.\n\n**9. Design monitoring around \"the intended output,\" not the process**\n\nThe launchd check in `automation-health.sh`\n\ntreats `last exit=0`\n\nas normal, but a lock-acquisition skip is also exit 0. The right answer is positive monitoring: alert if the success notification \"content generation completed\" (`discord_post_review()`\n\n, line 973) doesn't arrive within a certain window.\n\n**10. Have success notifications (don't rely on error notifications alone)**\n\nA design that only notifies on errors misses silent skips. The design where the *absence* of the \"🆕 o81 IG特集生成: ...\" success notification signals a problem was the only signal that made me notice this failure. Having a \"something good happened\" notification is what makes anomaly detection effective.\n\n**11. Keep several days' worth of queue inventory**\n\nThe combination of `QUEUE_TARGET = 3`\n\n(line 60) and `MAX_GEN_PER_RUN = 2`\n\n(line 61) is an inventory design where \"posting doesn't break even if generation fails one or two days in a row.\" Even if a lock failure stops things for a day, 3 items in stock keep the posting going. Having a buffer eliminates the single point of failure in automation.\n\n**12. Make it possible to check state without taking the lock, via a --dry-run option**\n\n`gen_feature.py --dry-run`\n\n(lines 981–993) shows only the remaining queue count without taking the lock. It's safe to call even while production is running. During incident investigation you can check state without worrying about \"going to grab the lock and causing contention.\"\n\n**13. Make it possible to run E2E tests against production logic with a --force option**\n\nNormally it only generates until `QUEUE_TARGET`\n\nis met, but `--force`\n\ngenerates one item regardless of the remaining queue count (lines 980–984). You can test the whole pipeline without changing production logic, and use it to verify a fix.\n\n**14. Always set an explicit User-Agent on urllib calls to external APIs**\n\nPython's default `Python-urllib/x.y`\n\ngets blocked by Cloudflare and some CDNs. Specify an identifiable string like `\"lily-o81-gen/1.0\"`\n\n. It's mandatory when hitting Discord, Shopify, or any other API behind Cloudflare.\n\npid 94799 occupied the lock for two hours because of one missing line. The check that confirms whether a process is alive with `os.kill(pid, 0)`\n\nwas absent, so a killed pid kept being misjudged as \"still running.\" The fix itself is a few lines.\n\nBut what really needs to be understood is the \"why.\" SIGKILL skips `finally`\n\n. launchd's exit 0 doesn't mean \"it worked correctly,\" only \"the process terminated.\" Monitoring should be designed around \"is the intended output occurring?\" rather than process liveness. Those three points are the substance of what it means to make an autonomous environment \"designed on the assumption of breakage.\"\n\nThe first, wrong fix (shrinking `LOCK_STALE_SEC`\n\nfrom 2 hours to 30 minutes) was a textbook mistake of trying to soften the symptom while ignoring the root cause. Shrinking it to 30 minutes doesn't solve \"a killed pid releases the lock within 2h.\" If anything, it creates the risk of stealing the lock from a legitimate long run. For a script that can take more than 30 minutes at `CLAUDE_TIMEOUT = 600`\n\nseconds (line 66) × 3 calls, a 30-minute threshold is too short.\n\nBuilding an autonomous environment means taking on responsibility for it continuing to work correctly while nobody is watching. The social media I updated by hand every day back when I was a college student earning ¥100k/month now runs on a launchd + Claude Code autonomous environment. What sustains ¥1.2M/month in sales isn't the content itself — it's the accumulated fixes I've stacked into \"a design that doesn't stop.\" This pid liveness check is one of them.\n\nThe full picture of the system, the breakdown of the ¥1.2M/month, and the 30-day playbook are collected in a paid note article\n\n📕 [Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート](https://note.com/bokuwalily/n/n849b3a07784a)\n\n*Written by **Lily** — I ship iOS apps and automate my content stack with Claude Code.\n\nFollow along: [Portfolio](https://bokuwalily.com) · [X](https://x.com/bokuwalily) · [GitHub](https://github.com/bokuwalily)*", "url": "https://wpnews.pro/news/a-dead-pid-held-my-lock-for-2-hours-one-missing-line-zero-output-exit-0-every", "canonical_source": "https://dev.to/bokuwalily/a-dead-pid-held-my-lock-for-2-hours-one-missing-line-zero-output-exit-0-every-time-370c", "published_at": "2026-08-19 00:00:07+00:00", "updated_at": "2026-08-19 00:12:08.125561+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "artificial-intelligence"], "entities": ["Claude Code", "Instagram", "launchd", "Shopify"], "alternates": {"html": "https://wpnews.pro/news/a-dead-pid-held-my-lock-for-2-hours-one-missing-line-zero-output-exit-0-every", "markdown": "https://wpnews.pro/news/a-dead-pid-held-my-lock-for-2-hours-one-missing-line-zero-output-exit-0-every.md", "text": "https://wpnews.pro/news/a-dead-pid-held-my-lock-for-2-hours-one-missing-line-zero-output-exit-0-every.txt", "jsonld": "https://wpnews.pro/news/a-dead-pid-held-my-lock-for-2-hours-one-missing-line-zero-output-exit-0-every.jsonld"}}