Four Traps in MCP Health Checking: What Broke My Overnight Batches

An MCP server dying at 2 a.m. used to mean waking up to a log full of connection refused

and zero work done. Building a health-check hook that records failures outside the context window fixed that — my overnight batches haven't been killed by an MCP outage since.

When I first started automating with Claude Code, there was one failure mode I hated more than any other: a batch job kicked off at night, stopped dead by an MCP server timeout. I'd wake up to a pile of connection refused

logs and nothing having advanced since the previous evening. For SNS post automation that's survivable, but the time it stopped a client deliverable generation run, I genuinely panicked.

MCP is the mechanism that lets Claude Code use browser operations, DB lookups, external API calls, and so on as tools. From Claude's side it just looks like calling a tool named something like mcp__obsidian__search

, but underneath, communication with a local process or a remote HTTP server is running. When that server stops responding for whatever reason, Claude Code returns the tool call as an error and the whole flow of the session jams up.

The problem wasn't only "why did it stop." There were cases where it was stopped by a 429 (rate limit), yet it would re-call 30 seconds later → another 429 → stopped again, looping forever. There were also cases where a 503 (service temporarily unavailable) was judged the same as a 401 (expired auth), pointlessly running a re-authentication flow. Unless you vary the strategy per status code, it doesn't matter how fast a model you use — it's wasted.

Maintaining an autonomous environment at ¥1.2M/month revenue, what I noticed is that the time spent on "mechanisms that keep things from stopping" has a higher long-term ROI than the code added to increase earnings.

Back at ¥600K/month I thought "more tasks means more earnings." After being laid off and dropping to zero, my thinking changed while rebuilding it from scratch. Adding tasks doesn't help if the environment is unstable — throughput pins to a ceiling. Conversely, killing a single infrastructure-level problem raises the completion rate across all existing tasks.

The MCP health check is the classic example of this. By wiring ~/.claude/scripts/hooks/mcp-health-check.js

into a hook, an HTTP probe runs before Claude Code calls a tool, and depending on the response status it dispatches to "block immediately," "retry after backoff," or "run a reconnect command and re-probe." The verdict is persisted to ~/.claude/mcp-health-cache.json

, so even when context is compacted, the health record carries over.

When a Claude Code session runs for a long time, the conversation history gets compacted. Even if there was information in the past saying "this server was down," that doesn't survive into the post-compaction context. The result is waste: repeatedly attempting tool calls against a server already known to be unhealthy, and receiving an error each time.

A file-based cache is independent of the context. ~/.claude/mcp-health-cache.json

doesn't disappear no matter how much the session is compacted. When the health-check hook runs on the next turn, it loads the previous state from the file, and until nextRetryAt

has passed it blocks immediately without even re-probing. The idea of holding state outside the context window is what matters fundamentally.

Some people think, "why not just do error handling in Claude's prompt?" I actually tried it. A system prompt saying "if this tool errors, try another approach" works reasonably well for a one-off error. But in a situation where the MCP server is down and failures happen back to back, the model burns a huge number of tokens on "trying." And on the next turn, it goes right back to calling the same server. Writing the fact that "the server is dead" outside the context and blocking at the hook level is overwhelmingly cleaner.

Another common misconception is "MCP's own retry settings are enough." The MCP protocol has transport-layer retries, but it has no feature for reading status codes and switching strategy. 429 and 503 call for different backoff durations, and 401/403 are cases that need re-authentication rather than a retry. Implementing this dispatch at the application layer is what this hook is for.

mcp-health-check.js

responds to two kinds of Claude Code hook events. It's written verbatim in the comment at the top of the code (lines 7–12).

- PreToolUse: probe MCP server health before MCP tool execution
- PostToolUseFailure: mark unhealthy servers, attempt reconnect, and re-probe

PreToolUse

is called before the tool is executed. Here it fires a probe, checks whether the server is alive, and decides whether to allow execution or block with exit code 2. PostToolUseFailure

is called after a tool returns an error. It parses the error text to identify the failure pattern, marks the server unhealthy, and attempts a reconnect.

Claude Code が mcp__* ツールを呼ぶ
        │
        ▼ PreToolUse フック起動
┌──────────────────────────────────────────────────────┐
│            mcp-health-check.js                       │
│                                                      │
│  ① mcp-health-cache.json を読む                      │
│     status=healthy かつ expiresAt が未来？            │
│     YES ─────────────────────────────────────────→  │ exit 0
│     NO  ↓                                            │ （ツール実行へ）
│                                                      │
│  ② nextRetryAt が未来（unhealthy クールダウン中）？   │
│     YES → ブロック ──────────────────────────────→  │ exit 2
│     NO  ↓                                            │ （ツールをスキップ）
│                                                      │
│  ③ プローブ実行                                       │
│     HTTPサーバー → GET リクエスト（5秒タイムアウト）  │
│     stdioサーバー → プロセス起動（5秒生存確認）       │
│                                                      │
│     レスポンスのステータスコード判定                   │
│     ┌──────────────────────────────────────────┐    │
│     │ ECONNREFUSED / ENOTFOUND / タイムアウト   │──→ │
│     │  → 即 markUnhealthy & exit 2             │    │
│     ├──────────────────────────────────────────┤    │
│     │ 401 / 403 / 429 / 503                    │──→ │
│     │  → reconnect コマンドを実行               │    │
│     │  → 成功すれば再プローブ                   │    │
│     │  → 再プローブ OK → markHealthy & exit 0  │    │
│     │  → 再プローブ NG → markUnhealthy & exit 2│    │
│     ├──────────────────────────────────────────┤    │
│     │ 200 系 / 400 / 401 / 403 / 405 / 406    │    │
│     │（"到達できた"証明として healthy 扱い）    │──→ │ exit 0
│     └──────────────────────────────────────────┘    │
│                                                      │
│  ④ 状態を mcp-health-cache.json に書き出す            │
└──────────────────────────────────────────────────────┘
        │
        ▼ ツール実行後にエラーが出た場合
┌──────────────────────────────────────────────────────┐
│ PostToolUseFailure フック                             │
│   エラーテキストを FAILURE_PATTERNS と照合            │
│   failureCode 特定 → markUnhealthy → reconnect試行  │
│   → 再プローブ OK なら markHealthy                   │
└──────────────────────────────────────────────────────┘

Reading the actual code, the design intent shows up in the numbers (lines 22–26).

const DEFAULT_TTL_MS    = 2 * 60 * 1000;   // 2分
const DEFAULT_TIMEOUT_MS = 5000;            // 5秒
const DEFAULT_BACKOFF_MS = 30 * 1000;       // 30秒（初回バックオフ）
const MAX_BACKOFF_MS     = 10 * 60 * 1000; // 10分（上限）

A TTL of 2 minutes is the trade-off between "I don't want to run a probe every single time" and "I don't want to hold a stale result too long." In scenarios where Claude Code calls tools back to back, hitting the same server multiple times within 2 minutes is not unusual. With a cache, there's no need to run an HTTP probe each time, and latency drops.

The 5-second timeout is the threshold for confirming that a local stdio server "can start as a process." Rather than startup completion, it judges normality by the process "staying alive for 5 seconds" (the timer logic in probeCommandServer

, lines 428–472). Even for a heavy server that takes more than 5 seconds to start, this can be adjusted with the environment variable ECC_MCP_HEALTH_TIMEOUT_MS

.

The backoff calculation in the markUnhealthy

function (lines 213–229) is compressed into one line.

const nextRetryDelay = Math.min(
  backoffBase * (2 ** Math.max(failureCount - 1, 0)),
  MAX_BACKOFF_MS
);

backoffBase

defaults to 30 seconds. When failureCount

is 1, 2 ** 0 = 1

gives 30 seconds; the second time 2 ** 1 = 2

gives 60 seconds; the third 120 seconds; the fourth 240 seconds, doubling each time, capping out at a maximum of 600 seconds (10 minutes). It waits 30 seconds after the first failure, and if it hasn't recovered there, widens the interval to 1 minute, 2 minutes, 4 minutes — which avoids the situation of endlessly firing pointless probes at a downed server.

The definition of HEALTHY_HTTP_CODES (line 32) looks odd at first glance.

const HEALTHY_HTTP_CODES = new Set([
  200, 201, 202, 204,
  301, 302, 303, 304, 307, 308,
  400, 401, 403, 405, 406
]);

Some 400-level codes are treated as "healthy." The reason is written in the code's comment (lines 29–32).

// The preflight HTTP probe only checks reachability; it does not have access to
// Claude Code's stored OAuth bearer token. Treat auth-gated responses as
// reachable so the real MCP client can attempt the authenticated call. A
// Streamable HTTP MCP server can also return 406 to a bare GET that omits
// Accept: text/event-stream; that still proves the endpoint is alive.

The preflight probe is a GET request with no OAuth token. An endpoint that requires authentication returns 401 or 403, but that means it is "correctly rejecting an unauthenticated request" — evidence that the server is alive. 406 is a normal rejection response to a request without the Accept: text/event-stream

header. These are all codes that prove "it was reachable."

Meanwhile, the set targeted for reconnect is a subset (line 33).

const RECONNECT_STATUS_CODES = new Set([401, 403, 429, 503]);

401 and 403 are treated as "reachable" in the probe context, but when they come back after an actual tool call, the meaning is different. When the detectFailureCode

function is called in PostToolUseFailure, a 401/403 in the error message is taken as "authentication failure" and becomes the trigger for a reconnect. The key point is that the probe verdict and the post-failure verdict operate at different layers.

MCP errors don't necessarily come back as HTTP status codes. Errors from stdio servers come back as text. FAILURE_PATTERNS

(lines 34–40) is the mechanism that identifies the failure type from that text via regular expressions.

const FAILURE_PATTERNS = [
  { code: 401, pattern: /\b401\b|unauthori[sz]ed|auth(?:entication)?\s+(?:failed|expired|invalid)/i },
  { code: 403, pattern: /\b403\b|forbidden|permission denied/i },
  { code: 429, pattern: /\b429\b|rate limit|too many requests/i },
  { code: 503, pattern: /\b503\b|service unavailable|overloaded|temporarily unavailable/i },
  { code: 'transport', pattern: /ECONNREFUSED|ENOTFOUND|EAI_AGAIN|timed? out|socket hang up|connection (?:failed|lost|reset|closed)/i }
];

From error messages containing strings like "unauthorized," "auth expired," "rate limit," or "ECONNREFUSED," it automatically identifies the failure code. The transport

code isn't an HTTP status — it's the case where the network connection itself is cut. This is a state of "can't even reach the server," where backoff takes priority over reconnect.

~/.claude/mcp-health-cache.json

is written out by the saveState

function (lines 95–102), formatted with JSON.stringify. The actual file has a structure like this.

{
  "version": 1,
  "servers": {
    "obsidian": {
      "status": "healthy",
      "checkedAt": 1753666200000,
      "expiresAt": 1753666320000,
      "failureCount": 0,
      "lastError": null,
      "lastFailureCode": null,
      "nextRetryAt": 1753666200000,
      "lastRestoredAt": 1753666200000,
      "source": "~/.claude/settings.json"
    },
    "agentmemory": {
      "status": "unhealthy",
      "checkedAt": 1753665900000,
      "expiresAt": 1753665900000,
      "failureCount": 3,
      "lastError": "ECONNREFUSED 127.0.0.1:3001",
      "lastFailureCode": "transport",
      "nextRetryAt": 1753666020000,
      "lastRestoredAt": null
    }
  }
}

It's a two-stage gate: if expiresAt

is in the future, pass without re-probing; if nextRetryAt

is in the future, block without even retrying. The more failureCount

accumulates, the longer the backoff stretches, so a repeatedly-down server gets progressively wider attempt intervals. lastRestoredAt

is a record of "when it recovered," usable for after-the-fact uptime analysis.

The source

field is the path to the config file. The configPaths

function (lines 54–72) searches in the order: the current directory's .claude.json

→ the current directory's .claude/settings.json

→ home's .claude.json

→ home's .claude/settings.json

, and the first one found is used. Even when you have per-project MCP configuration, the correct config is referenced.

The reconnectCommand

function (lines 516–527) reads the command from environment variables.

const key = `ECC_MCP_RECONNECT_${String(serverName).toUpperCase().replace(/[^A-Z0-9]/g, '_')}`;
const command = process.env[key] || process.env.ECC_MCP_RECONNECT_COMMAND || '';

If the server name is agentmemory

, it looks for an environment variable named ECC_MCP_RECONNECT_AGENTMEMORY

. If not found, it uses the global fallback ECC_MCP_RECONNECT_COMMAND

. If the command string contains {server}

, it is expanded to the server name (lines 524–526).

For example, if you use PM2 as your process manager, you can configure it like this.

export ECC_MCP_RECONNECT_COMMAND="pm2 restart {server}"

With this, when the agentmemory

server returns a 401 or 503, pm2 restart agentmemory

runs automatically, and tool execution is allowed only after a re-probe confirms it's normal. When a reconnect succeeds, reconnect-command

is recorded in markHealthy

's restoredBy

field (lines 607–609).

At this point you should have a grasp of the hook's overall shape. In the next section I'll dig into the actual stumbling points and the traps that are easy to fall into during configuration.

The first thing the hook does is identify "which MCP server is this call for." The extractMcpTarget

function (lines 133–167) handles that.

if (!toolName.startsWith('mcp__')) {
  return null;
}

const segments = toolName.slice(5).split('__');
if (segments.length < 2 || !segments[0]) {
  return null;
}

return {
  server: segments[0],
  tool: segments.slice(1).join('__')
};

Given a name like mcp__obsidian__search_notes

, slice(5)

makes it obsidian__search_notes

, and split('__')

makes it ['obsidian', 'search', 'notes']

. segments[0]

is the server name, and the rest joined with double underscores is the tool name.

That said, parsing the tool name is strictly a fallback. It looks for an explicit server

field first (lines 135–145).

const explicitServer = input.server
  || input.mcp_server
  || input.tool_input?.server
  || input.tool_input?.mcp_server
  || input.tool_input?.connector
  || null;

The reason it walks multiple paths is that the hook event's schema can have fields in shifted positions depending on the Claude Code version and connection method (HTTP/stdio). Furthermore, if JSON parsing fails and the truncated

flag is set, extractMcpTargetFromRaw

(lines 169–179) applies a regular expression to the raw string to extract the same information. It's a design that doesn't give up even when parsing fails.

handlePreToolUse

(lines 567–630) looks simple, but two cache checks run in succession.

// 第1ゲート: healthy かつキャッシュ有効 → プローブなしで通過
if (previous.status === 'healthy' && Number(previous.expiresAt || 0) > now) {
  return { rawInput, exitCode: 0, logs };
}

// 第2ゲート: unhealthy かつクールダウン中 → プローブなしでブロック
if (previous.status === 'unhealthy' && Number(previous.nextRetryAt || 0) > now) {
  logs.push(
    `[MCPHealthCheck] ${target.server} is marked unhealthy until ${new Date(previous.nextRetryAt).toISOString()}; skipping ${target.tool || 'tool'}`
  );
  return { rawInput, exitCode: shouldFailOpen() ? 0 : 2, logs };
}

"healthy and within TTL" passes, "unhealthy and in backoff" is blocked, and only in neither case (unhealthy but backoff expired, or the cache is empty) is a probe executed. In sessions that call tools at high frequency, almost all calls pass at the first gate, so HTTP request latency is effectively zero.

The conditions that trigger a reconnect need similar care (lines 601–603).

let reconnect = { attempted: false, success: false, reason: 'probe failed' };
if (probe.failureCode || previous.status === 'unhealthy') {
  reconnect = attemptReconnect(target.server);

A reconnect doesn't run just because "the probe failed." It requires either probe.failureCode

(a status code for which reconnect is enabled) or the condition "already unhealthy since last time." Running a reconnect command every time when you can't even connect due to ECONNREFUSED is wasteful, but when the server exists yet is unusable — as with 429 or 503 — actively attempting to reconnect is worthwhile; that judgment is embedded here.

For an HTTP server, firing a GET with requestHttp

tells you what you need. How do you check a stdio server? The approach in probeCommandServer

(lines 301–481) is distinctive.

Actually start the process and watch it for 5 seconds. If it stays alive, it's healthy; if it exits before then, it's not.

timer = setTimeout(() => {
  // タイムアウト到達 = 5秒間プロセスが生存 → 正常

  // ただし: ロードされたマシンではexitイベントがタイマーより
  // わずかに遅れて届くことがある。高速クラッシュを見逃さないため
  // プロセスの状態を再確認する
  if (child.exitCode !== null || child.signalCode !== null) {
    attemptFinish({
      ok: false,
      statusCode: child.exitCode,
      reason: stderr.trim() || `process exited before handshake (...)`
    });
    return;
  }

  // SIGTERMで終了させ、200ms後にSIGKILLで確実に始末する
  child.kill('SIGTERM');
  setTimeout(() => {
    try { child.kill('SIGKILL'); } catch { /* ignore */ }
  }, 200).unref?.();

  attemptFinish({
    ok: true,
    statusCode: null,
    reason: `${serverName} accepted a new stdio process`
  });
}, timeoutMs);

As the comment says, the "fast-crashing server" problem really is a nuisance. If the process exits right after startup, the exit

event and the timer callback can arrive at nearly the same moment. If the timer runs first, it would return ok: true

without a child.exitCode

check. As a countermeasure, the timer callback re-checks child.exitCode !== null

, and if it's already dead, drops it to ok: false

.

Windows behavior is worked out too (lines 329–334, 438–461). A command without an extension, like npx

, needs to resolve to npx.cmd

on Windows, and on top of that, from Node 18.20 onward, executing .cmd

via a shell was restricted as part of the CVE-2024-27980 fix. The code lines up fallbacks of command.cmd

/ command.exe

/ command.bat

in a candidates

array, and tries the next candidate in order on ENOENT. It also executes .cmd

/.bat

files via a shell, but has a safety valve that refuses shell execution if the command string contains shell metacharacters (&|<>^%()

etc.) (line 339, UNSAFE_SHELL_CHARS

).

failureSummary

(lines 231–244) is quietly important.

const pieces = [
  typeof input.error === 'string' ? input.error : '',
  typeof input.message === 'string' ? input.message : '',
  typeof input.tool_response === 'string' ? input.tool_response : '',
  typeof output === 'string' ? output : '',
  typeof output?.output === 'string' ? output.output : '',
  typeof output?.stderr === 'string' ? output.stderr : '',
  typeof input.tool_input?.error === 'string' ? input.tool_input.error : ''
].filter(Boolean);

return pieces.join('\n');

Which field the MCP error text lands in differs by server implementation and Claude Code version. The case where it's in error

, the case where it's in tool_response

, the case where it's in output.stderr

— I've encountered all of them in practice. By passing the string joined from all fields with pieces.join('\n')

to detectFailureCode

, the regular expressions can catch the error text no matter which field it's in.

shouldFailOpen

(lines 557–559) is a one-liner over an environment variable.

function shouldFailOpen() {
  return /^(1|true|yes)$/i.test(String(process.env.ECC_MCP_HEALTH_FAIL_OPEN || ''));
}

It's the setting that lets through tool calls to unhealthy servers. You use it during development when you want to "have the hook running but not blocking, just to observe." In PreToolUse

, it returns exit 0

(pass) instead of exit 2

(block).

The exitCode for PostToolUseFailure

is always 0

(line 677). This is an important design point: the post-failure hook's purpose is recording state, not blocking tool execution. Changing the exit code of a tool call that has already failed is meaningless; it only writes logs and updates the cache, handing the state over to the next PreToolUse

.

From wiring this mechanism into my own environment to having it stable, I got stuck in three situations. I'll write the symptom, the cause, and the fix in order.

Symptom: The agentmemory

server remained in the cache with status: healthy

, yet actual tool calls returned ECONNREFUSED

. The health check looked meaningless.

Cause: A stdio server probe only confirms that "the process stays alive for 5 seconds." The process starts, gets terminated with SIGTERM after 5 seconds, and then when the real tool call arrives, the server starts again from zero. But in a real startup, it takes time to complete the MCP handshake (JSON message exchange over stdio). A different error was occurring during that handshake.

Specifically, a server that tried to start without the environment variable ANTHROPIC_API_KEY

being set was crashing right after startup. At probe time it stayed alive for 5 seconds so it got recorded as healthy

, but during the actual handshake it would crash a few seconds in.

Fix: Check first whether the server process's environment variables are all in place. mcp-health-check.js

merges config.env

into the process's environment variables when starting (lines 306–309). Confirming that the same environment variables as the production startup are in the config file's env

section is the first order of business. Only after understanding that "healthy in the probe" and "healthy in the tool call" are strictly different verdicts did the debugging direction finally become clear.

Symptom: The Obsidian remote server temporarily returned 503s overnight. Checking the next morning, the server itself had long since recovered, but nextRetryAt

in mcp-health-cache.json

pointed to a future time and it stayed blocked.

Cause: Because 503 is included in RECONNECT_STATUS_CODES

, a reconnect is attempted when a 503 comes in. But I had set neither ECC_MCP_RECONNECT_OBSIDIAN

nor ECC_MCP_RECONNECT_COMMAND

. Without a reconnect command, attemptReconnect

returns attempted: false

(line 531), only markUnhealthy

runs, and failureCount

accumulates. Three 503s make failureCount=3

, and backoff is 30 * (2 ** 2) = 120 seconds

. Five more over the next 30 minutes makes failureCount=5

and 30 * (2 ** 4) = 480 seconds

. As a dozen-plus errors piled up overnight, the backoff was pinned at the ceiling of 600 seconds (10 minutes). Even after the server recovered, a probe only ran every 10 minutes, and all tool calls in between were blocked.

Fix: Either configure a reconnect command, or set ECC_MCP_HEALTH_BACKOFF_MS

to a small value (e.g. 10000

= 10 seconds) to lower the backoff base. I set ECC_MCP_RECONNECT_COMMAND=echo noop

(a do-nothing dummy) in my launchd plist, which gets it treated as "reconnect attempted" and suppresses failureCount accumulation. The proper approach is to manage the service with PM2 or systemd and configure a real reconnect command.

export ECC_MCP_RECONNECT_OBSIDIAN="pm2 restart obsidian-mcp"

Once this setting was in, the cycle started working: a 503 comes in → PM2 restarts → a re-probe confirms normal → immediate return to healthy

.

Symptom: When starting Claude Code in a particular project directory, the MCP health check logged "No MCP config found for obsidian" and skipped the probe, yet the tool call itself ran. The hook appeared to be functioning while not actually functioning.

Cause: The configPaths

function (lines 54–72) searches the current directory's config file first.

return [
  path.join(cwd, '.claude.json'),
  path.join(cwd, '.claude', 'settings.json'),   // ← これが先に見つかる
  path.join(home, '.claude.json'),
  path.join(home, '.claude', 'settings.json')
];

That project had a .claude/settings.json

, and it defined only project-specific MCP servers (like playwright

). resolveServerConfig('obsidian')

doesn't stop at the point of finding the first file in list order — if that file doesn't have obsidian

, it moves on to the next file. In readJsonFile

's loop, if data?.mcpServers?.[serverName]

is null it returns null

and continues the loop — but what actually caused the problem was a case where I had pointed the environment variable ECC_MCP_CONFIG_PATH

at the home config file and there was a typo in that path.

export ECC_MCP_CONFIG_PATH="/Users/~/.claude/settings.json"

export ECC_MCP_CONFIG_PATH="${HOME}/.claude/settings.json"

path.resolve

returned the nonexistent path /Users/~/.claude/settings.json

, readJsonFile

returned null, and it was treated as config not found. Because setting ECC_MCP_CONFIG_PATH

completely overrides the normal search paths (lines 55–61), a typo makes all server configuration disappear.

Fix: Use an absolute path for ECC_MCP_CONFIG_PATH

. Tilde expansion is something the shell does; Node's path.resolve

does not do it. When writing it in a launchd plist, ~

is not expanded, so you need to write it as /Users/your-home-directory

, ${HOME}

(via a shell script), or the equivalent of os.homedir()

.

<!-- launchd plist では ~ が展開されない -->
<key>ECC_MCP_CONFIG_PATH</key>
<string>/Users/lily/.claude/settings.json</string>

Writing the actual home directory path literally is the most reliable.

Symptom: The hook was working correctly and unhealthy

was recorded in the cache. Yet tool calls went through and errors flowed into Claude's context. It produced the token waste of receiving the same error 20 times in one session.

Cause: I had forgotten to remove ECC_MCP_HEALTH_FAIL_OPEN=1

, which I'd set during debugging, from my launchd plist. As long as this setting exists, shouldFailOpen()

returns true

and all tool calls to unhealthy servers become exit 0

(pass). The hook is running but blocking nothing.

So there was an inconsistency where looking at mcp-health-cache.json

showed status: unhealthy

recorded, and yet tool calls went through.

Fix: Treat ECC_MCP_HEALTH_FAIL_OPEN

as a debug-only flag and never write it in a permanent launchd plist. I made myself an operational rule to delete it as soon as debugging is over. To check whether the hook is blocking as intended, the quickest route is to read ~/.claude/mcp-health-cache.json

directly and, in a state where a server with status: unhealthy

exists, confirm that the skipping

wording appears in the hook's stderr log when a tool call runs.

tail -f ~/.claude/logs/hooks.log | grep MCPHealthCheck

After getting past these four sticking points, the environment stabilized. Now it's normal for a batch running overnight to have finished 100 items by the time I wake up. Since introducing this mechanism, a batch has never once been stopped by an MCP server going down.

The previous section covered four sticking points in detail. Here I'll enumerate, as a list, the traps I actually hit or that are easy to overlook. Ones that overlap with P2 are omitted.

CLAUDE_HOOK_EVENT_NAME

isn't passed, all calls are processed as PreToolUseLine 704 of the code has const eventName = process.env.CLAUDE_HOOK_EVENT_NAME || 'PreToolUse';

. If this environment variable is missing, event determination is pinned to PreToolUse. There are cases where a hook you meant to register as PostToolUseFailure was actually running as PreToolUse. The first step in isolating this is to dump process.env

to stderr and check whether Claude Code is passing ENV correctly.

truncated

flag is set, and unless fail-open is on it becomes an immediate blockMAX_STDIN = 1024 * 1024

(line 22) is that limit. If, on a large tool call that stuffs an entire file's contents into tool_input, the hook input exceeds this value, it tries to identify the target server with the parse still incomplete. If the target can be identified, you can raise the limit with ECC_HOOK_INPUT_MAX_BYTES

, but if it can't be parsed completely, it's blocked with exit 2

unless fail-open

is on (lines 695–701). Without knowing this behavior, it looks like the mysterious phenomenon of "the hook falsely blocks only certain tools."

saveState

is designed to swallow errors, so write failures continue silentlysaveState

at lines 95–102 catches all errors with try/catch and, as the comment says — "Never block the hook on state persistence errors." — silently continues. Disk full, insufficient permissions, the ~/.claude/

directory being gone: in every case the hook keeps returning exit 0 or exit 2. But because the cache isn't written out, healthy verdicts disappear after the TTL and a probe runs every time. If you feel like "for some reason an HTTP probe runs every time," start with a disk check via df -h ~/.claude/

and a permission check via ls -la ~/.claude/mcp-health-cache.json

.

resolveServerConfig

(lines 181–197) scans all files in configPaths() order and, at the point the server name is found, records that path in the source

field. If a server of the same name is defined both in a project-specific .claude/settings.json

and in home's ~/.claude/settings.json

, the project side wins. You can check whether an unintended config is being used via the source

field in ~/.claude/mcp-health-cache.json

. 80% of "I fixed the home config but it isn't reflected" symptoms are this.

attemptReconnect

(lines 528–555) executes with spawnSync(command, { shell: true, ... })

. Command chains containing &&

or ;

work fine, but command substitutions like $(date)

can get expanded unintentionally. Keeping the value of ECC_MCP_RECONNECT_COMMAND

to a simple single command like pm2 restart {server}

is safest. Using the {server}

placeholder (lines 524–526) saves you from having to set a separate environment variable per server.

markUnhealthy

accumulates previous.failureCount + 1

every time (line 215). Meanwhile, markHealthy

force-resets to failureCount: 0

(lines 200–210). The problem is that when no reconnect command is configured and the backoff pins at the ceiling (600 seconds), you're made to wait 10 minutes until the next re-probe. All tool calls in that window are blocked. Even after a long-unstable server recovers, recovery is only confirmed every 10 minutes. The fastest way to clear this state is to manually delete the target server's entry from the cache file.

  python3 -c "
  import json, pathlib
  p = pathlib.Path.home() / '.claude/mcp-health-cache.json'
  s = json.loads(p.read_text())
  s['servers'].pop('agentmemory', None)
  p.write_text(json.dumps(s, indent=2))
  print('reset done')
  "

probeCommandServer

(lines 306–309) merges config.env

into process.env

at process startup. In an interactive zsh session, ANTHROPIC_API_KEY

is automatically inherited from .zshrc

, but Claude started from launchd doesn't have the login shell's environment variables. Most cases of the symptom "works when I try it by hand, fails in the overnight batch" have their cause here. The fundamental countermeasure is to explicitly enumerate the required API keys in every server's config.env

section.

Because 429 is included in RECONNECT_STATUS_CODES

, a reconnect runs when the probe returns 429 (line 33). Even if the reconnect command executes, if the underlying problem (rate limiting) isn't resolved, the re-probe also returns 429, markUnhealthy

runs, and failureCount

increases. It's a vicious circle where the probe itself consumes the rate limit. To prevent this, you can either set ECC_MCP_HEALTH_TTL_MS

longer to reduce probe frequency, or change the target server's probe URL to a health endpoint that doesn't require authentication.

The return value of handlePostToolUseFailure

is always exitCode: 0

(line 677). This is correct by design — returning a block for a "tool call that has already failed" is too late. This hook's job is "writing state so the next PreToolUse blocks," and the exit code has no meaning. Most reports of "the PostToolUseFailure hook is running but nothing gets blocked" are resolved by checking whether the cache is being read correctly on the next PreToolUse.

Over six months of supporting a ¥1.2M/month autonomous environment with this hook, here are 12 things I stick to.

1. Always configure a reconnect command, even a dummy one

When RECONNECT_STATUS_CODES errors pile up without a reconnect command, failureCount

snowballs and the backoff pins at the ceiling. Access to an already-recovered server stays blocked for a long time. Under PM2 management, configure a real one; otherwise even just echo noop

gets it treated as "reconnect attempted" and suppresses failureCount accumulation.

export ECC_MCP_RECONNECT_COMMAND="pm2 restart {server}"

export ECC_MCP_RECONNECT_COMMAND="echo noop"

2. Don't write ECC_MCP_HEALTH_FAIL_OPEN in your launchd plist

While this variable is set, shouldFailOpen()

(lines 557–559) returns true

and all tool calls to unhealthy servers become exit 0

(pass). The hook looks like it's working while blocking nothing. Treat it as a debug-only flag and make an operational rule to remove it as soon as debugging is done.

3. Write an absolute path in ECC_MCP_CONFIG_PATH

~

is not expanded in a launchd plist. Writing /Users/your-home-directory-name/.claude/settings.json

literally is the most reliable. path.resolve

is a Node feature and does not perform the shell's tilde expansion (lines 55–61). A typo makes all server configuration disappear.

4. Register aliases for cache inspection, reset, and log monitoring

So you can move immediately when a problem occurs, put aliases in .zshrc

.

alias mcp-health="python3 -m json.tool ~/.claude/mcp-health-cache.json"
alias mcp-reset="echo '{\"version\":1,\"servers\":{}}' > ~/.claude/mcp-health-cache.json"
alias mcp-logs="tail -f ~/.claude/logs/hooks.log | grep MCPHealthCheck"

5. Explicitly list every API key a stdio server needs in config.env

Claude started from launchd doesn't have the login shell's environment variables. Enumerating all required keys in each server's env

section in ~/.claude/settings.json

makes interactive behavior and batch behavior match.

6. Tune the TTL to match session usage frequency

The default of 2 minutes (DEFAULT_TTL_MS = 2 * 60 * 1000

, line 23) is a general-purpose value. In environments that call one server at high frequency, like overnight batches, extending it to 5–10 minutes greatly reduces probe count and raises throughput. During debugging, shrinking it to 30 seconds makes reproducing problems faster.

export ECC_MCP_HEALTH_TTL_MS=600000   # 10分

export ECC_MCP_HEALTH_TTL_MS=30000    # 30秒

7. Match the backoff base value to the server's characteristics

The default 30 seconds assumes a local process restart. For an external API server that takes 1–5 minutes to recover, raising it to ECC_MCP_HEALTH_BACKOFF_MS=60000

(1 minute) makes the recovery-check interval match reality. Conversely, if a local stdio server recovers in a few seconds, 10 seconds is plenty.

8. Manage important servers with PM2

When PM2's auto-restart combines with this hook's reconnect → re-probe cycle, recovery from transient failures becomes fully automated. The cycle is: the server crashes → PM2 restarts it → the next PreToolUse hook calls the reconnect command → PM2 returns success as "already started" → the re-probe confirms healthy → tool execution is allowed. Since moving to this combination, cases of "the server was dead the next morning" have dropped to zero.

9. Periodically check config-file resolution results via the source field

The source

field on each entry in ~/.claude/mcp-health-cache.json

shows which config file the server configuration was read from. After adding a .claude/settings.json

to a project directory, use this field to confirm the intended config is being used. Most of "I fixed the home config but it doesn't work" is resolved here.

10. Build the habit of watching the hook's logs in real time

emitLogs

(lines 561–565) outputs logs to stderr. Because they flow into Claude Code's hook log, you can check them with tail -f

. If the word skipping

appears, blocking is working. If connection restored

appears, the reconnect succeeded. With the habit of reading the logs, you can confirm in seconds whether the hook is behaving as intended.

11. Centralize hook registration in the home config

You can write hooks in a project's .claude/settings.json

too, but I strongly recommend writing the health-check hook only in home's ~/.claude/settings.json

. Scattering it per project creates holes like "the hook doesn't take effect only in that project." To receive uniform protection across all projects, treat the home config as canonical.

12. When failureCount grows abnormally large, manually delete the entry to reset it

If markHealthy

is called it automatically resets to failureCount: 0

, but when the backoff is too long and the path to markHealthy is closed off, manual deletion is fastest. Registering the python3 one-liner above as an alias lets you reset a specific server's state in 10 seconds.

After being laid off and dropping to ¥0/month revenue, the thing I feared most in rebuilding my autonomous environment from scratch was the scenario of "the batch I started at night is completely wiped out by morning." What most reliably prevents that isn't a stronger model or better-polished prompts — it's a mechanism that detects failures and records them outside the context.

If I summarize the design of mcp-health-check.js

in one line: "don't lose health information even when context is compacted." mcp-health-cache.json

lives outside the session. The backoff calculation is handled by code. The model doesn't have to reason about "this server might be down." The tokens and attention saved can go to the actual task.

The reason the ¥1.2M/month environment keeps being ¥1.2M/month is that the overnight batches don't fall over before morning. And they don't fall over because this hook is quietly doing its job.

The implementation is 721 lines, but the core comes down to three points.

MAX_BACKOFF_MS

's 10 minutes prevents total abandonmentOnce you understand these three points, you'll be able to tune it and isolate problems on your own.

I've put the full picture of the mechanism, the breakdown of the ¥1.2M/month, and the 30-day procedure into a paid note.

📕 Claude Code自律環境で、実際どう稼ぐか ― 仕組み・実例・始め方・サポート

*Written by Lily — I ship iOS apps and automate my content stack with Claude Code.

Follow along: Portfolio · X · GitHub*

source & further reading

dev.to — original article Beyond the Hype: A Developer’s Critical Audit of Real-World AI Agents from OmniRoute to Eliza Running the blog's content autopilot on a Claude subscription, not API credit Olá, dev.to! Sou o Gabriel e automatizo o escritório jurídico onde trabalho