My scheduled task reported "success" every 5 minutes for 3 weeks. The process inside it had been crashing the whole time.

A solo developer running an AI-powered trading bot operation discovered that its scheduled tasks reported success for three weeks while the underlying Python process crashed on nearly every cycle, logging over 18,000 tracebacks. The monitoring system, including the Windows Task Scheduler and LLM observability tools, failed to detect the issue because they only checked process exit codes and log strings, not whether the job accomplished its actual purpose. The developer is now building a monitoring layer specifically for unattended AI agents that checks whether the job is still doing the thing it exists to do.

I run a one-person AI company: Claude Code writes and maintains the code, and most of what it builds runs unattended — several trading bots on Windows Task Scheduler, each polling a broker API every 5 minutes, 24/7, with nobody watching in real time. I wrote before about the agent breaking things it had write access to I let an AI agent run my trading bots unattended https://dev.to/tatsuyawwp/i-let-an-ai-agent-run-my-trading-bots-unattended-it-broke-twice-before-i-built-a-gate-to-stop-it-40db . This one's different: nothing broke the code. The monitoring itself was confidently wrong, twice, in two different ways. One bot has a circuit breaker — if cumulative paper losses cross a threshold, it force-closes everything and halts. It fired for real: losses crossed the line, the position closed, confirmed directly against the broker's own API. Except the close was never written to the trade history file. The reporting script had no record of it. So my daily automated status report — a script that reads every bot's logs and has an AI model summarize what's going on — looked at a "still open" position that had actually been closed for days, and confidently told me the bot might have crashed. It hadn't. It had done exactly what it was supposed to do, and the thing telling me otherwise was itself misreading stale data as current. Annoying, but honest about being wrong once you dug in. The next one wasn't. A different bot's scheduled task kept reporting success — exit code 0, every 5-minute run, for over three weeks straight. The scheduler's own logs showed nothing but green. Inside, the actual Python process had been crashing on nearly every cycle that whole time: an authentication error from the broker's API, unhandled, caught only by the outer process wrapper, which then dutifully reported "the wrapper ran and exited" as success — which was technically true and completely useless. Three weeks of 5-minute cycles is over 8,000 attempts; more than 18,000 tracebacks piled up in the log because a few different code paths kept trying and kept failing. Zero real trades got recorded in that entire window. Nothing about the scheduler's own view of the world ever turned red. I only found it because I went and read the raw log file directly, not because anything monitoring the system told me to. The LLM-observability tools I know of trace individual API calls while you're actively building — good for "why did this one prompt cost so much" or "why did this one call return garbage," not built to watch a background job nobody's looking at. The classic dead-man's-switch tools the "ping us every N minutes or we alert you" category would have shown green the entire three weeks, too — the wrapper process itself never stopped running or stopped pinging. That category answers "did the job run." It has no way to know what the job was actually supposed to accomplish, so it can't tell you the job ran and did nothing. What both incidents have in common: the failure was invisible to anything that only checks "did the process exit 0" or "did something get logged as a plain string." Neither incident involved the code lying — the wrapper genuinely didn't crash, and the "open position" genuinely had been open at some point. The gap was between "the shell of the job looks fine" and "the job actually did the thing it exists to do." An agent that's competent while you're watching it, and an agent whose failures you'll actually notice once you stop watching, are not the same property — same lesson as the write-access incidents, different failure shape. Uptime monitoring answers "is it alive." Nobody was asking the more useful question: "is it still doing the thing," specifically for a background AI agent where "the thing" is something more structured than "return HTTP 200." I'm looking at building a small monitoring layer specifically for solo-developer/small-team unattended AI agents — schedule-aware, understands that "the process exited 0" and "the agent did its job" are different claims, and flags the gap between them instead of only the process dying outright. If you're running any kind of unattended agent — a scraper, a bot, a pipeline — on a schedule with nobody watching, I'd like to know if you've had your own version of "everything said green and it wasn't." Trying to figure out if this generalizes past my own two data points, same as last time.