# WorkerDeck – Run one isolated background process per user (Python, zero deps) > Source: > Published: 2026-07-14 12:06:57+00:00 **Run one isolated background process per user — and manage them all from one dashboard.** **Zero dependencies: the core is pure Python standard library.** If you're building a platform where every user runs their own long-lived process — an AI agent, a data pipeline, a scheduled job, a bot, a scraper — you hit the same wall: starting one process per user, keeping them isolated, knowing which are alive, restarting the ones that died, and stopping them cleanly. It's fiddly, it's easy to get subtly wrong, and it has nothing to do with your actual product. WorkerDeck is that layer, done once. You bring the worker; it handles the lifecycle. ``` ┌─────────────────────────────────────────────┐ │ Dashboard │ │ ● Ada running pid 805 up 6s [⏹] │ │ ● Linus running pid 807 up 6s [⏹] │ │ ○ Grace stopped [▶] │ └─────────────────────────────────────────────┘ │ start / stop / restart / inspect ▼ ┌─────────────────────────────────────────────┐ │ WorkerManager │ │ user 1 → its own process, dir, config, log │ │ user 2 → its own process, dir, config, log │ │ user 3 → … │ └─────────────────────────────────────────────┘ ``` **One process per user, isolated.** Each user's worker runs in its own directory, with its own config/secrets injected as environment variables, writing its own log. Users never see each other's anything.**Safe lifecycle.** Start, stop, restart. Graceful shutdown (SIGTERM) that escalates to a kill if a worker ignores it — and signals the whole process group, so children die too. No orphans.**Survives its own restart.** This is the one most process managers get wrong. When*WorkerDeck itself*restarts — a deploy, a code reload, a crash — your workers keep running, and the new manager re-adopts them instead of reporting them as dead. It matches processes by pid**and start time**, so a recycled pid can never be mistaken for your worker. (More below — this is worth understanding.)** Self-healing, safely (opt-in).**Turn on`auto_restart` and crashed workers come back on their own — but behind a circuit breaker: exponential backoff between attempts, and after too many crashes in a window the worker is parked in a`failed` state for a human, instead of hot-looping and taking the box down with it.**Live status.** Running, stopped, crashed, or failed — since when, last log lines, per user, at a glance. Zombie (defunct) processes are reported as gone, not falsely "running".**Reconciliation.** One`reconcile()` call compares what should be running against what actually is, cleans up stale bookkeeping, and (if enabled) recovers crashed workers. Point a cron job or timer at it and walk away.**A dashboard, included.** A small Flask control panel so you can see and drive everything without writing your own UI first.**Your worker is just a Python script.** WorkerDeck doesn't care what it does. Swap the example for yours and the lifecycle stays identical. ``` pip install workerdeck ``` That's the engine — zero dependencies, one class. See [Use it in your own code](#use-it-in-your-own-code) below. The included Flask dashboard is a demo/dev tool, not part of the library. Run it from a clone of this repo: ``` git clone https://github.com/gritfeld-design/workerdeck.git cd workerdeck pip install -e ".[dashboard]" python -m dashboard.app # open http://localhost:5000 ``` You'll see three demo users. Hit **Start** on one and watch its log stream in. The worker it runs is [ examples/scheduled_task_worker.py](/gritfeld-design/workerdeck/blob/main/examples/scheduled_task_worker.py) — a trivial task on a timer, standing in for whatever yours will do. The dashboard is a demo; the engine is the point. It's one class: ``` python from workerdeck.manager import WorkerManager mgr = WorkerManager( worker_script="path/to/your_worker.py", work_root="./_work", ) # start a user's worker, injecting their own config as env vars mgr.start(user_id=42, config={"API_KEY": user_key, "INTERVAL": "30"}) # check on it status = mgr.status(42) print(status.state, status.uptime_seconds, status.last_log_lines) # stop it cleanly mgr.stop(42) ``` Your worker reads its per-user config from the environment and runs until WorkerDeck stops it: ``` python import os, signal, sys api_key = os.environ["API_KEY"] # this user's own key user_id = os.environ["WORKER_USER_ID"] # always provided running = True signal.signal(signal.SIGTERM, lambda *_: globals().__setitem__("running", False)) while running: do_one_unit_of_work(api_key) # ← your work here ... sys.exit(0) ``` That's the whole contract. See [ examples/scheduled_task_worker.py](/gritfeld-design/workerdeck/blob/main/examples/scheduled_task_worker.py) for a complete, runnable version. Here's the failure mode WorkerDeck exists to prevent. A naive manager tracks its workers in memory — a dict of process handles. That works right up until the manager process itself restarts (you deployed, the code reloaded, it crashed and came back). Now that dict is empty. The workers are all still running — they're separate processes — but the manager has amnesia. It reports them as stopped. Click "start" and you get *a second copy* fighting the first for the same port. Click "stop" and nothing happens, because the manager no longer holds their handles. WorkerDeck avoids this by persisting a small registry to disk — each worker's pid and **start time** — and re-adopting live workers on startup: ``` mgr = WorkerManager(worker_script="worker.py", work_root="./_work") mgr.start(42) # worker running as some pid # ... later, the manager process is restarted entirely ... mgr = WorkerManager(worker_script="worker.py", work_root="./_work") mgr.status(42).state # "running" — it re-adopted the live worker mgr.stop(42) # and yes, it can still stop it ``` Why start time and not just pid? Because the OS recycles pid numbers. A pid that was your worker yesterday might be an unrelated process today. A live pid alone is not proof; a live pid whose start time matches what we recorded *is*. That check is the difference between "usually works" and "safe to restart". Crashes happen. Turn on `auto_restart` and let `reconcile()` bring workers back — with a brake so a genuinely-broken worker can't spin forever: ``` mgr = WorkerManager( worker_script="worker.py", work_root="./_work", auto_restart=True, max_crashes=5, # give up after 5 crashes... crash_window=300, # ...within 5 minutes ) # run this on a timer / cron. Each call: # - reports current state per user # - cleans up dead bookkeeping # - restarts crashed workers (backoff between tries) # - parks hopeless ones in state "failed" (no hot-loop) report = mgr.reconcile() # after you've fixed a failed worker's root cause: mgr.reset_failed(42) ``` Backoff is exponential (1s, 2s, 4s, …, capped), so a worker that crashes on startup doesn't hammer your machine. Once it trips the breaker it stays `failed` — visible, not silently retried into oblivion — until you clear it. Pass an `on_event` callback and WorkerDeck tells you about every meaningful state change — wire it to your logging or your alerting: ``` python def on_event(ev): # ev.user_id, ev.type, ev.timestamp, ev.detail if ev.type == "failed": alert(f"Worker for user {ev.user_id} gave up after " f"{ev.detail['crashes']} crashes") log.info("[%s] user %s %s", ev.type, ev.user_id, ev.detail) mgr = WorkerManager(worker_script="w.py", work_root="./_work", on_event=on_event) ``` Event types: `started` , `stopped` , `crashed` , `restarted` , `failed` , and `adopted` (a live worker re-attached after the manager restarted). Each carries relevant `detail` — pid, crash count, backoff seconds. A callback that raises is caught and logged, never allowed to take the manager down: your notifications can be buggy without breaking your workers. It's a **foundation**, not a finished platform. It deliberately leaves the parts that are specific to *your* product to you: **Auth & users**— the demo uses three hardcoded users. Bring your own users table and session auth; the`WorkerManager` calls don't change.**Persistence of**— WorkerDeck tracks process lifecycle, not your domain data. Store config in your own database.*what*each user runs**Scaling across machines**— this manages workers on one host. Multi-host orchestration is out of scope (and usually premature). It handles the annoying, easy-to-get-wrong part — per-user process lifecycle on a single host — so you can get to the part that's actually your product. Because that's where most projects actually start, and because the failure modes of per-user subprocess management are real and annoying long before you need Kubernetes: - a worker that **ignores SIGTERM**→ escalated to SIGKILL - child processes **orphaned** when a worker dies → whole process group signalled - a **zombie**(defunct) process that looks alive but isn't → reported as gone - one user's worker **crashing**→ isolated, and optionally auto-restarted - the manager **restarting after a deploy** and losing track of everything → workers re-adopted by pid + start time Every one of those is handled here, on one host, with no orchestration layer. Solve the real problem you have now. ``` git clone https://github.com/gritfeld-design/workerdeck.git cd workerdeck pip install -e ".[dev]" pytest -v ``` The test suite exercises the library's core claims: the start/stop lifecycle, re-adoption of live workers after a manager restart (matched by pid **and** start time), rejection of stale registry entries, the crash circuit breaker, and the event callback. Tests treat zombie (defunct) processes as dead — using the library's own liveness check — because a naive `os.kill(pid, 0)` reports zombies as alive. Ask me how I know. MIT — use it, change it, ship it. *WorkerDeck started life as the process-management core of a real multi-user platform, extracted and generalized. If it saves you the week it took to get right the first time, that's the point.* "Running per-user workers in production? I'm collecting failure modes — open an issue or mail me."