{"slug": "sleeper-service-agents-as-a-service-one-agent-one-task-a-thousand-of-them", "title": "Sleeper Service: Agents as a Service. One agent. One task. A thousand of them", "summary": "Sleeper Service has launched as an open-source, self-hosted platform for running fleets of narrow, single-purpose AI agents as API endpoints, letting orchestrators such as n8n, Airflow, Temporal, cron, or plain code treat each agent like any other workflow node. The platform makes every edit to an agent's prompt, model, parameters, tools, or output schema a new immutable version, records which agent version and memory version ran each job, and supports pluggable inference across Anthropic, OpenAI, Google, and OpenRouter with per-job token and cost tracking. Sleeper Service is described as the agent-execution layer of CI Everywhere, decomposing back-office processes into small, observable, testable tasks.", "body_md": "**Agents as a Service.** One agent. One task. A thousand of them.\n\nSleeper Service is an open-source, self-hosted platform for running fleets of narrow, single-purpose AI agents as API endpoints. Instead of one autonomous agent trying to do everything, you define many small agents that each do one job well — repeatedly, auditably, and inside your existing orchestrated workflows.\n\nEvery agent is a function: it takes an input, does analysis (optionally using tools), and returns output in a shape you define. Your orchestrator (n8n, Airflow, Temporal, cron, plain code) treats it like any other workflow node.\n\nSleeper Service is the agent-execution layer of [CI Everywhere](https://zero2data.substack.com/p/ci-everywhere): decompose back-office processes into small, observable and testable tasks, then use AI where the logic is teachable but not concrete enough for traditional automation. Agents do not need to impersonate whole employees; they take the specific decisions inside workflows that benefit from judgment.\n\n- **Repeatable, not autonomous.** Agents are built for processes that run over and over, where AI makes one decision or takes one action per invocation.\n- **Auditable by construction.** Every edit to an agent's prompt, model, parameters, tools, or output schema creates a new immutable version. Every job records exactly which agent version (and memory version) ran.\n- **Owned by humans.** Every agent belongs to a team, every team has an owner, and the risky switches — learning, memory, delegation — are owner-gated.\n- **Pluggable inference.** Anthropic, OpenAI, Google, OpenRouter — swap per agent, track tokens and cost per job, rolled up per agent.\n- **Composable.** Agents discover and delegate to each other (permission-gated, depth-capped, cycle-checked, fully traced as a job tree).\n\n| Concept | What it is | \n|---|---|\n| **Tenant** | Top-level org. Holds the base system prompt every agent inherits. Multi-tenant out of the box. | \n| **Team** | Owns agents. Users join teams as owner / editor / viewer; every team keeps at least one owner. | \n| **Agent** | A named, single-purpose worker: prompt + model + tool and data store grants + output schema + options (delegation, memory, learning, human escalation, spending limit). | \n| **Version** | Immutable snapshot of an agent's configuration. Jobs pin any version or alias ( `dev` /`staging` /`prod` ); promotion/rollback just repoints`current` or the alias. | \n| **Job** | One invocation of one agent version. Async by default with HMAC-signed webhook callbacks; `?sync=true` for fast calls. Full event audit trail per job. | \n| **Work item** | A durable request for human attention, assigned to the agent's team. Pending memory changes and agent-raised business questions share one inbox while retaining their own approval rules and audit history. | \n| **Data store** | A registered storage backend (S3/MinIO, Azure Blob, GCS, Box, local) an agent is granted access to — path-prefix-scoped, read-only by default. Box grants pin a folder ID: credentials are downscoped to that subtree and paths resolve by name from it, so nothing outside is addressable. | \n| **Event source** | Webhook ingress that turns external events into jobs, with per-source secrets and dedup. Scheduling and polling stay in your orchestrator — Sleeper Service just receives. | \n| **Hooks** | Pre-hooks (prompt-injection screening, default on) and post-hooks (output schema validation, opt-in PII redaction) around every job. | \n| **Memory / Learning** | Opt-in per-agent memory document, versioned like everything else, steerable by signed per-job feedback votes. Optionally gated: owners approve every memory change, informed by an automatic eval run. | \n| **Eval suite** | Saved inputs + deterministic field checks per agent. Runs grade any version — branch comparison, promotion decisions, and the gate on memory edits. | \n\n**API & auth** — FastAPI with OpenAPI docs at `/docs`. Two kinds of API keys, hashed at rest: *user keys* (act as a user, inherit team RBAC — the management plane) and *invoke keys* (tenant/team/agent-scoped, can only submit jobs, read results, post feedback — the data plane for orchestrators). Per-key rate limiting. RBAC enforced at the API: 404 for what you can't see, 403 for what you can't do.\n\n**Execution** — PydanticAI runtime: prompt sandwich (tenant system prompt → agent prompt → memory), structured output enforced from the stored JSON Schema, per-version model params. Runtime guardrails: `max_iterations` (request cap) and `timeout_s` (wall clock) with first-class `iteration_limit` / `timeout` statuses. Redis + arq workers with transient-error retries, exponential backoff, and dead-lettering; idempotency keys dedupe submissions.\n\n**Tools & data** — MCP server registry (streamable HTTP / SSE, plus instance-superuser-approved stdio) with per-version tool grants filtered to named tools. Caller `user_ctx` is paired with server-derived principal identity and HMAC-signed using a per-MCP secret before forwarding. Data-store file tools (list/read/write via fsspec) are scoped to a granted path prefix. Payload file uploads go to MinIO. External links use a per-tenant domain allowlist; callbacks reject non-public destinations and may use a separate `callback_allowlist`.\n\n**Safety & spend** — Prompt-injection screening over all untrusted content (payload, files, links) with `rejected` status and audit events: on by default, tenant-tunable (add custom patterns, suppress a built-in rule that false-positives on your domain), disable-able per tenant or agent; memory writes and feedback comments pass the same screen (poisoning defense). An opt-in second tier (`hooks.injection_classifier_model`) asks a cheap model for a structured verdict on anything the heuristics pass — fail-open, hard-timeboxed, and not billed to job spend. Monthly spending limits per agent: pre-flight refusal with auditable `budget_exceeded` rows; per-job token/cost accounting via genai-prices. Provider credentials encrypted at rest (Fernet).\n\n**Events & notifications** — Webhook event sources with `{{path}}` payload templates and `dedup_key_path` dedup. Apprise notification channels per team (Slack/email/SMS/100+ services) subscribe to operational alerts such as `dead_letter`, `budget`, and `eval_regression`, plus `human_attention` when an inbox item needs action. Repeated operational alerts are deduplicated per agent per window; every distinct work item is delivered once. Channel URLs are a server-side outbound path like callbacks, so schemes are limited to a vetted set (`NOTIF_EXTRA_SCHEMES` widens it, `notif_scheme_allowlist` narrows it per tenant) and any host in one is re-resolved and rejected if it is not public.\n\n**Delegation** — Built-in `list_agents` (the rolodex: names, descriptions, I/O schemas) and `call_agent` tools, gated per agent (none/team/tenant). Child jobs carry `parent_job_id`; `GET /v1/jobs/{id}/tree` returns the audited tree. Depth caps and cycle detection.\n\n**Memory & learning** — Opt-in memory document injected after the agent prompt; the agent proposes edits via an `update_memory` tool, applied post-run (screened, size-capped). Learning adds signed single-job feedback URLs; votes fold deterministically into memory (a − comment becomes a corrective rule) — or, opt-in per tenant, an LLM fold distills feedback into generalizable lessons and condenses over-cap memory instead of dropping oldest-first, always falling back to the deterministic path. Governance: enabling any of this requires the team owner, and `memory_approval` mode queues every memory change for owner approval — with the gating eval run's pass rate shown alongside — plus one-click rollback.\n\n**Human escalation** — Opt an agent into the built-in `escalate_to_human` tool and it can stop autonomous work with a first-class `escalated` result, recording the reason, severity, requested action, job and agent as a durable work item. The owning team is notified through its `human_attention` channels. Editors or owners resolve business escalations; memory changes remain owner-only. Resolution is audited back onto the source job, and the job callback carries the work-item ID so the external orchestrator can route the human branch.\n\n**Evals** — Cases are saved inputs + checks (`equals`, `contains`, `in_range`, `matches_regex`, `is_valid`); grading is deterministic and free. For logic beyond assertions, a `code` check runs an editor-supplied `grade(output)` function in a hard-capped sandbox — in-process [Pydantic Monty](https://github.com/pydantic/monty) by default (wall-clock/memory/recursion limits, no imports, filesystem, or network), or a hardened throwaway Docker container per call (real CPython with packages, no network, capabilities dropped) where the operator has enabled the `docker` runner backend. Runs execute through the normal pipeline (hooks and tracing apply) against any version, excluded from production spend. Pending memory versions auto-trigger a gated run; regressions alert the team.\n\n**Admin UI** — Ships in the api container (server-rendered, no node toolchain): per-tenant dashboard with live-agent count, success rate, spend, and jobs/tokens charts; teams → agents with option badges and budget meters; a unified human-work inbox for memory approvals and agent escalations; version promotion and rollback; gating-eval pass rates against baseline; eval run history; job detail with payload, output, audit events, the delegation tree, and one-click dead-letter retry. Session login with the same users and RBAC as the API; optional per-tenant OIDC SSO (Keycloak/Authentik/any discovery-speaking IdP) sits alongside — configure it at `PUT /v1/tenants/{id}/oidc` and a \"Continue with … SSO\" button appears on the login page. Local auth always keeps working, and SSO users must already exist (no just-in-time provisioning).\n\n**Observability** — Langfuse (self-hosted, opt-in compose profile) ingests every agent run via OTLP — prompts, responses, tokens, tool calls. The seam is plain OpenTelemetry, so any OTLP backend works.\n\n**Ops** — Everything ships as Docker Compose (api, worker, Postgres, Redis, MinIO; `--profile langfuse`, `--profile demo`). Alembic migrations; CI via GitHub Actions. Hourly retention: per-tenant file TTLs and job payload retention (rows and spend stats survive). Per-tenant worker concurrency caps. Deep health checks for api and worker. `sleeper` CLI: `init` (bootstrap; refuses placeholder secrets), `seed-models`, `demo-setup`. A `test` provider runs the entire pipeline without vendor keys (and `test/flaky` exercises retry/DLQ/alerting paths).\n\n| *Per-tenant dashboard: live agents, success rate, spend, jobs & tokens* | *Teams → agents with option badges and budget meters* | \n| *Versions with promote, memory approval queue with gating-eval scores* | *Job detail: typed output, audit events, delegation tree* | \n\nEverything ships as one Docker Compose stack. Your orchestrator and event feeds talk to the API; workers do the thinking; everything the platform learns or decides lands in Postgres, versioned.\n\n```\nflowchart LR\n  subgraph yours [\"Your side\"]\n    O[\"Orchestrator<br/>n8n · Airflow · Temporal · cron · code\"]\n    F[\"Event feeds\"]\n    U[\"Browser\"]\n  end\n\n  subgraph stack [\"Sleeper Service — one docker compose\"]\n    API[\"<b>api</b> — FastAPI<br/>API keys · RBAC · rate limits<br/>admin UI · OIDC\"]\n    R[(\"<b>redis</b><br/>arq job queue\")]\n    W[\"<b>worker</b><br/>pre-hooks → PydanticAI loop → post-hooks<br/>injection screen · iteration/timeout/budget guards<br/>schema check · memory writes · evals\"]\n    PG[(\"<b>postgres</b><br/>tenants · agents · versions<br/>jobs · memory · evals\")]\n    MIO[(\"<b>minio</b><br/>payload files\")]\n    SBX[\"code runners<br/>monty in-process · docker throwaway\"]\n    LFU[\"<b>langfuse</b> (opt-in)<br/>traces · tokens · costs\"]\n  end\n\n  subgraph ext [\"External services\"]\n    LLM[\"Model providers<br/>Anthropic · OpenAI · Google · OpenRouter\"]\n    MCP[\"Your MCP servers\"]\n    DST[(\"Data stores<br/>S3 · Azure Blob · GCS · Box · local\")]\n    APP[\"Slack · email · SMS · 100+<br/>via Apprise\"]\n  end\n\n  O -- \"submit job (invoke key)\" --> API\n  F -- \"signed webhooks, deduped\" --> API\n  U -- \"/ui · /docs\" --> API\n  API -- \"enqueue\" --> R\n  R -- \"run_job\" --> W\n  API <--> PG\n  API <--> MIO\n  W <--> PG\n  W -- \"prompt sandwich ⇄ structured output\" --> LLM\n  W -- \"granted tools\" --> MCP\n  W -- \"granted file tools\" --> DST\n  W -. \"traces\" .-> LFU\n  W -- \"eval code graders\" --> SBX\n  W -- \"alerts: dead-letter · budget · regression\" --> APP\n  W -- \"HMAC-signed callback + feedback URL\" --> O\n```\n\nData flows worth noting:\n\n- **Two planes, two key kinds.** Orchestrators hold*invoke keys* (submit/read/feedback only); humans and management tooling use*user keys* or sessions. Event feeds hold only per-source webhook secrets — never platform keys.\n- **Nothing untrusted touches a prompt unscreened.** Payloads, fetched links, feedback comments, and memory writes all pass the same injection screen before the model sees them or anything persists.\n- **Every write that changes behavior is a version.** Agent configs, memory documents, and promotions are immutable rows in Postgres; a job records exactly which of each it ran with.\n- **Results push, don't poll.** Workers deliver HMAC-signed callbacks with retries; exhausted retries dead-letter the job and page the owning team via Apprise.\n\n```\nsequenceDiagram\n  autonumber\n  participant O as Orchestrator\n  participant A as api\n  participant R as redis/arq\n  participant W as worker\n  participant P as Model provider\n  participant T as MCP · data stores · agents\n\n  O->>A: POST /v1/agents/{id}/jobs\n  A->>A: auth · rate limit · idempotency · budget pre-flight\n  A->>R: enqueue\n  A-->>O: 202 (job id)\n  R->>W: run_job\n  W->>W: pre-hooks: injection screen\n  loop until done (≤ max_iterations, ≤ timeout_s, budget checked between calls)\n    W->>P: model call (tenant prompt + agent prompt + memory + payload)\n    P-->>W: tool calls / structured output\n    W->>T: MCP tools · file tools · call_agent delegation\n  end\n  W->>W: post-hooks: schema check · redaction · memory proposal (screened)\n  W-->>O: HMAC-signed callback (+ signed feedback URL)\n  O->>A: GET /v1/jobs/{id} · POST feedback vote\n```\n\nPython / FastAPI, PydanticAI agent runtime, Postgres, Redis + arq workers, MCP for tool access, fsspec for data stores, pluggable sandboxed code runners, Langfuse for tracing.\n\n```\ngit clone https://github.com/willjohnson/sleeper-service.git && cd sleeper-service\ncp .env.example .env        # set SECRET_KEY, MINIO_*, REDIS_PASSWORD, and a provider API key\ndocker compose up -d\ndocker compose exec api sleeper init          # first tenant, team, superuser → prints your API key\ndocker compose exec api sleeper seed-models   # register starter models (incl. keyless test provider)\n```\n\n**Why MinIO credentials up front?** They guard the payload bucket holding\nevery tenant's uploads, and BYO s3 endpoints are an intended feature — a\ntenant admin can point a data store at any endpoint the worker reaches,\nincluding this one. So there is no shipped default to leave unrotated:\ncompose refuses to start without the pair. Rotating later needs\n`docker compose up -d --force-recreate minio` and a matching update to any\ndata store configured with the old pair.\n\n**Redis also requires a credential up front.** Set `REDIS_PASSWORD` to a\nURL-safe random value; Compose uses it for the server, health check, API, and\nworker and refuses to start without it. Host-side tools should use an\nauthenticated `REDIS_URL`, as shown in `.env.example`.\n\n**Deploying anywhere shared?** The optional Langfuse profile bootstraps itself\nfrom `.env` and compose defaults: project keys (`LANGFUSE_PUBLIC_KEY` /\n`LANGFUSE_SECRET_KEY`), an admin login (` LANGFUSE_INIT_USER_PASSWORD`), and\n`LANGFUSE_SALT` / `LANGFUSE_ENCRYPTION_KEY`. The dev placeholders are public\nknowledge, and Langfuse stores full prompt/response traces — set random values\n**before the profile's first boot**. `sleeper init` warns if it sees the defaults.\n\nCreate an agent, give it a version, run a job:\n\n```\n# The agent is the stable identity...\ncurl -X POST localhost:8000/v1/agents \\\n  -H \"Authorization: Bearer $SLEEPER_KEY\" \\\n  -d '{\"team_id\": \"…\", \"name\": \"risk-analyzer\", \"description\": \"Assesses business risk\"}'\n\n# ...its configuration lives in immutable versions (first one auto-promotes)\ncurl -X POST localhost:8000/v1/agents/$AGENT_ID/versions \\\n  -H \"Authorization: Bearer $SLEEPER_KEY\" \\\n  -d '{\n    \"model\": \"anthropic/claude-sonnet-5\",\n    \"prompt\": \"Assess business risk for the event in the payload.\",\n    \"output_schema\": {\n      \"type\": \"object\",\n      \"properties\": {\n        \"risk_level\": {\"enum\": [\"low\", \"medium\", \"high\"]},\n        \"factors\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\n        \"summary\": {\"type\": \"string\"}\n      }\n    }\n  }'\n\n# Submit a job (async — result arrives at your callback, HMAC-signed)\ncurl -X POST localhost:8000/v1/agents/$AGENT_ID/jobs \\\n  -H \"Authorization: Bearer $SLEEPER_KEY\" \\\n  -d '{\n    \"context\": {\"prompt\": \"AAPL dropped 6% in 20 minutes; storm warnings in STL\"},\n    \"callback_url\": \"https://yourapp.com/hooks/risk\"\n  }'\n# → 202 { \"id\": … }         also pollable at GET /v1/jobs/{id}\ndocker compose exec api sleeper demo-setup    # demo tenant, agents, event sources, reference data, alerts\ndocker compose --profile demo up -d           # external poller + alert sink\ndocker compose logs -f demo-poller\n```\n\nA poller script (playing the role of *your* orchestrator — it holds only webhook secrets, no platform key) posts synthetic market/weather events. The `risk-analyzer` reads a risk playbook from a granted S3 data store, and on high risk discovers and **delegates** to a `notifier` agent — auditable as a job tree. Along the way: duplicate events are deduped, an injected prompt is caught and logged, and a deliberately flaky agent retries, dead-letters, and pages the demo alert channel. Add `--profile langfuse` for traces at `localhost:3000`.\n\nOther things people build with this pattern: accounts-receivable agents matching deposits to invoices, customer-service agents answering tickets, classification and enrichment steps inside data pipelines.\n\n-  Core: tenants, teams, agents, versioning, jobs, callbacks *(Phases 0–1)*\n-  Hooks, spending limits, MCP tool grants, data stores, event sources, alerting *(Phase 2)*\n-  Delegation, memory, feedback-driven learning *(Phase 3)*\n-  Eval harness + memory approval governance *(Phase 4)*\n-  Admin UI: dashboard, promotion, memory approvals, job trees *(Phase 4)*\n-  OIDC login, version aliases, sandboxed code runners (in-process + docker) *(Phase 4)*\n- Opt-in LLM tiers: injection classifier, memory fold & compaction\n- Unified human-work inbox, agent escalation, and human-attention notifications\n- Hosted sandbox backends (E2B / Modal) — drop-in registry extension, if ever needed\n\nSee [docs/BUILD_PLAN.md](https://github.com/willjohnson/sleeper-service/blob/main/docs/BUILD_PLAN.md) for the full plan, data model, and decision log.\n\nThe *Sleeper Service* is a General Systems Vehicle from Iain M. Banks' *Excession* — an eccentric ship that spent decades quietly building and maintaining a fleet of eighty thousand autonomous units, ready the moment they were needed. That's the idea here: not one agent doing everything, but a service that keeps a fleet of narrow, reliable agents on station.\n\nApache-2.0", "url": "https://wpnews.pro/news/sleeper-service-agents-as-a-service-one-agent-one-task-a-thousand-of-them", "canonical_source": "https://github.com/willjohnson/sleeper-service", "published_at": "2026-09-21 11:30:18+00:00", "updated_at": "2026-09-21 11:53:58.098216+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "developer-tools", "artificial-intelligence"], "entities": ["Sleeper Service", "CI Everywhere", "Anthropic", "OpenAI", "Google", "OpenRouter", "FastAPI", "n8n"], "alternates": {"html": "https://wpnews.pro/news/sleeper-service-agents-as-a-service-one-agent-one-task-a-thousand-of-them", "markdown": "https://wpnews.pro/news/sleeper-service-agents-as-a-service-one-agent-one-task-a-thousand-of-them.md", "text": "https://wpnews.pro/news/sleeper-service-agents-as-a-service-one-agent-one-task-a-thousand-of-them.txt", "jsonld": "https://wpnews.pro/news/sleeper-service-agents-as-a-service-one-agent-one-task-a-thousand-of-them.jsonld"}}