Sleeper Service: Agents as a Service. One agent. One task. A thousand of them

Agents as a Service. One agent. One task. A thousand of them.

Sleeper Service is an open-source, self-hosted platform for running fleets of narrow, single-purpose AI agents as API endpoints. Instead of one autonomous agent trying to do everything, you define many small agents that each do one job well — repeatedly, auditably, and inside your existing orchestrated workflows.

Every agent is a function: it takes an input, does analysis (optionally using tools), and returns output in a shape you define. Your orchestrator (n8n, Airflow, Temporal, cron, plain code) treats it like any other workflow node.

Sleeper Service is the agent-execution layer of CI Everywhere: decompose back-office processes into small, observable and testable tasks, then use AI where the logic is teachable but not concrete enough for traditional automation. Agents do not need to impersonate whole employees; they take the specific decisions inside workflows that benefit from judgment.

• Repeatable, not autonomous. Agents are built for processes that run over and over, where AI makes one decision or takes one action per invocation.
• Auditable by construction. Every edit to an agent's prompt, model, parameters, tools, or output schema creates a new immutable version. Every job records exactly which agent version (and memory version) ran.
• Owned by humans. Every agent belongs to a team, every team has an owner, and the risky switches — learning, memory, delegation — are owner-gated.
• Pluggable inference. Anthropic, OpenAI, Google, OpenRouter — swap per agent, track tokens and cost per job, rolled up per agent.
• Composable. Agents discover and delegate to each other (permission-gated, depth-capped, cycle-checked, fully traced as a job tree).

API & auth — FastAPI with OpenAPI docs at /docs. Two kinds of API keys, hashed at rest: user keys (act as a user, inherit team RBAC — the management plane) and invoke keys (tenant/team/agent-scoped, can only submit jobs, read results, post feedback — the data plane for orchestrators). Per-key rate limiting. RBAC enforced at the API: 404 for what you can't see, 403 for what you can't do.

Execution — PydanticAI runtime: prompt sandwich (tenant system prompt → agent prompt → memory), structured output enforced from the stored JSON Schema, per-version model params. Runtime guardrails: max_iterations (request cap) and timeout_s (wall clock) with first-class iteration_limit / timeout statuses. Redis + arq workers with transient-error retries, exponential backoff, and dead-lettering; idempotency keys dedupe submissions.

Tools & data — MCP server registry (streamable HTTP / SSE, plus instance-superuser-approved stdio) with per-version tool grants filtered to named tools. Caller user_ctx is paired with server-derived principal identity and HMAC-signed using a per-MCP secret before forwarding. Data-store file tools (list/read/write via fsspec) are scoped to a granted path prefix. Payload file uploads go to MinIO. External links use a per-tenant domain allowlist; callbacks reject non-public destinations and may use a separate callback_allowlist.

Safety & spend — Prompt-injection screening over all untrusted content (payload, files, links) with rejected status and audit events: on by default, tenant-tunable (add custom patterns, suppress a built-in rule that false-positives on your domain), disable-able per tenant or agent; memory writes and feedback comments pass the same screen (poisoning defense). An opt-in second tier (hooks.injection_classifier_model) asks a cheap model for a structured verdict on anything the heuristics pass — fail-open, hard-timeboxed, and not billed to job spend. Monthly spending limits per agent: pre-flight refusal with auditable budget_exceeded rows; per-job token/cost accounting via genai-prices. Provider credentials encrypted at rest (Fernet).

Events & notifications — Webhook event sources with {{path}} payload templates and dedup_key_path dedup. Apprise notification channels per team (Slack/email/SMS/100+ services) subscribe to operational alerts such as dead_letter, budget, and eval_regression, plus human_attention when an inbox item needs action. Repeated operational alerts are deduplicated per agent per window; every distinct work item is delivered once. Channel URLs are a server-side outbound path like callbacks, so schemes are limited to a vetted set (NOTIF_EXTRA_SCHEMES widens it, notif_scheme_allowlist narrows it per tenant) and any host in one is re-resolved and rejected if it is not public.

Delegation — Built-in list_agents (the rolodex: names, descriptions, I/O schemas) and call_agent tools, gated per agent (none/team/tenant). Child jobs carry parent_job_id; GET /v1/jobs/{id}/tree returns the audited tree. Depth caps and cycle detection.

Memory & learning — Opt-in memory document injected after the agent prompt; the agent proposes edits via an update_memory tool, applied post-run (screened, size-capped). Learning adds signed single-job feedback URLs; votes fold deterministically into memory (a − comment becomes a corrective rule) — or, opt-in per tenant, an LLM fold distills feedback into generalizable lessons and condenses over-cap memory instead of dropping oldest-first, always falling back to the deterministic path. Governance: enabling any of this requires the team owner, and memory_approval mode queues every memory change for owner approval — with the gating eval run's pass rate shown alongside — plus one-click rollback.

Human escalation — Opt an agent into the built-in escalate_to_human tool and it can stop autonomous work with a first-class escalated result, recording the reason, severity, requested action, job and agent as a durable work item. The owning team is notified through its human_attention channels. Editors or owners resolve business escalations; memory changes remain owner-only. Resolution is audited back onto the source job, and the job callback carries the work-item ID so the external orchestrator can route the human branch.

Evals — Cases are saved inputs + checks (equals, contains, in_range, matches_regex, is_valid); grading is deterministic and free. For logic beyond assertions, a code check runs an editor-supplied grade(output) function in a hard-capped sandbox — in-process Pydantic Monty by default (wall-clock/memory/recursion limits, no imports, filesystem, or network), or a hardened throwaway Docker container per call (real CPython with packages, no network, capabilities dropped) where the operator has enabled the docker runner backend. Runs execute through the normal pipeline (hooks and tracing apply) against any version, excluded from production spend. Pending memory versions auto-trigger a gated run; regressions alert the team.

Admin UI — Ships in the api container (server-rendered, no node toolchain): per-tenant dashboard with live-agent count, success rate, spend, and jobs/tokens charts; teams → agents with option badges and budget meters; a unified human-work inbox for memory approvals and agent escalations; version promotion and rollback; gating-eval pass rates against baseline; eval run history; job detail with payload, output, audit events, the delegation tree, and one-click dead-letter retry. Session login with the same users and RBAC as the API; optional per-tenant OIDC SSO (Keycloak/Authentik/any discovery-speaking IdP) sits alongside — configure it at PUT /v1/tenants/{id}/oidc and a "Continue with … SSO" button appears on the login page. Local auth always keeps working, and SSO users must already exist (no just-in-time provisioning).

Observability — Langfuse (self-hosted, opt-in compose profile) ingests every agent run via OTLP — prompts, responses, tokens, tool calls. The seam is plain OpenTelemetry, so any OTLP backend works.

Ops — Everything ships as Docker Compose (api, worker, Postgres, Redis, MinIO; --profile langfuse, --profile demo). Alembic migrations; CI via GitHub Actions. Hourly retention: per-tenant file TTLs and job payload retention (rows and spend stats survive). Per-tenant worker concurrency caps. Deep health checks for api and worker. sleeper CLI: init (bootstrap; refuses placeholder secrets), seed-models, demo-setup. A test provider runs the entire pipeline without vendor keys (and test/flaky exercises retry/DLQ/alerting paths).

| Per-tenant dashboard: live agents, success rate, spend, jobs & tokens | Teams → agents with option badges and budget meters | | Versions with promote, memory approval queue with gating-eval scores | Job detail: typed output, audit events, delegation tree |

Everything ships as one Docker Compose stack. Your orchestrator and event feeds talk to the API; workers do the thinking; everything the platform learns or decides lands in Postgres, versioned.

flowchart LR
  subgraph yours ["Your side"]
    O["Orchestrator<br/>n8n · Airflow · Temporal · cron · code"]
    F["Event feeds"]
    U["Browser"]
  end

  subgraph stack ["Sleeper Service — one docker compose"]
    API["<b>api</b> — FastAPI<br/>API keys · RBAC · rate limits<br/>admin UI · OIDC"]
    R[("<b>redis</b><br/>arq job queue")]
    W["<b>worker</b><br/>pre-hooks → PydanticAI loop → post-hooks<br/>injection screen · iteration/timeout/budget guards<br/>schema check · memory writes · evals"]
    PG[("<b>postgres</b><br/>tenants · agents · versions<br/>jobs · memory · evals")]
    MIO[("<b>minio</b><br/>payload files")]
    SBX["code runners<br/>monty in-process · docker throwaway"]
    LFU["<b>langfuse</b> (opt-in)<br/>traces · tokens · costs"]
  end

  subgraph ext ["External services"]
    LLM["Model providers<br/>Anthropic · OpenAI · Google · OpenRouter"]
    MCP["Your MCP servers"]
    DST[("Data stores<br/>S3 · Azure Blob · GCS · Box · local")]
    APP["Slack · email · SMS · 100+<br/>via Apprise"]
  end

  O -- "submit job (invoke key)" --> API
  F -- "signed webhooks, deduped" --> API
  U -- "/ui · /docs" --> API
  API -- "enqueue" --> R
  R -- "run_job" --> W
  API <--> PG
  API <--> MIO
  W <--> PG
  W -- "prompt sandwich ⇄ structured output" --> LLM
  W -- "granted tools" --> MCP
  W -- "granted file tools" --> DST
  W -. "traces" .-> LFU
  W -- "eval code graders" --> SBX
  W -- "alerts: dead-letter · budget · regression" --> APP
  W -- "HMAC-signed callback + feedback URL" --> O

Data flows worth noting:

• Two planes, two key kinds. Orchestrators holdinvoke keys (submit/read/feedback only); humans and management tooling useuser keys or sessions. Event feeds hold only per-source webhook secrets — never platform keys.
• Nothing untrusted touches a prompt unscreened. Payloads, fetched links, feedback comments, and memory writes all pass the same injection screen before the model sees them or anything persists.
• Every write that changes behavior is a version. Agent configs, memory documents, and promotions are immutable rows in Postgres; a job records exactly which of each it ran with.
• Results push, don't poll. Workers deliver HMAC-signed callbacks with retries; exhausted retries dead-letter the job and page the owning team via Apprise.

sequenceDiagram
  autonumber
  participant O as Orchestrator
  participant A as api
  participant R as redis/arq
  participant W as worker
  participant P as Model provider
  participant T as MCP · data stores · agents

  O->>A: POST /v1/agents/{id}/jobs
  A->>A: auth · rate limit · idempotency · budget pre-flight
  A->>R: enqueue
  A-->>O: 202 (job id)
  R->>W: run_job
  W->>W: pre-hooks: injection screen
  loop until done (≤ max_iterations, ≤ timeout_s, budget checked between calls)
    W->>P: model call (tenant prompt + agent prompt + memory + payload)
    P-->>W: tool calls / structured output
    W->>T: MCP tools · file tools · call_agent delegation
  end
  W->>W: post-hooks: schema check · redaction · memory proposal (screened)
  W-->>O: HMAC-signed callback (+ signed feedback URL)
  O->>A: GET /v1/jobs/{id} · POST feedback vote

Python / FastAPI, PydanticAI agent runtime, Postgres, Redis + arq workers, MCP for tool access, fsspec for data stores, pluggable sandboxed code runners, Langfuse for tracing.

git clone https://github.com/willjohnson/sleeper-service.git && cd sleeper-service
cp .env.example .env        # set SECRET_KEY, MINIO_*, REDIS_PASSWORD, and a provider API key
docker compose up -d
docker compose exec api sleeper init          # first tenant, team, superuser → prints your API key
docker compose exec api sleeper seed-models   # register starter models (incl. keyless test provider)

Why MinIO credentials up front? They guard the payload bucket holding every tenant's uploads, and BYO s3 endpoints are an intended feature — a tenant admin can point a data store at any endpoint the worker reaches, including this one. So there is no shipped default to leave unrotated: compose refuses to start without the pair. Rotating later needs docker compose up -d --force-recreate minio and a matching update to any data store configured with the old pair.

Redis also requires a credential up front. Set REDIS_PASSWORD to a URL-safe random value; Compose uses it for the server, health check, API, and worker and refuses to start without it. Host-side tools should use an authenticated REDIS_URL, as shown in .env.example.

Deploying anywhere shared? The optional Langfuse profile bootstraps itself from .env and compose defaults: project keys (LANGFUSE_PUBLIC_KEY / LANGFUSE_SECRET_KEY), an admin login ( LANGFUSE_INIT_USER_PASSWORD), and LANGFUSE_SALT / LANGFUSE_ENCRYPTION_KEY. The dev placeholders are public knowledge, and Langfuse stores full prompt/response traces — set random values before the profile's first boot. sleeper init warns if it sees the defaults.

Create an agent, give it a version, run a job:

curl -X POST localhost:8000/v1/agents \
  -H "Authorization: Bearer $SLEEPER_KEY" \
  -d '{"team_id": "…", "name": "risk-analyzer", "description": "Assesses business risk"}'

curl -X POST localhost:8000/v1/agents/$AGENT_ID/versions \
  -H "Authorization: Bearer $SLEEPER_KEY" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "prompt": "Assess business risk for the event in the payload.",
    "output_schema": {
      "type": "object",
      "properties": {
        "risk_level": {"enum": ["low", "medium", "high"]},
        "factors": {"type": "array", "items": {"type": "string"}},
        "summary": {"type": "string"}
      }
    }
  }'

curl -X POST localhost:8000/v1/agents/$AGENT_ID/jobs \
  -H "Authorization: Bearer $SLEEPER_KEY" \
  -d '{
    "context": {"prompt": "AAPL dropped 6% in 20 minutes; storm warnings in STL"},
    "callback_url": "https://yourapp.com/hooks/risk"
  }'
docker compose exec api sleeper demo-setup    # demo tenant, agents, event sources, reference data, alerts
docker compose --profile demo up -d           # external poller + alert sink
docker compose logs -f demo-poller

A poller script (playing the role of your orchestrator — it holds only webhook secrets, no platform key) posts synthetic market/weather events. The risk-analyzer reads a risk playbook from a granted S3 data store, and on high risk discovers and delegates to a notifier agent — auditable as a job tree. Along the way: duplicate events are deduped, an injected prompt is caught and logged, and a deliberately flaky agent retries, dead-letters, and pages the demo alert channel. Add --profile langfuse for traces at localhost:3000.

Other things people build with this pattern: accounts-receivable agents matching deposits to invoices, customer-service agents answering tickets, classification and enrichment steps inside data pipelines.

• Core: tenants, teams, agents, versioning, jobs, callbacks (Phases 0–1)
• Hooks, spending limits, MCP tool grants, data stores, event sources, alerting (Phase 2)
• Delegation, memory, feedback-driven learning (Phase 3)
• Eval harness + memory approval governance (Phase 4)
• Admin UI: dashboard, promotion, memory approvals, job trees (Phase 4)
• OIDC login, version aliases, sandboxed code runners (in-process + docker) (Phase 4)
• Opt-in LLM tiers: injection classifier, memory fold & compaction
• Unified human-work inbox, agent escalation, and human-attention notifications
• Hosted sandbox backends (E2B / Modal) — drop-in registry extension, if ever needed

See docs/BUILD_PLAN.md for the full plan, data model, and decision log.

The Sleeper Service is a General Systems Vehicle from Iain M. Banks' Excession — an eccentric ship that spent decades quietly building and maintaining a fleet of eighty thousand autonomous units, ready the moment they were needed. That's the idea here: not one agent doing everything, but a service that keeps a fleet of narrow, reliable agents on station.

Apache-2.0