Sleeper Service: Agents as a Service. One agent. One task. A thousand of them Sleeper Service has launched as an open-source, self-hosted platform for running fleets of narrow, single-purpose AI agents as API endpoints, letting orchestrators such as n8n, Airflow, Temporal, cron, or plain code treat each agent like any other workflow node. The platform makes every edit to an agent's prompt, model, parameters, tools, or output schema a new immutable version, records which agent version and memory version ran each job, and supports pluggable inference across Anthropic, OpenAI, Google, and OpenRouter with per-job token and cost tracking. Sleeper Service is described as the agent-execution layer of CI Everywhere, decomposing back-office processes into small, observable, testable tasks. Agents as a Service. One agent. One task. A thousand of them. Sleeper Service is an open-source, self-hosted platform for running fleets of narrow, single-purpose AI agents as API endpoints. Instead of one autonomous agent trying to do everything, you define many small agents that each do one job well — repeatedly, auditably, and inside your existing orchestrated workflows. Every agent is a function: it takes an input, does analysis optionally using tools , and returns output in a shape you define. Your orchestrator n8n, Airflow, Temporal, cron, plain code treats it like any other workflow node. Sleeper Service is the agent-execution layer of CI Everywhere https://zero2data.substack.com/p/ci-everywhere : decompose back-office processes into small, observable and testable tasks, then use AI where the logic is teachable but not concrete enough for traditional automation. Agents do not need to impersonate whole employees; they take the specific decisions inside workflows that benefit from judgment. - Repeatable, not autonomous. Agents are built for processes that run over and over, where AI makes one decision or takes one action per invocation. - Auditable by construction. Every edit to an agent's prompt, model, parameters, tools, or output schema creates a new immutable version. Every job records exactly which agent version and memory version ran. - Owned by humans. Every agent belongs to a team, every team has an owner, and the risky switches — learning, memory, delegation — are owner-gated. - Pluggable inference. Anthropic, OpenAI, Google, OpenRouter — swap per agent, track tokens and cost per job, rolled up per agent. - Composable. Agents discover and delegate to each other permission-gated, depth-capped, cycle-checked, fully traced as a job tree . | Concept | What it is | |---|---| | Tenant | Top-level org. Holds the base system prompt every agent inherits. Multi-tenant out of the box. | | Team | Owns agents. Users join teams as owner / editor / viewer; every team keeps at least one owner. | | Agent | A named, single-purpose worker: prompt + model + tool and data store grants + output schema + options delegation, memory, learning, human escalation, spending limit . | | Version | Immutable snapshot of an agent's configuration. Jobs pin any version or alias dev / staging / prod ; promotion/rollback just repoints current or the alias. | | Job | One invocation of one agent version. Async by default with HMAC-signed webhook callbacks; ?sync=true for fast calls. Full event audit trail per job. | | Work item | A durable request for human attention, assigned to the agent's team. Pending memory changes and agent-raised business questions share one inbox while retaining their own approval rules and audit history. | | Data store | A registered storage backend S3/MinIO, Azure Blob, GCS, Box, local an agent is granted access to — path-prefix-scoped, read-only by default. Box grants pin a folder ID: credentials are downscoped to that subtree and paths resolve by name from it, so nothing outside is addressable. | | Event source | Webhook ingress that turns external events into jobs, with per-source secrets and dedup. Scheduling and polling stay in your orchestrator — Sleeper Service just receives. | | Hooks | Pre-hooks prompt-injection screening, default on and post-hooks output schema validation, opt-in PII redaction around every job. | | Memory / Learning | Opt-in per-agent memory document, versioned like everything else, steerable by signed per-job feedback votes. Optionally gated: owners approve every memory change, informed by an automatic eval run. | | Eval suite | Saved inputs + deterministic field checks per agent. Runs grade any version — branch comparison, promotion decisions, and the gate on memory edits. | API & auth — FastAPI with OpenAPI docs at /docs . Two kinds of API keys, hashed at rest: user keys act as a user, inherit team RBAC — the management plane and invoke keys tenant/team/agent-scoped, can only submit jobs, read results, post feedback — the data plane for orchestrators . Per-key rate limiting. RBAC enforced at the API: 404 for what you can't see, 403 for what you can't do. Execution — PydanticAI runtime: prompt sandwich tenant system prompt → agent prompt → memory , structured output enforced from the stored JSON Schema, per-version model params. Runtime guardrails: max iterations request cap and timeout s wall clock with first-class iteration limit / timeout statuses. Redis + arq workers with transient-error retries, exponential backoff, and dead-lettering; idempotency keys dedupe submissions. Tools & data — MCP server registry streamable HTTP / SSE, plus instance-superuser-approved stdio with per-version tool grants filtered to named tools. Caller user ctx is paired with server-derived principal identity and HMAC-signed using a per-MCP secret before forwarding. Data-store file tools list/read/write via fsspec are scoped to a granted path prefix. Payload file uploads go to MinIO. External links use a per-tenant domain allowlist; callbacks reject non-public destinations and may use a separate callback allowlist . Safety & spend — Prompt-injection screening over all untrusted content payload, files, links with rejected status and audit events: on by default, tenant-tunable add custom patterns, suppress a built-in rule that false-positives on your domain , disable-able per tenant or agent; memory writes and feedback comments pass the same screen poisoning defense . An opt-in second tier hooks.injection classifier model asks a cheap model for a structured verdict on anything the heuristics pass — fail-open, hard-timeboxed, and not billed to job spend. Monthly spending limits per agent: pre-flight refusal with auditable budget exceeded rows; per-job token/cost accounting via genai-prices. Provider credentials encrypted at rest Fernet . Events & notifications — Webhook event sources with {{path}} payload templates and dedup key path dedup. Apprise notification channels per team Slack/email/SMS/100+ services subscribe to operational alerts such as dead letter , budget , and eval regression , plus human attention when an inbox item needs action. Repeated operational alerts are deduplicated per agent per window; every distinct work item is delivered once. Channel URLs are a server-side outbound path like callbacks, so schemes are limited to a vetted set NOTIF EXTRA SCHEMES widens it, notif scheme allowlist narrows it per tenant and any host in one is re-resolved and rejected if it is not public. Delegation — Built-in list agents the rolodex: names, descriptions, I/O schemas and call agent tools, gated per agent none/team/tenant . Child jobs carry parent job id ; GET /v1/jobs/{id}/tree returns the audited tree. Depth caps and cycle detection. Memory & learning — Opt-in memory document injected after the agent prompt; the agent proposes edits via an update memory tool, applied post-run screened, size-capped . Learning adds signed single-job feedback URLs; votes fold deterministically into memory a − comment becomes a corrective rule — or, opt-in per tenant, an LLM fold distills feedback into generalizable lessons and condenses over-cap memory instead of dropping oldest-first, always falling back to the deterministic path. Governance: enabling any of this requires the team owner, and memory approval mode queues every memory change for owner approval — with the gating eval run's pass rate shown alongside — plus one-click rollback. Human escalation — Opt an agent into the built-in escalate to human tool and it can stop autonomous work with a first-class escalated result, recording the reason, severity, requested action, job and agent as a durable work item. The owning team is notified through its human attention channels. Editors or owners resolve business escalations; memory changes remain owner-only. Resolution is audited back onto the source job, and the job callback carries the work-item ID so the external orchestrator can route the human branch. Evals — Cases are saved inputs + checks equals , contains , in range , matches regex , is valid ; grading is deterministic and free. For logic beyond assertions, a code check runs an editor-supplied grade output function in a hard-capped sandbox — in-process Pydantic Monty https://github.com/pydantic/monty by default wall-clock/memory/recursion limits, no imports, filesystem, or network , or a hardened throwaway Docker container per call real CPython with packages, no network, capabilities dropped where the operator has enabled the docker runner backend. Runs execute through the normal pipeline hooks and tracing apply against any version, excluded from production spend. Pending memory versions auto-trigger a gated run; regressions alert the team. Admin UI — Ships in the api container server-rendered, no node toolchain : per-tenant dashboard with live-agent count, success rate, spend, and jobs/tokens charts; teams → agents with option badges and budget meters; a unified human-work inbox for memory approvals and agent escalations; version promotion and rollback; gating-eval pass rates against baseline; eval run history; job detail with payload, output, audit events, the delegation tree, and one-click dead-letter retry. Session login with the same users and RBAC as the API; optional per-tenant OIDC SSO Keycloak/Authentik/any discovery-speaking IdP sits alongside — configure it at PUT /v1/tenants/{id}/oidc and a "Continue with … SSO" button appears on the login page. Local auth always keeps working, and SSO users must already exist no just-in-time provisioning . Observability — Langfuse self-hosted, opt-in compose profile ingests every agent run via OTLP — prompts, responses, tokens, tool calls. The seam is plain OpenTelemetry, so any OTLP backend works. Ops — Everything ships as Docker Compose api, worker, Postgres, Redis, MinIO; --profile langfuse , --profile demo . Alembic migrations; CI via GitHub Actions. Hourly retention: per-tenant file TTLs and job payload retention rows and spend stats survive . Per-tenant worker concurrency caps. Deep health checks for api and worker. sleeper CLI: init bootstrap; refuses placeholder secrets , seed-models , demo-setup . A test provider runs the entire pipeline without vendor keys and test/flaky exercises retry/DLQ/alerting paths . | Per-tenant dashboard: live agents, success rate, spend, jobs & tokens | Teams → agents with option badges and budget meters | | Versions with promote, memory approval queue with gating-eval scores | Job detail: typed output, audit events, delegation tree | Everything ships as one Docker Compose stack. Your orchestrator and event feeds talk to the API; workers do the thinking; everything the platform learns or decides lands in Postgres, versioned. flowchart LR subgraph yours "Your side" O "Orchestrator