Production-grade
templatefor multi-agent LangGraph systems — hardened FastAPI API, domain packs, Helm, Terraform stubs, CI, and observability. Multi-tenant identity isyourlayer ([Security guide]).
A real session against the built-in mock provider — no API key, zero token cost, deterministic output (ideal for exploring the API, tests, and CI):
A deployable starting point for ML / data engineers who want a real agent stack—not a notebook demo. The default pipeline is Research → Analysis (ResearchAgent
AnalystAgent
), exposed over FastAPI with SSE streaming, session history, cost tracking, and pack-based routing.
Included: Docker, Helm chart, Terraform entry points (GKE / EKS / AKS), rate limiting, input validation, structured logging, Prometheus metrics, CI security scans, GHCR images with SBOM + Cosign on main
.
Not included: OAuth2/OIDC, per-tenant API keys, or billing. The built-in API_KEY
is a single shared Bearer secret for internal / single-tenant use.
…LangGraph Platform? LangGraph Platform is a legitimate, well-supported managed option — use it if you want a hosted control plane and don't want to run infrastructure yourself. This repo is for the opposite case: you self-host on your own Kubernetes cluster, keep full control of observability (Prometheus/OTel) and per-run cost data, and ship under the MIT license with zero vendor lock-in. Neither is objectively "better" — it's a build-vs-buy trade-off, and this template is for teams who'd rather own the stack.
…a generic agent-service-toolkit template? Beyond the usual FastAPI-around-LangGraph scaffolding, this repo ships production concerns already wired in: per-run USD budgets that return HTTP 402
on overrun, multi-version pack routing with canary traffic weights, and a supply chain that signs container images with Cosign and publishes an SBOM on every release.
Prerequisites: Python 3.12+, uv (package manager).
git clone https://github.com/brescou/langgraph-agent-stack.git
cd langgraph-agent-stack
uv sync --extra anthropic
cp .env.example .env # set ANTHROPIC_API_KEY
uv run uvicorn api.main:app --reload
Try it without any API key.SetLLM_PROVIDER=mock
in.env
(orLLM_PROVIDER=mock uv run uvicorn api.main:app --reload
) to get deterministic, zero-cost responses from every endpoint — useful for exploring the API, running the test suite, or CI, before wiring up a real provider. Web search is also mocked by default (SEARCH_PROVIDER=mock
); setSEARCH_PROVIDER=tavily
(orserpapi
) for real results.
curl -X POST http://localhost:8000/run \
-H "Content-Type: application/json" \
-d '{"query": "What are the latest advances in quantum computing?"}'
curl http://localhost:8000/packs
curl -X POST http://localhost:8000/packs/meeting_prep/run \
-H "Content-Type: application/json" \
-d '{"company": "Acme", "person": "Jane", "meeting_goal": "discovery"}'
Interactive API docs: http://localhost:8000/docs
(disabled when ENVIRONMENT=production
).
Cost: with a real provider, a research_analysis
run (6 LLM calls) costs roughly $0.01–0.05 on Claude Sonnet 5 pricing ($0.003 / $0.015 per 1K input/output tokens) — a rough order of magnitude from the pricing table in core/cost.py
, not a measured benchmark. Set PACK_DEFAULT_BUDGET_USD=0.50
to cap spend per run; requests over budget return HTTP 402
.
Client → FastAPI (auth · rate limit · validation)
→ PackRegistry / control_plane policies
→ domain_packs/* (LangGraph workflows)
→ agents/* (reusable agent nodes)
→ core/* (LLM · memory · security · cost · observability)
→ connectors/* (optional retrieval)
| Layer | Path | Role |
|---|---|---|
| HTTP | api/ |
|
FastAPI app (app.py ), middlewares, endpoints, pack router factory |
||
| Kernel | pack_kernel/ |
|
BaseDomainPack , PackRegistry , versioning, traffic split |
||
| Workflows | domain_packs/ |
|
| Packs grouped by domain — see | ||
agents/
ResearchAgent
, AnalystAgent
, …)control_plane/
control_plane/README.mdconnectors/
connectors/README.md(core/connectors.py
is a compat shim)core/
infra/
core/graph.py
is a compatibility shim (MultiAgentGraph
→ ResearchAnalysisPack
). New orchestration belongs in a domain pack.
13 built-in packs registered in pack_kernel/builtin_packs.py
:
| Category | Examples |
|---|---|
Research (domain_packs/research/ ) |
|
research_analysis , research_only , analysis_only |
|
Productivity (domain_packs/productivity/ ) |
|
summariser , meeting_prep , rfp_assistant , support_triage , executive_brief |
|
HR (domain_packs/hr/ ) |
|
talent_screening , job_description_writer , hr_policy_qa |
|
Finance (domain_packs/finance/ ) |
|
financial_memo |
|
Legal (domain_packs/legal/ ) |
|
contract_reviewer |
Each pack gets typed POST /packs/{pack_id}/run
and /run/stream
when schemas are declared. Versioning, traffic weights, and sticky sessions: GET /packs
, GET /packs/{id}/versions
, headers X-Pack-Version
/ X-Pack-Version-Used
.
Full catalogue and authoring guide: ** domain_packs/README.md**.
The HR, legal, and finance packs demonstrate the pack system on regulated-adjacent use cases, but they are off by default (REGULATED_PACKS_ENABLED=false
) — calling them with a valid body returns HTTP 403
until you complete the pack's COMPLIANCE.md
checklist and explicitly opt in (a body that fails schema validation returns 422
first, as on any pack route).
Third-party packs ship as regular Python packages declaring an entry point in
the langgraph_agent_stack.packs
group:
[project.entry-points."langgraph_agent_stack.packs"]
sentiment = "acme_packs.sentiment:SentimentPack"
Discovery is opt-in and allowlisted (PACK_PLUGINS_ENABLED=true
PACK_PLUGINS_ALLOWLIST=sentiment
): a plugin executes third-party
code, so nothing loads by default. At load time each class is validated
against the pack contract — BaseDomainPack
subclass, complete metadata, and
strict (extra="forbid"
) input/output schemas — and a broken plugin is
logged and skipped, never crashing startup. Built-in pack ids cannot be
overridden. Registered plugins get the same typed /packs/{id}/run
routes, versioning, and canary weights as built-ins. Packaging walkthrough: examples/custom_pack/README.md.
| Method | Path | Description |
|---|---|---|
GET |
||
/packs |
||
| List registered packs and metadata | ||
POST |
||
/packs/{pack_id}/run |
||
| Run a pack (typed body per pack schema) | ||
POST |
||
/packs/{pack_id}/run/stream |
||
| SSE stream for a pack | ||
POST |
||
/run , /run/stream |
||
Legacy routes → DEFAULT_PACK_ID (research_analysis ) |
||
POST |
||
/research |
||
| Research phase only | ||
GET |
||
/health , /ready |
||
| Probes | ||
GET |
||
/sessions/{id}/history |
||
| Session run history | ||
GET |
||
/metrics |
||
Prometheus (with observability extra) |
Responses include cost_usd
when cost tracking is active; HTTP 402 on budget exceed. See /docs
for request/response schemas.
Opt-in streamable-HTTP MCP endpoint that auto-generates one tool per registered domain pack (same Pydantic schemas as POST /packs/{id}/run
). Every tool call goes through the existing kernel (auth, rate limits, budgets → tool error mirroring HTTP 402, regulated-pack gating).
uv sync --extra mcp
MCP_SERVER_ENABLED=true
LLM_PROVIDER=mock # optional — CI / no API key
Endpoint: http://localhost:8000/mcp
(mounted only when the flag is on). Regulated packs are omitted from the tool list while REGULATED_PACKS_ENABLED=false
.
When API_KEY
is set, pass the same Bearer token the REST API expects — for example in Claude Desktop / a generic MCP client config:
{
"mcpServers": {
"langgraph-agent-stack": {
"url": "http://localhost:8000/mcp",
"headers": {
"Authorization": "Bearer YOUR_API_KEY"
}
}
}
}
Install the matching client transport for streamable HTTP; stdio is not shipped in this release.
Set LLM_PROVIDER
and install the matching extra. Details and gateway overrides: .env.example
.
| Provider | Value | Extra | Key env vars |
|---|---|---|---|
| Anthropic | anthropic |
||
--extra anthropic |
|||
ANTHROPIC_API_KEY |
|||
| OpenAI | openai |
||
--extra openai |
|||
OPENAI_API_KEY |
|||
google |
|||
--extra google |
|||
GOOGLE_API_KEY |
|||
| Bedrock | bedrock |
||
--extra bedrock |
|||
AWS_REGION , BEDROCK_MODEL |
|||
| Azure OpenAI | azure |
||
--extra openai |
|||
AZURE_OPENAI_* |
|||
| Ollama | ollama |
||
--extra ollama |
|||
OLLAMA_BASE_URL |
|||
| Mock (CI/dev) | mock |
||
| (none) | |||
| — |
Docker Compose
docker compose -f infra/docker-compose.yml up
docker compose -f infra/docker-compose.yml --profile redis up # Redis memory backend
Helm
helm install langgraph ./infra/helm/langgraph-agent-stack \
-f infra/helm/langgraph-agent-stack/values.prod.yaml \
--namespace langgraph-agents --create-namespace
Production: set secrets.existingSecret
(External Secrets Operator), config.environment=production
, networkPolicy.enabled=true
. Autoscaling defaults to KEDA on active_pipelines
(not CPU). See chart values.yaml
/ values.prod.yaml
.
Scaling note.MEMORY_BACKEND=sqlite
(default) is for development and single-replica deployments only — it's a local file, so state is not shared across pods. For production, multi-replica deployments, switch toMEMORY_BACKEND=redis
orMEMORY_BACKEND=postgres
so checkpointing and session history are consistent across replicas.
Terraform — entry points under infra/terraform/{gke,eks,aks}/
(no shared root module). Configure a remote backend before production apply. GKE module expects External Secrets Operator installed before ClusterSecretStore
resources.
Infra CI locally: make infra-check
(template Checkov profile). Before production hardening: make infra-check-prod
(checklist).
Rate limiting (memory or Redis), request body cap, prompt-injection / SSRF input validation, security headers, optional API_KEY
Bearer auth, graceful shutdown drain. Full model, env vars, K8s hardening, and scanning pipeline: ** docs/security.md**.
make help # all targets
make check # ruff + pyright (CI lint)
make test # 800+ tests, mocked by default — no network, no API key required
make eval # golden-dataset pack evaluations (deterministic)
make infra-check # helm lint + kubeconform + checkov
| Symptom | Cause / fix |
|---|---|
POST /run returns 502 with LLM provider 'anthropic' rejected the request credentials |
|
Your API key is missing or invalid. Set ANTHROPIC_API_KEY (or your provider's key) in .env , or set LLM_PROVIDER=mock to run without one. |
|
Responses say Mock insight 1: Key trend identified. |
|
You are on LLM_PROVIDER=mock (deterministic canned output, $0). Set a real provider + key in .env . |
|
GET /metrics returns 404 |
|
Prometheus metrics need the observability extra: uv sync --extra observability . |
|
Regulated pack (talent_screening , contract_reviewer , …) returns 403 |
|
Expected: these packs are gated behind REGULATED_PACKS_ENABLED=false until you complete the pack's COMPLIANCE.md . A 422 means your request body doesn't match the pack's input schema — check /docs . |
|
402 on /run or a pack route |
|
The per-run USD budget (PACK_DEFAULT_BUDGET_USD ) was exceeded. Raise it or unset it. |
|
/docs is missing |
|
Interactive docs are disabled when ENVIRONMENT=production . |
|
| State/history lost across replicas | MEMORY_BACKEND=sqlite (default) is single-replica only — switch to redis or postgres for multi-replica deployments. |
Not covered here? Check the pinned FAQ in Discussions Q&A before opening an issue.
evals/
runs golden datasets (evals/datasets/<pack_id>.yaml
) through the
real pack code with scripted LLM responses — deterministic, no network. Each
case declares an input, the scripted responses, and checks
(required_fields
, contains
, min_length
, numeric_range
, or
expect_error
for guard rejections). Compare two registered versions of a pack before shifting canary weights:
uv run python -m evals --pack summariser # one pack
uv run python -m evals --pack summariser --version 1.0 --compare 2.0
uv run python -m evals --all --json # CI-friendly output
Contributor workflow, pre-commit, and PR expectations: ** CONTRIBUTING.md**.
LangGraph patterns (standalone scripts, not served by the API): examples/README.md.
langgraph-agent-stack/
├── api/ # FastAPI (app.py, middleware, endpoints, router_factory)
├── pack_kernel/ # Pack contract + PackRegistry
├── domain_packs/ # research/, productivity/, hr/, finance/, legal/, common/
├── agents/ # Reusable LangGraph agents
├── connectors/ # Retrieval connector implementations
├── control_plane/ # Pack policies
├── core/ # Config, LLM, memory, security, cost, observability
├── infra/ # Dockerfile, compose, helm/, terraform/
├── examples/ # LangGraph pattern demos
├── tests/
├── docs/ # security.md, architecture.md, …
└── scripts/ # infra-devsecops.sh, …
| Doc | Contents |
|---|---|
domain_packs/README.mdconnectors/README.mdcontrol_plane/README.mdexamples/README.mdCONTRIBUTING.mdCHANGELOG.mdMIT © brescou