{"slug": "tracker-a-lightweight-self-hosted-document-system-for-agents-in-go", "title": "Tracker – a lightweight self-hosted document system for agents in Go", "summary": "A developer released Tracker, a lightweight self-hosted document system for coding agents built in Go, using Postgres for indexing and coordination with S3 or local file storage. The tool provides leases, versioned writes, and a task queue to prevent conflicts when multiple agents edit documents simultaneously.", "body_md": "A tiny, coordinated, document store for a fleet of coding agents. Postgres holds the index + lease/coordination state; content blobs live in local files or RustFS (S3). Single static Go binary, low footprint, reachable by agents over the network (e.g. LAN/Tailscale/ZeroTier).\n\n`mcp/tracker_mcp.py`\n\nis an MCP server (a self-contained `uv`\n\nscript — no install)\nthat exposes tracker to any coding agent: `search_docs`\n\n, `list_tags`\n\n,\n`list_folios`\n\n, `get_folio`\n\n, `read_doc`\n\n, `who_is_editing`\n\n, `create_doc`\n\n,\n`create_folio`\n\n, `update_doc`\n\n(lease + version-check + release, for you), `retag`\n\n(tags/metadata without a content rewrite), `list_actors`\n\n, and the task tools.\nConfigure per agent via env:\n`TRACKER_URL`\n\n, `TRACKER_ACTOR`\n\n(the agent's identity, stamped on writes),\n`TRACKER_TOKEN`\n\n(only if `API_TOKENS`\n\nis set).\n\nRegister with Claude Code:\n\n```\nclaude mcp add tracker --scope user \\\n  --env TRACKER_URL=http://127.0.0.1:8080 --env TRACKER_ACTOR=claude-code \\\n  -- uv run --quiet --script /path/to/tracker/mcp/tracker_mcp.py\n```\n\n`skills/tracker/SKILL.md`\n\nis the matching Claude Code skill (copy to\n`~/.claude/skills/tracker/`\n\n) describing when/how to consult tracker.\n\nAgents need a shared source of truth and a way to see **if a doc is already\nbeing written by another agent**. This is a database problem, not a\nknowledge-app problem — so: Postgres + S3/Files\n\n**Leases, not advisory locks.** A`doc_locks`\n\nrow with a TTL + heartbeat answers \"who is writing this right now\". A crashed agent's lease auto-expires, so it can never block a doc forever.**Two-layer write safety.** A write requires (a) a live lease the caller holds (`X-Lease-Token`\n\n) and (b)`If-Match: <version>`\n\noptimistic concurrency, so a stale or lease-less write can't clobber.**Content-addressed blobs.** Bytes are stored under`sha256/<hash>`\n\n(immutable, deduped) in either a local directory (`STORAGE_TYPE=file`\n\n) or an S3 bucket (`STORAGE_TYPE=s3`\n\n). Agents fetch them via a presigned URL or direct local URL.**Task queue.**`tasks`\n\nwith`FOR UPDATE SKIP LOCKED`\n\nclaiming — no two agents grab the same task.\n\nBoth Postgres and the service run via Docker Compose (no sudo needed). The\n`tracker`\n\ncontainer uses host networking, so it binds the loopback/LAN/Tailscale/ZeroTier\nIPs in `LISTEN_ADDR`\n\nand reaches Postgres (and optionally S3) on the host.\n\n```\ncp .env.example .env          # fill in secrets (e.g. STORAGE_TYPE=file) + set API_TOKENS\ndocker compose up -d          # starts pgvector Postgres + tracker\n```\n\nOps (the Makefile stamps the version from git into the binary):\n\n```\nmake deploy                            # rebuild image w/ version + restart (no sudo)\nmake version                           # show the version that would be embedded\ndocker compose logs -f tracker         # logs\ncurl http://127.0.0.1:8080/version     # version the running binary reports\n```\n\nThe version is `git describe --tags --always --dirty`\n\n— logged at startup, served\nat `/version`\n\n, and recorded in each backup's `manifest.json`\n\n.\n\nFor local dev without a container: `make build && set -a && . ./.env && set +a && ./tracker`\n\n.\n\n| Method | Path | Purpose |\n|---|---|---|\n| GET | `/healthz` · `/version` · `/openapi.yaml` · `/llms.txt` |\nhealth, version, spec, agent index |\n| POST · GET | `/docs` |\ncreate (`content` seeds v1); list/search (`?q=&mode=&kind=&tag=&view=&limit=&offset=` ) |\n| GET · PUT · PATCH | `/docs/{id}` |\nread `{document,content_url,lock}` ; write content (lease + `If-Match` ); relabel tags/metadata (no lease, no version bump) |\n| GET | `/docs/{id}/raw` · `/docs/{id}/revisions[/{v}/raw]` |\ncontent bytes; version history |\n| POST · GET · DELETE | `/docs/{id}/lock` |\nacquire/renew (`409` if held) · status · release |\n| GET | `/tags` |\ntag vocabulary with counts |\n| GET · POST | `/folios` · `/folios/{slug}` · `/folios/{slug}/files[/{filename}[/raw]]` |\ncollections + their files |\n| POST | `/tasks` · `/tasks/claim` · `/tasks/{id}/complete` |\ntask queue |\n| GET | `/actors` · `/actors/{name}/activity` |\nentity registry + activity |\n\nThe **authoritative reference is openapi.yaml** (served live at\n\n`/openapi.yaml`\n\n).**Conventions.** Every response is wrapped — a single resource under its type\n(`{\"document\":…}`\n\n, `{\"folio\":…}`\n\n, …), lists as `{\"<type>s\":[…],\"count\",\"total\",…}`\n\n,\nerrors as `{\"error\":{\"code\",\"message\",…}}`\n\nwith machine codes. Lists default to a\ntrimmed `view=summary`\n\n; `view=table`\n\nis a compact columnar `{cols,rows}`\n\n,\n`view=full`\n\nwhole objects. Search is `websearch_to_tsquery`\n\n(`mode=web`\n\ndefault;\n`mode=plain`\n\nfor strict AND).\n\nA **folio** is a little collection of related documents (think: a GitHub gist).\nIt's modelled tableless: the folio is itself a document with `kind='folio'`\n\nwhose `metadata`\n\nholds `{description, public, github_id, ...}`\n\n; its files are\ndocuments tagged `folio:<slug>`\n\nwith slug `<folio-slug>/<filename>`\n\n. So a folio\nfile inherits everything (versioning, leases, attribution, search). Create one\nwith `POST /folios`\n\nand add files with `POST /folios/{slug}/files`\n\n; import your\nrecent gists with `scripts/import_gists.py`\n\n.\n\n`{id}`\n\naccepts a UUID or a slug — including multi-segment folio slugs like\n`myfolio/file.md`\n\n; only `/raw`\n\nand `/lock`\n\nfor those still need the\n`/folios/{slug}/files/…`\n\nroute or the UUID.\n\nEvery **mutating** request must send an `X-Actor: <name>`\n\nheader naming the\nentity performing it (missing → `400`\n\n). That value is stamped into\n`created_by`\n\n/`updated_by`\n\n, the revision `author`\n\n, the lease `owner`\n\n, and task\n`claimed_by`\n\n, and upserted into the `actors`\n\nregistry. A write must come from\nthe entity that **holds the lease** (actor ≠ lease owner → `423`\n\n).\n\nOn a trusted (non-internet) network `X-Actor`\n\nis self-asserted attribution, not\nauthenticated identity. Set `API_TOKENS`\n\nand bind actor→token if you need it to\nbe tamper-proof.\n\nState lives in two places that must be captured together: Postgres (the index)\nand the blobs (the content). One self-contained tarball holds both —\n`db.dump`\n\n+ `blobs/`\n\n+ `manifest.json`\n\n. That tarball is the portable unit;\n\"R2 vs S3 vs a local directory\" is just where you keep it.\n\n``` php\nscripts/backup.sh                 # -> ./backups/tracker-backup-<ts>.tar.gz\nscripts/backup.sh --upload        # also push to R2/S3 (set BACKUP_S3_* in .env)\n\nscripts/restore.sh ./backups/tracker-backup-<ts>.tar.gz   # from a local file\nscripts/restore.sh --from-s3 tracker-backup-<ts>.tar.gz   # pull from R2/S3 first\ndocker compose up -d tracker                               # then start the service\n```\n\nThe backup dumps Postgres **first**, then copies blobs — and since writes are\nblob-first, every `content_key`\n\nin the dump is guaranteed to have its blob, so\nthe tarball is always internally consistent. Restore is verified round-trip:\nrestoring into a scratch DB+bucket reproduces the exact doc/blob counts and a\ntracker booted against it serves the content. `scripts/s3util.py`\n\nmoves blobs and\ntarballs to any S3-compatible store (RustFS, AWS S3, Cloudflare R2).\n\nTo restore on a **fresh machine**: clone the repo, create `.env`\n\n(point\n`S3_*`\n\n/`DATABASE_URL`\n\nat that host's RustFS+Postgres), `docker compose up -d postgres`\n\n, run `restore.sh`\n\n, then `docker compose up -d tracker`\n\n.\n\nBlobs are content-addressed and Postgres stores only the `sha256/<hash>`\n\nkey —\nnever the backend location — so switching between local files and S3 is just a\nblob copy plus a config flip. The `migrate-blobs`\n\nsubcommand does the copy:\n\n``` php\ntracker migrate-blobs --to file --blob-dir ./data/blobs   # S3 -> local files\ntracker migrate-blobs --to s3                             # local files -> S3\n#   --dry-run   hash-check + count, write nothing\n#   --verify    also re-read each blob from the destination\n```\n\nIt reads every referenced blob from the current backend (`STORAGE_TYPE`\n\n), verifies\neach against its hash, and writes it to the destination. It is **non-destructive**\n(the source is left intact) and **idempotent**. On success it prints the cutover\nstep — set `STORAGE_TYPE`\n\n(and `BLOB_DIR`\n\nfor file) in `.env`\n\nand restart — so the\nswitch is deliberate and reversible.\n\nRunning in \"production\" (lol). I've been using it heavily for several weeks. Known follow-ups:\n\n- pre-check lease/version before blob upload (rejected writes can leave GC-able orphans)\n- pgvector semantic search\n- scheduled backups\n- orphan/expired-lease GC\n- CI/CD, more pacakaging, etc.\n- Even simpler example MCP/skill usage\n\nPRs welcome but I can't promise I'll get to them!", "url": "https://wpnews.pro/news/tracker-a-lightweight-self-hosted-document-system-for-agents-in-go", "canonical_source": "https://github.com/chicagobuss/tracker/", "published_at": "2026-07-09 18:59:21+00:00", "updated_at": "2026-07-09 19:07:20.793874+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents"], "entities": ["Postgres", "S3", "Claude Code", "Docker", "Tailscale", "ZeroTier", "RustFS", "Go"], "alternates": {"html": "https://wpnews.pro/news/tracker-a-lightweight-self-hosted-document-system-for-agents-in-go", "markdown": "https://wpnews.pro/news/tracker-a-lightweight-self-hosted-document-system-for-agents-in-go.md", "text": "https://wpnews.pro/news/tracker-a-lightweight-self-hosted-document-system-for-agents-in-go.txt", "jsonld": "https://wpnews.pro/news/tracker-a-lightweight-self-hosted-document-system-for-agents-in-go.jsonld"}}