A single Python agent that runs nightly and produces a serendipity dispatch: one surprising discovery, one deep dive, one build idea, and one markets note. Not morning news — things the owner wouldn't have found on their own.
Feed this document to an AI assistant (Muse, etc.) and ask it to implement the agent. Everything below is a specification, not code.
- What it does (pipeline)
Discovery (no LLM) → interest-aware ranking → URL verification →
LLM curation (picks by index) → deterministic rendering → dispatch + teaser
Key design rule: the LLM only selects (returns indices). All fetching, rendering, and link insertion is deterministic code. The model can never hallucinate a URL because it never writes one.
- Schedule
Runs once nightly at 2:00 AM local time via cron.
Single-flight: use a lock file so overlapping runs are impossible.
Timeout budget: ~15 minutes max (LLM retries are the long pole).
- Discovery sources (no LLM, pure HTTP)
Source
What to fetch
Hacker News
Top stories via the official Firebase API (hacker-news.firebaseio.com)
arXiv
Recent papers via the arXiv API, filtered to the owner's interest areas
GitHub Trending
Scrape github.com/trending (daily)
RSS feeds
A configurable list of feeds (tech, science, finance); parse with stdlib xml.etree
Each candidate is a dict: {title, url, source, summary, published_at}.
Respectful fetching: declare a User-Agent identifying the bot. For SEC EDGAR (if used), the UA must contain an email-shaped contact or requests get 403.
- Interest-aware ranking (no LLM)
Maintain a keyword list representing the owner's interests (e.g. AI agents, local LLMs, prediction markets, options, FPL, home automation).
Score each candidate: keyword hits in title/summary, recency boost, source diversity cap (max N per source so one source can't dominate).
Dedupe against a seen.json of URLs already dispatched (persist across runs).
- URL verification (no LLM)
HEAD/GET each shortlisted URL with a short timeout (~8s).
Drop dead links before they reach the LLM. A dispatch with a 404 is a failure.
- LLM curation
Send the verified shortlist (numbered, with titles + summaries) to the LLM with a prompt like:
"You are curating a nightly serendipity dispatch. Pick the 4 most surprising, non-obvious items from the list below. Return ONLY the indices, e.g. 3, 7, 12, 18. One pick each for: a discovery (something new), a deep dive (worth 20 minutes), a build idea (something to make), a markets note (interesting prediction-market or options movement). Skip anything that reads like press-release hype."
The model returns indices only. Map them back to candidates in code.
Retry the LLM call up to 3 times on failure; after that, fall back to the top-ranked deterministic picks so the dispatch still ships.
- Markets note (deterministic)
Query the Kalshi public API (anonymous, read-only) for the biggest price movers in the last 24h.
Render the top 5 movers with before/after prices. No LLM needed, no trading, no orders — read-only market data.
- Wonderings rotation
Keep a wonderings.md file: a running list of open questions/curiosities.
Each night, pick the oldest uncovered wondering for the deep-dive slot, then mark it covered. When the list is exhausted, the LLM generates fresh wonderings from recent discoveries.
Never repeat a covered wondering.
- Output
Two files per night in data/dispatches/YYYY-MM-DD.md:
Dispatch — markdown with the 4 sections, each with title, link, 2–3 sentence why-it-matters, and source attribution.
Teaser — a 2-line push notification text (sent via ntfy, see below).
Also write data/latest.md (symlink or copy of the newest dispatch) and data/teaser.txt for the notification pipeline.
- Notifications
Send the teaser via ntfy (self-hosted or ntfy.sh) when the dispatch is ready.
If the run fails entirely, send a failure notice instead of staying silent.
Never send "ready" if the dispatch wasn't actually written.
- State files (all JSON, atomic writes)
File
Purpose
data/seen.json
URLs already dispatched (dedupe)
data/wonderings.md
Open questions rotation
data/status.json
Last run status: ok flag, source counts, timestamp
data/dispatches/
One markdown file per night, kept indefinitely
Write state atomically (temp file + rename), never in-place rewrites.
- Failure discipline (non-negotiable)
Never mark a candidate seen until it is successfully dispatched.
Every retry loop: attempt counting, a ceiling, a fallback, a notification.
If all LLM calls fail, still write the dispatch from deterministic picks — a thinner dispatch beats a missing one.
Log every run; a silent failure is the worst outcome.
- Dependencies
Python 3 stdlib + requests. That's it.
One LLM API endpoint (any OpenAI-compatible API works — LiteLLM proxy, OpenAI, Groq, local Ollama, etc.).
Cron. A lock file. A directory.
- What it is NOT
Not a news aggregator — no headlines the owner already saw.
Not a trading bot — the markets note is observational only.
Not a chatbot — it produces one artifact per night, no conversation.
Status as of 2026-10-17. Diagrams are Mermaid and render on GitHub. Facts that drift (agent list, schedules) are in CLAUDE.md and manifests/; this document explains how it fits together and why.
- What Hermes is
About 15 small, cron-driven Python agents on one free Oracle Cloud ARM VM, plus a phone-first dashboard, a private push-notification server, and two paper forecasting/trading labs. It watches email, school, flyers, fantasy football, prediction markets and options, and tells the owner only what matters. Everything is $0: free VM, free-tier model APIs, local Ollama.
flowchart LR
subgraph World["Outside world (read-only)"]
MAIL[iCloud mail<br/>IMAP read-only]
WEB[School site, flyers,<br/>FPL API, ESPN, FRED]
MKT[Kalshi + CBOE + Yahoo<br/>anonymous market data]
end
subgraph VM["One OCI Always Free ARM VM"]
CRON[cron<br/>generated from manifests]
AG[~15 agents + 2 labs]
LLM[LiteLLM proxy<br/>free lanes + local Ollama]
DASH[Dashboard<br/>auth_server + static PWA]
NTFY[Self-hosted ntfy]
STORE[(SQLite files<br/>per agent)]
end
OWNER((Owner's phone))
MAIL --> AG
WEB --> AG
MKT --> AG
CRON --> AG
AG <--> LLM
AG --> STORE
AG --> NTFY --> OWNER
STORE --> DASH
DASH -- "Cloudflare tunnel" --> OWNER
Why one VM: it is free, simple to back up, and a failure has one place to look. Nothing needs more.
The two paper labs, run as Hermes-framework agents
memory/, chief/
Searchable memory and the ranked daily "needs you" brief
hermes/proposals.py
Scans skill and canary proposals; approve opens a proposal/* branch + draft PR
docs/
This tour
cd/auto-pull.sh
Deploy: fast-forward only, CI-green commits only
- How a job runs, and how code gets deployed
flowchart TD
M["agent.toml / manifests/*.toml<br/>schedule in PT"] --> G[tools/gen_crontab.py<br/>explicit UTC, DST-aware]
G --> S[tools/sync_crontab.py<br/>managed block only]
S --> C[cron line]
C --> R["runner/hermes_run.py<br/>flock + manifest timeout"]
R --> A[agent script]
R --> DB[(runner/data/runs.db<br/>one row per run)]
A --> OUT[agent data/ + card.hermes.json]
OUT --> B[dashboard/build_data.py]
B --> J[public/data/*.json]
J --> PWA[static PWA]
DB --> CAN[canary liveness checks]
flowchart LR
DEV[Edit + test locally] --> PUSH[git push master]
PUSH --> CI["GitHub CI<br/>compile, unittest, bash -n,<br/>gitleaks, cron goldens,<br/>paper-only / IMAP / canary greps"]
CI -- green --> PULL["VM cron every 15 min<br/>cd/auto-pull.sh (ff-only)"]
PULL --> SYNC[sync crontab] --> RESTART[restart hermes-dash if dashboard/ changed]
CI -- red --> HOLD[hold + alert once]
Decisions and why
Schedules live in manifests, crontab is generated. Ubuntu cron ignores CRON_TZ, and a hand-edited crontab drifts. So PT is authored, UTC is generated, and golden files for both PDT and PST are checked in CI, so a DST change needs no human.
Pull-based deploy, CI-green only, fast-forward only. The VM never accepts a red or conflicting commit and never needs inbound access. The cost is up to ~15 minutes of deploy latency, which nothing here cares about.
stdlib-only agents. No dependency drift on a VM nobody wants to babysit.
- Models: lanes, not providers
flowchart LR
AGENT[Agent] -- "task label + lane name" --> PROXY[LiteLLM proxy<br/>stateless]
PROXY --> D["agent-default<br/>Groq 120b > Groq 20b > NVIDIA > Pollinations > Ollama"]
PROXY --> RS["agent-reason<br/>Groq 120b > Groq 20b > NVIDIA"]
PROXY --> V["agent-vision<br/>Gemini Flash only"]
PROXY --> CH["agent-cheap<br/>local Ollama only"]
PROXY --> USAGE[(usage.db<br/>every call logged)]
MAILAG[email-brief / home-chief] -- "local only" --> OLL[Ollama on the VM]
Agents name a lane, never a model. Providers can be swapped or reordered in one file and failover is automatic (3 strikes then 5-minute cooldown). Why: free tiers rate-limit and disappear; the stack must survive that.
Vision never fails over to text models, so an image can never be silently mis-read.
Family email never leaves the VM. Email agents use local Ollama by policy and a CI grep enforces read-only IMAP.
Task labels are mandatory so usage.db shows which agent burns which quota.
- Notifications and the dashboard
One sender:hermes.notify.send(title, msg, channel="alerts"|"family") to the self-hosted ntfy server. A test forbids posting to public ntfy.sh (only the heartbeat ping is exempt). Why: topics are secrets; one choke point makes redaction and rate limits enforceable.
Dashboard:auth_server.py on 127.0.0.1:8899 behind a Cloudflare tunnel, ntfy-code login, two roles. The family role can reach only /family/ and the flyer upload. Pages: Control Room /, /classic/, /family/, /chat/, /fpl/, /options/ (with a Lab learning panel: lessons and memory index), /proposals/ (the approval inbox, full role only).
Contract-first data: agents emit versioned JSON (hermes.card/1, hermes.fleet/1, hermes.equity/1, ...), validated by hermes/cards.py. Why: the UI can change without touching agents, and a bad card fails validation instead of breaking the page.
- The forecasting labs (the interesting part)
The Hermes framework underneath
The labs run on Hermes Agent by Nous Research (open source; the VM tracks upstream, v0.21.5). It supplies what we would otherwise hand-build: skills (SKILL.md playbooks), an agent loop with memory, MCP tool support and a job scheduler. Our cron stays the clock; the framework does the thinking.
Each is a skill-driven agent job with a matching -fallback job (--no-agent deterministic script)
options-lab-debate-{bull,bear,synth} + -finalize
options-lab
Blind bull and bear sessions, a synthesis, then a deterministic MC-checked finalize
MCP server options-lab
options-lab
Our stdlib tool server (market, risk, fills, ledger) registered with hermes mcp add
Dashboard chat bridge
dashboard
/api/hermes/chat hands prompts to the framework
recall tool (inside the options-lab MCP server)
options-lab
Past similar decisions with outcomes, plus current lessons; lab sources only; used by the decide skill
Everything else (fpl, email-brief, home-chief, school, flyer-events, after-dark, lane-eval, ledger, canary, hygiene, vm-backup, memory, chief) is plain cron plus our Python, not framework-hosted. The framework's own messaging gateway service is not running; notifications go through our ntfy path instead.
How much of the framework we use, and why not more. In use: scheduler (as a trigger), skills, MCP, the LiteLLM provider, and now memory-backed recall. Deliberately off: delegation (more calls than an ~8k tokens/min free tier can pay for), web/browser/ terminal/file/code toolsets (no powerful tools next to the ledger), and the messaging gateway (ntfy does the job). Every tool schema costs tokens on every call, so the lab agents run with 10 tools, and the free-tier token budget, not missing features, is the real limit.
Both labs are paper only. There is no order code, no broker login; a CI grep and a canary invariant enforce it.
flowchart TD
CRONT["cron = the clock only"] --> FW[fw.py wrapper]
FW --> AGJ["Hermes framework AGENT job<br/>skill + memory + MCP tools"]
FW -. "no ledger evidence / lane down" .-> FB["deterministic --no-agent<br/>fallback job"]
AGJ --> MCP["stdlib MCP server<br/>market, risk, fills, ledger"]
FB --> MCP
MCP --> L[("append-only ledger<br/>integer cents / kalshi.db")]
L --> SC[Brier scoring vs market baseline]
SC --> REV[weekly rule-based review<br/>tunes policy inside fixed envelopes]
REV --> MCP
L --> INV["loop-closure invariant<br/>re-checked independently by canary"]
Lab
Idea
Success test
kalshi-hermes
LLM estimates event probabilities from a numbered evidence packet (source + as-of time), blind to the market price
Beats the market's own price by Brier score, n >= 30 with a bootstrap CI
options-lab
$5k x 6 sleeves = $30k virtual; scout, debate (blind bull and bear), decide, manage, learn
Staged rollout (baseline alone first); phase gates before it may be called "live"
Decisions and why
Agents propose, deterministic code disposes. Models never touch money logic: sizing, risk limits, fills and ledger writes are plain code behind MCP tools. A confused model can waste tokens but cannot corrupt the books.
Always a fallback job. If the agent leaves no ledger evidence, a deterministic run still happens, so the clock never stalls.
The ledger is append-only with a loop-closure invariant (every decision ends in a recorded outcome). Calibration is only trustworthy if nothing is rewritten.
"Beat the baseline or it doesn't count." Every forecast stores model P, market P and later resolution at decision time. Fair tests only: markets whose outcome is already effectively known (games underway, prices near 0 or 1) are skipped.
No self-congratulation. Docs may not say "live", "working" or "learning" until the phase gates are met.
Policy tuning is bounded: weekly, rule-based, inside fixed envelopes, versioned.
- Safety rails (why it can run unattended)
flowchart LR
subgraph Prevent["Prevent (CI)"]
P1[gitleaks secret scan]
P2[grep: no order code]
P3[grep: IMAP no STORE/DELETE/MOVE]
P4[manifest + cron goldens PDT/PST]
P5[test: no public ntfy.sh]
end
subgraph Detect["Detect (canary)"]
D1[run liveness vs manifest]
D2[loop-closure + cash reconciliation]
D3[paper-only / IMAP re-check]
end
subgraph Respond["Respond"]
R1[ntfy alert]
R2[" agent only for<br/>paper-only, IMAP, outbound-mail rules"]
R3[propose fix in a throwaway worktree<br/>never auto-apply]
end
Prevent --> Detect --> Respond
The canary is propose-only. Its fixes land on a canary/* branch built in a throwaway git worktree, never the live checkout, and it cannot edit its own policy files. It can an agent only for a short list of safety rules. Why: an autonomous fixer with write access to production is a bigger risk than the bugs it fixes.
Backups: nightly encrypted vm-backup, pushed to OCI Object Storage via instance principal, create-only (it cannot delete its own backups). RESTORE.md is the runbook; the options lab also does a weekly restore drill.
Secrets (.env, ntfy topics, tokens, Kalshi key, FPL team id) are never committed, printed or logged; crontabs are redacted when shown.
The VM checkout is never reset or cleaned. It has intentional drift; ff-only pulls tolerate that.
- Memory, Chief of Staff and the PR-writing canary
flowchart LR
SRC["Cards, briefs, ledger rationales,<br/>run logs, derived email briefs"] --> IDX["memory/index.py<br/>nightly 2:30am PT"]
IDX --> RED[redact secrets, emails, phones]
RED --> MEM[("memory.db<br/>SQLite FTS5 + optional embeddings")]
MEM --> API["hermes.memory<br/>add / seen / mark_pushed / search"]
API --> CHIEF["chief/ ranks 'needs you today'<br/>7:00am + 6:00pm PT"]
CARDS[fleet + agent cards] --> CHIEF
CHIEF -- "max 3/run, 6/day, quiet hours" --> NT[ntfy push + Control Room card]
API --> CHAT["chat retrieval (planned)<br/>private sources excluded"]
INC[Agent failure] --> CAN["canary propose_fix<br/>CANARY_PR_MODE off / local / draft"]
CAN --> BUN["patch + regression test + PR.md<br/>reviewed by a human"]
Piece
Value
Key guardrail
Memory
One searchable place for everything agents saw or produced; dedupes alerts, gives agents their own history, base for chat
Redaction before storage; private sources excluded from cloud chat; local-only embeddings
Chief of Staff
Turns ~15 agents' output into 1 to 3 pushes that matter; the antidote to notification fatigue
Hard caps, quiet hours, private text summarised only by local Ollama
PR-writing canary
Failures arrive as a ready-to-review fix plus regression test instead of a bare alert
Default off; never merges; draft mode only pushes a draft PR when the owner enables it
Learning loop and the Proposals inbox
flowchart TD
LEDGER[("options ledger<br/>closed trades + outcomes")] --> IDXL["memory index_lab<br/>lab_outcome / lab_debate / lab_review"]
IDXL --> MEM[(memory.db)]
MEM --> RECALL["recall tool (kind=recall)<br/>5 compact hits"]
RECALL --> DECIDE[decide agent]
LEDGER --> REVIEW["weekly rule-based review<br/>(no LLM)"]
REVIEW --> LES["lessons.md / lessons.json<br/>max 10 bullets, 1500 chars, 8-week expiry"]
LES --> RECALL2["recall tool (kind=lessons)"] --> DECIDE
LES --> PROP["tools/lab_skill_proposals.py<br/>one LLM call, proposes a SKILL.md edit"]
CAN[canary propose_fix<br/>CANARY_PR_MODE=local bundle] --> INBOX
PROP --> INBOX["Dashboard Proposals inbox<br/>summary, evidence, diff, tests badge"]
INBOX -- "Dismiss" --> DONE[status only]
INBOX -- "Approve (full role only)" --> BR["throwaway worktree off master<br/>push proposal/<id> + draft PR"]
BR --> YOU[["You review and merge on GitHub"]]
YOU --> CI[CI] --> DEPLOY[VM auto-pull + install.py refreshes skills]
Lessons are statistics, not opinions. They come from ledger numbers with minimum evidence, so noise is not promoted; they expire so the agents do not anchor on stale conclusions.
Nothing self-modifies. A proposal is a file until a human merges a PR. The approve button can only create a proposal/* branch from master (never force, never master), restricted to the proposal's own files; the framework installer ignores the proposals folder.
Proposals and approvals are visible only to the full role. Skill proposals are generated outside options-lab/ because the canary forbids model calls inside lab code.