Walsh is a multi-agent stock research system that routes each ticker through four specialized agents in parallel — Fundamental, Technical, Sentiment, and Macro — aggregates their theses via a confidence-weighted Portfolio Manager, then gates the final call through a Rule-Based Risk Manager that can veto or downgrade before any trade is executed.
Live demo: walsh-demo.vercel.app · Demo GIF
Backtest run: AAPL · MSFT · NVDA
, Jan 2023 – Dec 2023, $100k initial capital, rule-based signal agents, strict no-lookahead enforcement.
| Metric | Walsh Pipeline | Buy & Hold (equal-weight) |
|---|---|---|
| Total Return | +75.52% | |
| +119.75% | ||
| Sharpe Ratio | 2.36 | |
| 2.86 | ||
| Max Drawdown | −13.50% | |
| −13.22% | ||
| Win Rate | 76.2% | |
| — | ||
| # Trades | 21 | |
| — | ||
| Final Value | $175,518 | |
| $219,749 |
Walsh underperformed buy-and-hold on this bull-market period. In Oct 2023, the pipeline exited all positions when momentum signals broke down and held cash for the entire month, missing a subsequent NVDA recovery rally. This is the intended behavior of the risk gate — it correctly identified a high-uncertainty period — but reveals the cost of a conservative threshold in trending markets.
Per-agent accuracy / confidence calibration: pending. The eval/agent_attribution.py
module is ready; results require ≥50 completed pipeline runs with follow-up price data to produce meaningful calibration curves.
Cost and latency per run: pending. The eval/cost_latency.py
PipelineTracker
is instrumented; numbers will be populated once the LLM-backed agents replace the current rule-based stubs.
Adversarial robustness: 18 test cases across three agent types; 5 flagged for overconfidence (see Known limitations).
┌──────────────────────────┐
ticker ──────► │ Orchestrator │
│ (asyncio.gather) │
└──┬──┬──┬──┬──────────────┘
│ │ │ │ (parallel)
┌─────────────┘ │ │ └─────────────┐
▼ ▼ ▼ ▼
┌────────────┐ ┌──────────┐ ┌──────────┐ ┌───────────┐
│Fundamental │ │Technical │ │Sentiment │ │ Macro │
│ Analyst │ │ Analyst │ │ Analyst │ │ Analyst │
│ │ │ │ │ │ │ │
│PE, growth, │ │RSI, MACD,│ │news, │ │GDP, rates,│
│margins, │ │golden │ │social, │ │inflation, │
│D/E ratio │ │cross │ │insider │ │sector │
└─────┬──────┘ └────┬─────┘ └────┬─────┘ └─────┬─────┘
└──────────────┴──────────────┴──────────────┘
│
▼ AgentThesis × 4
┌──────────────────────┐
│ Portfolio Manager │
│ confidence-weighted │
│ signal synthesis │
└──────────┬───────────┘
│ PortfolioDecision
▼
┌──────────────────────┐
│ Risk Manager │
│ veto / downgrade │
│ • conf < 45% → veto │
│ • spread ≥ 4 → veto │
│ • conf < 65% → │
│ downgrade STRONG │
└──────────┬───────────┘
│ FinalDecision
▼
approved / vetoed
Fundamental Analyst fetches valuation metrics (PE ratio, revenue growth, profit margin, debt-to-equity) and scores them against thresholds. A PE below 15 adds a bullish vote; above 30 adds a bearish one. Revenue growth above 15% or profit margins above 20% each add a bullish vote. The raw score maps to a STRONG_BUY / BUY / HOLD / SELL / STRONG_SELL
signal with confidence proportional to conviction strength.
Technical Analyst evaluates SMA crossovers (20d and 50d), MACD histogram direction, and optionally a golden-cross flag. Each indicator votes +1 or −1, with a dead band to suppress spurious signals on flat prices. In the backtest harness, the rule-based TechnicalSignalAgent
uses realized momentum over 20d and 50d rolling windows against yfinance price data.
Sentiment Analyst aggregates a news sentiment score (−1 to +1) and a social sentiment score, averages them, then applies an insider-buying bonus. Outputs a directional signal and a confidence value dampened for near-neutral sentiment.
Macro Analyst checks GDP growth, the Fed Funds rate, inflation, and a sector outlook tag. Favorable macro conditions (growth > 2.5%, rate < 4%, inflation < 3%, bullish sector) each add a bullish vote; unfavorable conditions subtract. The macro signal acts as a regime filter — a strong bearish macro can offset bullish technical or sentiment readings.
Portfolio Manager synthesizes the four theses using a confidence-weighted average of their numeric signal scores. Consensus penalty: confidence is scaled down by the spread between the highest and lowest agent signals (conf × max(0.5, 1 − spread × 0.1)
). A tight consensus stays near full confidence; a 4-point spread (STRONG_BUY vs STRONG_SELL) halves it.
Risk Manager is the only decision-maker. Two hard veto rules: (1) portfolio confidence below 45% → forced HOLD regardless of signal, (2) agent signal spread ≥ 4 → veto as extreme disagreement. A soft rule downgrades STRONG_BUY → BUY or STRONG_SELL → SELL when confidence is 45–65%. If no rule fires, the decision is passed through with "No risk flags."
Backtest underperformance (Jan–Dec 2023): Walsh's rule-based backtest returned +75.5% vs +119.8% for equal-weight passive buy-and-hold on AAPL/MSFT/NVDA — underperforming by ~44 pp. The primary cause is the volatility damper (vol_cap=40%
in backtest/engine.py
) which zeroed out NVDA allocation for most of the year (NVDA realized vol was ~70-90%), combined with the MeanReversionSignalAgent
issuing SELL signals on stocks hitting 52-week highs throughout the bull run. Signal weighting and the vol cap need tuning before this strategy is suitable for live deployment. More broadly, contrarian strategies structurally underperform during sustained trending regimes — a more robust Walsh would detect trend vs. mean-reverting market conditions and scale the MeanReversionSignalAgent
's weight accordingly, rather than holding it constant across all regimes.
Backtest methodology clarification: The backtest engine uses deterministic rule-based agents (SMA crossover, momentum, mean-reversion) — not the LLM-powered agents shown in the demo dashboard. The numbers above reflect the algorithmic signal engine only.
These are documented findings from eval/adversarial.py
(18 test cases, 5 flagged):
TECH_003 — Zigzag prices, overconfidence (confidence=0.80
, max expected 0.65
) SMA signals from oscillating prices cancel imperfectly; the agent still returns 0.80 because the vote mean slightly favors one side. A price series that crosses SMA20/SMA50 repeatedly should produce near-zero confidence, but the dead band (0.1%) is too tight to suppress all noise.
TECH_004 — Crash after uptrend, stale bullish signal (confidence=0.80
, max expected 0.65
) When prices run up then crash sharply, SMA20 and SMA50 lag by construction. The agent may still read "price above SMA" for several bars after the crash top, giving a confident BUY into a falling market. The 20d momentum partially corrects this but cannot always overcome two SMA votes.
TECH_006 — Single-day 50% jump after flat history, spurious signal (confidence=0.80
, max expected 0.75
) A one-day spike inflates the 20d momentum calculation and pushes price above both SMAs immediately, generating a STRONG_BUY at 0.80 confidence despite no trend substance. Input validation (e.g., cap single-bar moves) is not currently applied.
MOM_005 — Negative prices (data corruption), overconfident (confidence=0.85
, max expected 0.65
)
MomentumSignalAgent
does not validate that prices are positive before computing log-returns. Negative prices produce a defined (if nonsensical) momentum value and a confident directional signal. A price sanity check is the fix.
REV_006 — Zigzag mid-range, overconfident (confidence=0.76
, max expected 0.50
)
MeanReversionSignalAgent
sees the price oscillating exactly in the middle of its 52-week range (no reversion signal), yet outputs 0.76 confidence. The agent currently gives a non-zero confidence for mid-range prices when it should default to "no clear reversion target."
Structural limitations not yet tested:
- The Risk Manager's 45% confidence threshold is a single hard cutoff; it doesn't account for drawdown regime or portfolio concentration.
- Agent signals are computed independently with no cross-agent information sharing — a macro veto doesn't lower a technical agent's confidence.
- The current backtest uses rule-based stub agents; LLM-backed agents (which can hallucinate or overstate certainty) will likely surface additional failure modes.
- No live market data integration; the orchestrator points to a mock API (
api.finance.example.com
).
Live dashboard at demo-cyan-delta.vercel.app
The dashboard shows 5 pre-generated ticker runs selectable via dropdown, each illustrating a distinct pipeline outcome:
| Ticker | Story |
|---|---|
| AAPL | |
| All four agents agree → BUY, approved | |
| MSFT | |
| Extreme disagreement (Fundamental=SELL vs Technical=STRONG_BUY) → vetoed | |
| NVDA | |
| All four agents strongly agree → STRONG_BUY, approved | |
| TSLA | |
| Agent split: Fundamental=BUY vs Technical=SELL (180° opposition) → vetoed, no position | |
| META | |
| Agents agree (STRONG_BUY consensus) but Risk Manager vetoes due to 76% annualized volatility exceeding the 40% risk cap |
Terminal reproduction (no GIF yet — recording pending):
cd demo
python3 gen_demo.py # outputs the same JSON as demo/data/MSFT.json
To record the GIF once the LLM-backed pipeline is wired:
pip install vhs
vhs scripts/record_pipeline.tape # not yet authored
- Python 3.11+
uv
orpip
- An Anthropic API key (for the LLM-backed agents; not required for the backtest harness)
pip install -e ".[dev]" # installs yfinance, pydantic, pytest
pip install -r requirements.txt # installs httpx
ANTHROPIC_API_KEY=sk-ant-... # required for LLM-backed agents
The backtest engine and adversarial eval do not call the Anthropic API.
python -m backtest.engine --tickers AAPL,MSFT,NVDA --start 2023-01-01 --end 2024-01-01
python -m eval.adversarial
python -m eval.agent_attribution
python -m eval.cost_latency
python -m orchestrator.main AAPL # outputs FinalDecision JSON to stdout
pytest # all tests
pytest tests/test_backtest.py -v
pytest tests/test_eval.py -v
cd demo
npm install
npm run dev # http://localhost:3000