{"slug": "prediction-log-2-first-verdicts-two-experiments-promoted-two-new-bots-flopped", "title": "Prediction Log #2 — First Verdicts: Two Experiments Promoted, Two New Bots Flopped", "summary": "Two experimental AI trading bots earned promotion after four weeks of testing, while two new bots flopped with 41.5% and 30.8% accuracy, according to a developer's prediction log published on July 4, 2026. Claude's declare-first prompt achieved a 64.0% accuracy and +43.2% annualized rate versus its main's 59.3% and +27.5%, winning 60% of head-to-head disagreements, leading to its promotion. Gemma's indicator compression experiment posted 55.1% accuracy but a −1.7% annualized rate, losing to main's 52.1% and +7.9% on the annualized metric, though it won 64% of head-to-head disagreements.", "body_md": "[Part 1](/blog/prediction-log-01-baseline) set the baseline. After a month of running six AI bots, the free Gemma on my laptop sat in first place — but it was statistically tied with the simple rule-based bots, so it was too early to claim AI had any edge. As promised, I then started changing the bots one piece at a time and measuring what happens. Today is the first verdict day.\n\nThe headline: **both experiments earned promotion**. And the two new bots I added for fun during the same stretch — a trending bot that hunts the day’s hottest stocks, and a chart bot that draws lines on charts and cites them as evidence — **flopped at 41.5% and 30.8% accuracy**. Here’s what worked and what didn’t, in that order.\n\n## First, the score changed\n\nPart 1 used an “adjusted average” as its main metric. Since then, LDBD’s headline score itself has been redesigned as an **annualized return rate** — the old score rewarded betting often on volatile assets more than it rewarded skill. The full story is in [dev log #3](/blog/growth-features-and-rescoring). From this entry on, every table uses the annualized rate. Read it roughly as “if you had followed this bot’s calls, what would your annualized return have been,” adjusted for sample size (0 is neutral; the fewer the samples, the harder the value gets pulled toward 0).\n\n## The setup — a promotion system\n\nThe setup is simple. The six baseline bots stay untouched as a reference. For each model, the current best configuration runs as **main**, and an experimental bot with exactly one change layered on top runs alongside it as **exp**. Once enough data has accumulated, if exp looks better than main, the change gets absorbed into main — I’ll call that a **promotion**. If it loses, the change is dropped and the result stays on the record. This round had two changes:\n\n**Gemma experiment — indicator compression**: instead of a raw 10-day price sequence, the prompt gets a single computed line — moving averages (SMA), an overbought/oversold gauge (RSI), a trend gauge (MACD), volatility, 52-week position, and 5-day return. The hypothesis: a small model can’t pull patterns out of raw numbers, so let Python do the math and the model do the judging.**Claude experiment — declare before you analyze**: before starting any analysis, the bot must state (a) which price basis it uses, (b) an estimated base rate for each asset (the share of weeks it typically closes up), and (c) what data is missing and what it’s assuming. The hypothesis: this blocks the pattern of picking a conclusion first and back-filling the reasons.\n\n## Four weeks of results\n\nResolved predictions from 2026-06-05 through 2026-07-04.\n\n| Bot | Resolved | Accuracy | Annualized rate |\n|---|---|---|---|\n| Gemma main (existing config) | 94 | 52.1% | +7.9% |\nGemma exp (indicator compression) | 69 | 55.1% | −1.7% |\n| Claude main (existing config) | 81 | 59.3% | +27.5% |\nClaude exp (declare-first) | 86 | 64.0% | +43.2% |\n\nFiltering down to the cases where both bots called the same asset on the same day gives a head-to-head view. When the two disagreed, whoever turned out right is the most direct measure of what the change actually did.\n\n| Matchup | Matched | Agreement | Exp win rate on disagreements |\n|---|---|---|---|\n| Gemma main vs exp | 68 | 79% | 64% (9–5) |\n| Claude main vs exp | 80 | 81% | 60% (9–6) |\n\n## Verdict 1 — Claude’s declare-first prompt: a clear promotion\n\nClaude was the easy call. Accuracy up 4.7 percentage points, annualized rate +43.2% versus +27.5%, head-to-head 60% — exp was ahead on every metric. The telling stretch was June 30 through July 1, when the market rolled over: main kept insisting on up and missed again and again, while exp flipped to down and got it right. A simple prompt change — declare your assumptions before you analyze — seems to have made the bot less stubborn.\n\nSo it’s promoted. The declare block is now part of Claude main’s prompt, and the next experiment forks from this new main.\n\n## Verdict 2 — Gemma’s indicator compression: promoted, with an asterisk\n\nGemma’s signals pointed in different directions. Accuracy (+3.0 points) and head-to-head (64%) went to exp, and **Bitcoin accuracy jumped from 52% to 80%** — the same asset that sat at 38.8% accuracy in Part 1 and lost even to the always-bull baseline. With overbought/oversold gauges like RSI in the prompt, it started catching rebounds after sharp drops.\n\nBut the annualized rate flips the story: main +7.9%, exp −1.7%. The culprit is KODEX 200, the Korean KOSPI 200 ETF on the watchlist. Exp got only 4 of its 14 calls right there (main: 7 of 18), and the misses landed exactly in the stretch of June when the Korean market swung hard, so the losses were heavy. Indicator compression worked on US assets and Bitcoin, but backfired on fast-moving Korean equities.\n\nThe promotion criteria, fixed before the round started, required two of three: a score advantage, a head-to-head of at least 55%, and improvement on a known weak asset. Exp missed the score but cleared the other two — 64% head-to-head and the improvement on Bitcoin, Part 1’s biggest weak spot — so it’s **promoted**. In other words, this isn’t a promotion won on score; it’s a conditional promotion built on head-to-head results and a weak-asset fix. The rate deficit and the KODEX regression go on the record. One thing did factor into the judgment: the exp bot sat dead for eight days in mid-June because of an ops mistake (the file the scheduler ran existed only on another git branch, so the bot silently died every night). The two bots’ samples don’t cover identical windows, which contaminates any whole-period rate comparison — and that’s exactly why I judged on the same-day, same-asset head-to-head instead.\n\nThe KODEX weakness becomes a future experiment: Yahoo Finance carries almost no news for Korean assets, so wiring in Korean media news is next in the queue.\n\n## Meanwhile, I built two new bots\n\nWhile the experiments ran their four weeks, I had time on my hands (and needed something to catch users’ eyes), so I built two bots of a very different character. Spoiler: both are getting crushed — but each failed in an instructive way, so here they are, as is.\n\n### The trending bot — finds the day’s hottest tickers on its own\n\nThe existing bots watch a fixed list of five assets. The trending bot scans social-media mentions (Reddit), news volume, and the IPO calendar every day and **picks the day’s most talked-about stocks by itself**. There are two guardrails: market cap of at least $10 billion (no junk tickers), and it can’t add a new symbol to the database on its own — new finds go into a queue that a human (me) approves.\n\nIts best moment: it flagged [SpaceX (SPCX)](/asset/SPCX), fresh off its listing, two days in a row and put it in the approval queue. Thanks to that, hot names like SpaceX, Rocket Lab, and Nebius are now on LDBD for anyone to predict. The loop worked end to end, once: the bot scouts, a human opens the door, the community joins in.\n\nThe scoreboard is brutal: 41 resolved, 41.5% accuracy, annualized rate −79.9%. The cause is plain — 68% of its calls were up, while only 49% of those stocks actually rose. It became **a bot that buys hype because it’s hype**. As it happens, there’s [empirical research](https://www.sciencedirect.com/science/article/pii/S1057521924006537) showing that retail “attention buying” loses money on average — positions opened when attention on Reddit’s stock boards peaked averaged −8.5%. Buying at peak buzz is precisely the worst timing. So the next improvement is teaching the bot to tell “buzz still building” from “buzz already peaked.”\n\n### The chart bot — draws lines on the chart and cites them\n\nLDBD recently gained candlestick charts and a way to attach drawn lines to a prediction as evidence ([dev log #3](/blog/growth-features-and-rescoring)). The chart bot is the first AI to use it — it computes support, resistance, and trendlines mathematically from price and volume data, draws them on the chart, and cites them in its reasoning. Showing chart images to a vision model was ruled out from the start: a recent [benchmark](https://arxiv.org/abs/2604.12659) suggests vision models can read an obvious trend from a candlestick chart but can’t yet handle ordinary market conditions reliably. So every line is computed from the numbers.\n\nThe line drawing itself worked. All 46 predictions shipped with chart annotations, and the lines land cleanly on real highs and lows. The predictions are the problem — **26 resolved, 30.8% accuracy, annualized rate −19.8%**. That’s far below a coin flip: the kind of record that makes you want to bet against it.\n\nDigging into the failures, the problem wasn’t the lines but **how the model reads them**. Give a model only support and resistance and it tells the same story almost every time: “price has come down near support, so it should bounce.” In a falling market, this bot kept catching falling knives. During a stretch when the actual outcome was up less than half the time, over 60% of its calls were up — and while KODEX 200 was dropping nearly 15%, it kept calling for upside on “recovery momentum.”\n\nSo the chart bot is headed for a redesign, not a patch. The redesign makes two core changes. First, the input gets a new hierarchy: **market regime (trending or ranging) at the top, then trend state, overextension and pullback, then volume** — with support and resistance demoted to supporting detail underneath. Second, instead of jumping to a conclusion, the model has to **list bullish evidence and bearish evidence separately, score each side, and only then decide** — a prompt structure that recent stock-prediction research suggests helps against exactly this kind of one-sided storytelling. I’ll keep reporting the redesign’s results in this log.\n\n## Wrapping up\n\n| What | Verdict | Next |\n|---|---|---|\n| Claude: declare-first | ✅ Promoted (won on every metric) | Absorbed into main; next experiment forks from it |\n| Gemma: indicator compression | ✅ Promoted (head-to-head + BTC improvement; lost on rate) | Absorbed into main; KODEX weakness → Korean-news experiment |\n| Trending bot | ❌ 41.5% accuracy — buys the hype | Buzz-peak detection is up next |\n| Chart bot | ❌ 30.8% accuracy — support-bounce bias | Redesign: input hierarchy + evidence scoring |\n\nIf this round has one lesson, it’s this — **what you show a model determines what it says.** Give a bot computed indicators and it starts catching oversold rebounds; make it declare its assumptions and it gets less stubborn. But give one only support lines and it calls for a bounce everywhere; give one only buzz and it buys the top. The wins and the failures were both input-design problems.\n\nEvery bot’s predictions and reasoning are public in real time on the [leaderboard](/leaderboard) and the feed, and the chart bot’s lines are visible on each prediction’s detail page.", "url": "https://wpnews.pro/news/prediction-log-2-first-verdicts-two-experiments-promoted-two-new-bots-flopped", "canonical_source": "https://ldbd.app/blog/prediction-log-02-round2", "published_at": "2026-07-07 00:00:00+00:00", "updated_at": "2026-07-22 23:41:08.998470+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "machine-learning"], "entities": ["Gemma", "Claude", "LDBD"], "alternates": {"html": "https://wpnews.pro/news/prediction-log-2-first-verdicts-two-experiments-promoted-two-new-bots-flopped", "markdown": "https://wpnews.pro/news/prediction-log-2-first-verdicts-two-experiments-promoted-two-new-bots-flopped.md", "text": "https://wpnews.pro/news/prediction-log-2-first-verdicts-two-experiments-promoted-two-new-bots-flopped.txt", "jsonld": "https://wpnews.pro/news/prediction-log-2-first-verdicts-two-experiments-promoted-two-new-bots-flopped.jsonld"}}