{"slug": "i-graded-my-own-ml-option-forecasts-here-s-the-brier-score", "title": "I graded my own ML option forecasts. Here's the Brier score.", "summary": "Helium's published prob_itm forecasts for AAPL options contracts expiring June 2026 were graded, yielding a mean Brier loss of 0.3846 on two contracts, worse than a coin flip. The developer also released a free 212×37 news-outlet framing corpus on Hugging Face.", "body_md": "In May I promised — out loud, on the internet, where people can screenshot you — that I'd grade Helium's published `prob_itm`\n\nforecasts when the June 2026 AAPL contracts expired.\n\nJune 26 came. The contracts died. So I graded them.\n\nThis is not a victory lap. Mean Brier loss on **n=2** is **0.3846**. A coin flip that always says 0.5 scores 0.25 when the world resolves to 0 or 1. We did worse than a coin flip. That is information.\n\n| Contract | Helium `prob_itm`\n|\nMarket-ish implied |\n|---|---|---|\n| AAPL $310C 2026-06-26 | 0.42 | ~0.50 |\n| AAPL $295P 2026-06-26 | 0.23 | ~0.24 |\n\n**AAPL NASDAQ close on 2026-06-26: $283.78**\n\n```\ngit clone https://github.com/connerlambden/helium-mcp-cookbook\ncd helium-mcp-cookbook\npython calibration/grade_june_expiry.py\n```\n\nFrozen inputs live in [ calibration/june_2026_forecasts.json](https://github.com/connerlambden/helium-mcp-cookbook/blob/main/calibration/june_2026_forecasts.json).\n\nBecause the alternative is the usual ML options blog post: a backtest, a vibe, a screenshot of a green P&L. Publishing `prob_itm`\n\nwith a fixed expiry is a tiny discipline. Grading it after the fact is the second half of that discipline.\n\nIf you log your own forecasts with [recipe 02](https://github.com/connerlambden/helium-mcp-cookbook/blob/main/recipes/02_options_calibration_tracker.py), you can run the same scorer when your contracts expire. I'll keep adding rows.\n\n—\n\nAlso adjacent, if you're here for news rather than options: we published a free [212×37 news-outlet framing corpus](https://huggingface.co/datasets/HeliumTrades/helium-news-bias-corpus) with an [explorer](https://connerlambden.github.io/helium-news-explorer/). Different animal. Same \"put numbers where people can grade them\" instinct.", "url": "https://wpnews.pro/news/i-graded-my-own-ml-option-forecasts-here-s-the-brier-score", "canonical_source": "https://dev.to/connerlambden/i-graded-my-own-ml-option-forecasts-heres-the-brier-score-20fp", "published_at": "2026-07-14 22:24:55+00:00", "updated_at": "2026-07-14 22:57:11.992235+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools"], "entities": ["Helium", "AAPL", "NASDAQ", "Conner Lambden", "Hugging Face"], "alternates": {"html": "https://wpnews.pro/news/i-graded-my-own-ml-option-forecasts-here-s-the-brier-score", "markdown": "https://wpnews.pro/news/i-graded-my-own-ml-option-forecasts-here-s-the-brier-score.md", "text": "https://wpnews.pro/news/i-graded-my-own-ml-option-forecasts-here-s-the-brier-score.txt", "jsonld": "https://wpnews.pro/news/i-graded-my-own-ml-option-forecasts-here-s-the-brier-score.jsonld"}}