I graded my own ML option forecasts. Here's the Brier score.

Helium's published prob_itm forecasts for AAPL options contracts expiring June 2026 were graded, yielding a mean Brier loss of 0.3846 on two contracts, worse than a coin flip. The developer also released a free 212×37 news-outlet framing corpus on Hugging Face.

In May I promised — out loud, on the internet, where people can screenshot you — that I'd grade Helium's published prob itm forecasts when the June 2026 AAPL contracts expired. June 26 came. The contracts died. So I graded them. This is not a victory lap. Mean Brier loss on n=2 is 0.3846 . A coin flip that always says 0.5 scores 0.25 when the world resolves to 0 or 1. We did worse than a coin flip. That is information. | Contract | Helium prob itm | Market-ish implied | |---|---|---| | AAPL $310C 2026-06-26 | 0.42 | ~0.50 | | AAPL $295P 2026-06-26 | 0.23 | ~0.24 | AAPL NASDAQ close on 2026-06-26: $283.78 git clone https://github.com/connerlambden/helium-mcp-cookbook cd helium-mcp-cookbook python calibration/grade june expiry.py Frozen inputs live in calibration/june 2026 forecasts.json https://github.com/connerlambden/helium-mcp-cookbook/blob/main/calibration/june 2026 forecasts.json . Because the alternative is the usual ML options blog post: a backtest, a vibe, a screenshot of a green P&L. Publishing prob itm with a fixed expiry is a tiny discipline. Grading it after the fact is the second half of that discipline. If you log your own forecasts with recipe 02 https://github.com/connerlambden/helium-mcp-cookbook/blob/main/recipes/02 options calibration tracker.py , you can run the same scorer when your contracts expire. I'll keep adding rows. — Also adjacent, if you're here for news rather than options: we published a free 212×37 news-outlet framing corpus https://huggingface.co/datasets/HeliumTrades/helium-news-bias-corpus with an explorer https://connerlambden.github.io/helium-news-explorer/ . Different animal. Same "put numbers where people can grade them" instinct.