{"slug": "clear-hour-i-committed-an-air-quality-forecast-to-git-before-the-day-started-it", "title": "Clear Hour: I Committed an Air-Quality Forecast to Git Before the Day Started, Then Scored It in Public", "summary": "A developer built Clear Hour, an open-source tool that ranks all 24 hours of the next day by air quality in Islamabad/Rawalpindi using TabPFN v2, an open-weight tabular foundation model running locally on a laptop CPU, to correct a free government CAMS forecast against a year of real sensor readings. Forecasts are committed to git before each day begins and scored afterward against actual measurements; on the first scored day (2026-10-11), all three models ranked the hours backwards (Spearman −0.512 for raw CAMS, −0.369 for TabPFN v2, −0.409 for a CAMS+hour-bias baseline), with none selecting a window inside the day's actual cleanest hours of 17:00–19:00 at 20.9 µg/m³.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass](https://dev.to/challenges/hacktoberfest-week1-2026-10-05)* 🌫️\n\n**TL;DR:** Clear Hour ranks tomorrow's 24 hours by how breathable the air will be in Islamabad/Rawalpindi, using **TabPFN v2**, an open-weight tabular foundation model running locally on my laptop CPU, to correct a free government air-quality forecast against a year of real sensor readings. Each day's forecast is **committed to git before that day starts**, then scored against what a sensor actually measured. I scored the first one this morning. All three models, TabPFN included, ranked the hours backwards. I'm publishing that result anyway, because being able to lose in public is the entire reason to pre-register a forecast instead of picking a flattering day afterwards.\n\n\"Touch grass\" is good advice most places. Where I live it comes with an asterisk for about four months a year.\n\nI'm in Islamabad. Every October the smog settles in, and \"go for a walk\" stops being a simple decision. The air at 9am is nothing like the air at 5pm, and nothing tells you which is which until you're already outside with a sore throat. The apps here give you *today's number*: one figure, already stale, that flattens a day with a 4× swing in it into a single color-coded badge.\n\n**Clear Hour ranks all 24 hours of tomorrow instead**, cleanest to dirtiest. It picks the two-hour window worth going outside for, and the one to avoid. And it commits to that answer *before the day happens*.\n\nThat last part is the whole discipline. It would be trivial to run a forecaster retrospectively, on a day that happened to go well, and write a post about how well it did. So the rule I set on day one was: **the forecast for a day gets committed to git before that day begins, and I never touch that file again.** After the day ends, a separate script pulls real measurements and scores it, flattering or not.\n\nThis morning I ran that scoring script on a real day for the first time. I'm leading with the result, because it's the most honest thing I can tell you about this project.\n\n**Live:** [https://abeera81.github.io/clear-hour/](https://abeera81.github.io/clear-hour/)\n\n**Repo:** [https://github.com/Abeera81/clear-hour](https://github.com/Abeera81/clear-hour)\n\nOne static page, no login, no build step. Scroll past the headline and you land on the pre-registered forecast for 2026-10-11, now with what the sensor actually measured drawn over it:\n\n*The black line is reality. Every colored curve sitting above it after midday is a model saying \"dirtier than it actually got.\" The air was genuinely cleanest from 5pm to 11pm; all three models put their cleanest window earlier than that, and the two corrected models never saw the evening drop coming.*\n\nRun it yourself. The default model needs no API key at all:\n\n```\npython -m clearhour                      # today, instant, no key needed\npython -m clearhour --date 2026-10-13    # any of the next few days\npython -m clearhour --model tabpfn       # the real TabPFN v2 correction (free OpenAQ key, ~30-90 min first fit on CPU)\n```\n\nActual day mean at the sensor: **45.4 µg/m³**. The real cleanest two hours: **17:00–19:00, at 20.9 µg/m³**. The real worst: midnight, at **79.2**.\n\n*Every value in the Spearman column is negative. Negative means the ranking ran **opposite** to reality: hours the model called cleaner tended to be dirtier.*\n\n| Model | MAE | Bias | Spearman (24h) | Chosen cleanest 2h | Actual there | \n|---|---|---|---|---|---|\n| B0 raw CAMS | 21.9 | −3.5 | **−0.512** | 05:00–07:00 | 73.7 | \n| M1 TabPFN v2 | 21.9 | +14.6 | **−0.369** | 15:00–17:00 | 29.8 | \n| B2 CAMS+hour bias | 25.3 | +18.8 | **−0.409** | 15:00–17:00 | 29.8 | \n\nNot one of them picked a window inside the day's actual cleanest hours. Raw CAMS did worst in the way that matters most to a human: it sent you outside at 5am, into 73.7 µg/m³, the **second-dirtiest two-hour window of the entire day**. TabPFN and B2 at least picked an hour below the day's mean, but both missed the real window by two hours and badly underestimated how clean the evening would get.\n\nSo: **on the single day this project has been scored against reality, it ranked the hours backwards.** That's a true statement about one day, and it's in the same post as the project rather than in a follow-up nobody clicks.\n\nWhat it is *not* is a verdict on TabPFN. One day is one sample. A good day tomorrow wouldn't prove the model works either. The 30-day held-out evaluation below is where the actual evidence lives, and there TabPFN earns its place.\n\nThree ingredients, all free, all open:\n\n| Piece | Source | What it contributes | \n|---|---|---|\n| Air-quality forecast | **Open-Meteo** (CAMS global model), CC BY 4.0 | hourly PM2.5 + 5 other pollutants, up to 7 days ahead | \n| Weather forecast | **Open-Meteo** archive + forecast API | boundary-layer height, wind, humidity, temperature, precipitation: the features that explain *why* CAMS is wrong | \n| Ground truth | **OpenAQ v3** , sensor 4554236 (AirGradient, operated by The Urban Unit, CC BY 4.0) | real hourly PM2.5 5.2 km from the CAMS grid point, used both to train the correction and to score the forecast | \n\nCAMS is a serious atmospheric model, but at this specific point it is *predictably* wrong. Measured against the US Embassy reference monitor over ~28,000 overlapping hours, CAMS under-reads by a mean of **18.9 µg/m³**, and the error is strongly hour- and season-dependent: in Oct–Dec it misses the monitor's 9am peak by up to 90 µg/m³ and its evening peak by up to 70. That structured, repeating error is exactly the kind of thing a tabular model can learn.\n\nEvery forecast card displays all three side by side. The weak baseline doesn't get quietly dropped when the fancy model wins:\n\n```\n# src/forecast.py: TabPFN v2, local CPU, no tuning\nm = TabPFNRegressor.create_default_for_version(\n    ModelVersion.V2, device=\"cpu\", fit_mode=\"fit_with_cache\",\n    ignore_pretraining_limits=True, random_state=0)\nm.fit(tr[FEATURES], tr[\"y\"].to_numpy())\npred = pd.Series(m.predict(live[FEATURES]), index=live.index)\n```\n\n`ignore_pretraining_limits=True` is the load-bearing flag: TabPFN v2 refuses more than ~1,000 training rows without it, and pushing to 10,000 rows on CPU is also why a fit takes 30–90 minutes rather than seconds. The fitted model is cached to disk (**6.3 GB** at 10,000 rows) so repeat runs reload in ~2 minutes instead of refitting. On this laptop the real first fit took **3,351 seconds, or 56 minutes**.\n\nAn hour-long fit does not fit neatly into \"commit before the day starts.\" Rather than fudge it, the Oct 11 forecast shipped in two explicitly-labeled stages:\n\n`69c7d9e`` 802344a`\nThe card says in plain words: *hours before 06:51 had already occurred and are not pre-registered for TabPFN.* The scoring script then reports the full day **and** a separate ≥07:00 view, re-deriving each model's window from only the hours that were genuinely still in the future. An ugly two-stage commit with an honest caveat beats a tidy story that quietly implies a 56-minute model finished at midnight.\n\n```\n# scripts/score_forecast.py: reads the frozen forecast, never writes to it\nfc = json.loads((ROOT / \"results\" / f\"forecast_{day}.json\").read_text(encoding=\"utf-8\"))\n...\nfor name, m in fc[\"models\"].items():\n    p = pd.Series(m[\"hourly\"], index=idx).reindex(sel)\n    both = pd.concat([p, ys], axis=1, keys=[\"p\", \"y\"]).dropna()\n    err  = both[\"p\"] - both[\"y\"]\n    rho  = both[\"p\"].corr(both[\"y\"], method=\"spearman\") if len(both) > 2 else float(\"nan\")\n```\n\nScoring writes two *new* files and never edits the committed forecast. The CLI enforces the same rule from the other direction: `--save` refuses to overwrite an existing `results/forecast_<date>.json` without `--force`, because that file may be a pre-registration. The numbers can't be nudged after the fact, including by me.\n\n**Repo:** [https://github.com/Abeera81/clear-hour](https://github.com/Abeera81/clear-hour) (MIT; data and model weights keep their own licenses)\n\nInteresting files:\n\n`src/forecast.py`: the engine (live feature fetch, the three models, window selection)`src/clearhour/`: the `python -m clearhour` CLI, with packaged hour-bias offsets so the default path needs no API key`eval/evaluate.py`: the temporal split; `results/eval_test.json` exists`eval/learning_curve.py`: the 6-fold rolling-origin \"how much data does TabPFN need?\" experiment`scripts/score_forecast.py`: pulls real measurements and scores a frozen forecast` results/forecast_2026-10-11.json`: the frozen pre-registration itself\nEvery forecast run writes its exact 24 × 26 feature matrix to CSV and prints the sha256, so any forecast can be audited against the inputs that produced it.\n\nStrictly temporal split: train through Aug 10, validate on the following month, and reserve a final 30-day TEST month (Sep 10 – Oct 9) that every model sees **once**. `eval/evaluate.py` refuses a second TEST run, so a disappointing result can't quietly be re-rolled.\n\n| Model | MAE | RMSE | Bias | Within-day ρ (median) | Top-2h hit % | \n|---|---|---|---|---|---|\n| B0 raw CAMS | 16.1 | 20.7 | −3.4 | 0.564 | 30% | \n| B2 CAMS+hour bias | 22.1 | 28.7 | +20.2 | 0.544 | 43% | \n| B5 HistGBR | 10.8 | 15.2 | −4.1 | 0.644 | 40% | \n| **M1 TabPFN v2** | **10.3** | **14.6** | −4.2 | **0.675** | 37% | \n\nTabPFN has the lowest error of anything I tried, and its MAE advantage over B2 is solidly real (95% CI of the difference −17.0 to −6.9 µg/m³). But its *ranking* edge over B2 is **not statistically significant** at this sample size (CI −0.04 to +0.29), and an untuned HistGBR actually beat it on the DEV month. Picking the exact best 2-hour window tops out at 43% for every model. I'd rather print those three caveats than a single bolded win.\n\nThis was the question I most wanted answered, and it produced the result I'd have been most tempted to leave out.\n\n*Six rolling-origin folds, training on the N most recent hours before each test window: recency, never random sampling. The whiskers are as wide as most of the gaps, which is itself the finding.*\n\n| N (training hours) | TabPFN ρ | B2 ρ | TabPFN MAE | B2 MAE | \n|---|---|---|---|---|\n| 100 | 0.317 | **0.439** | 19.7 | 19.2 | \n| 500 | 0.431 | **0.555** | 17.9 | 21.3 | \n| 1,000 | **0.544** | 0.535 | 17.2 | 24.1 | \n| 2,500 | **0.616** | 0.543 | 16.0 | 30.2 | \n| 5,000 | **0.590** | 0.476 | 16.9 | 44.5 | \n\n**TabPFN v2 needs roughly 1,000–2,500 training hours (about 6–15 weeks of hourly data) before it matches, then edges, a one-line hour-of-day baseline on hour ranking. Below that it is worse and erratic** (at N=100 its ranking swings ±0.34 across folds). For a foundation model sold on few-shot strength, \"you need six weeks of data before you beat one line of arithmetic\" is a genuinely useful negative result, and it's the kind of thing a hackathon post usually buries.\n\nOne result I didn't expect: I trained the correction on sensor 4554236 and applied it, with **no retraining whatsoever**, to a different sensor also ~5 km from the CAMS point. It transferred: TabPFN MAE 16.8 / ρ 0.661 there, versus 27.5 / 0.570 for raw CAMS. A correction learned at one point in the city carries to another point without being re-fit. That matters for the next section.\n\nThis is not an abstract question for this project. It's the reason the project is possible at all.\n\n**There is no one to buy this from.** Pakistan has no public hourly air-quality forecasting service. The only reference-grade monitor in Islamabad, the US Embassy BAM, stopped publishing in **February 2026**, and nothing replaced it. What's left is a scatter of uncalibrated low-cost sensors run by a local organization. No commercial forecasting vendor is going to build an hour-resolution smog model for Rawalpindi; the addressable market doesn't justify it. If this gets built, it gets built out of open components by someone who lives here.\n\n**Every single layer is open, and each one is load-bearing:**\n\nThat last point deserves specifics. A 10,000-row fit takes 56 minutes on this laptop. If TabPFN were a closed, metered, hosted API, that cost would land on every city, every sensor, every refit, and the project would die on its unit economics before it helped anyone. Because the weights are open and inference is local, the marginal cost of correcting a new sensor is *CPU time on hardware someone already owns*. The cross-sensor transfer result above is what makes that concrete: the correction generalizes to a neighboring sensor without retraining, so someone in Lahore or Peshawar can clone this repo, point it at their own OpenAQ sensor, and get a working forecaster, with no key, no quota, and no permission from me or anyone else.\n\n**Open is also what makes the honesty enforceable.** A pre-registered forecast is only meaningful if a stranger can verify it. Mine is a git commit with a timestamp, a public hash, and a feature matrix checksum. Anyone can confirm that the Oct 11 prediction existed at 06:10 on Oct 11 and hasn't been edited since. You cannot audit a closed forecaster's track record; you can only read its marketing. The negative result in this post is published *because* the architecture made hiding it harder than publishing it.\n\nAnd the license cuts both ways, which I found out the hard way (see below).\n\n`requirements.txt` wasn't installable by anyone but me.`--extra-index-url` plus a `+cpu` pin made `uv` reject PyPI versions of unrelated packages. I only found out by installing from a clean clone. Torch is now its own documented step.\nClear Hour exists *because* CAMS has a structured, hour- and season-dependent bias, and TabPFN v2 is the component that corrects it: **locally, on CPU, no fine-tuning, no hosted inference**. It isn't a wrapper around an API call. The regressor is fit fresh on up to 10,000 rows of real sensor history, cached to disk, reloaded on repeat runs, and its predictions drive the product's actual output (which two hours to go outside).\n\nIt's also evaluated the way I'd want someone to evaluate a forecasting claim made *against* me:\n\n**License compliance (Prior Labs License 1.1):** \"Built with PriorLabs-TabPFN\" appears in the README header, in every CLI run's footer, and on the live demo page. Full license copy: [`docs/licenses/PriorLabs-TabPFN-v2-LICENSE.txt`](https://github.com/Abeera81/clear-hour/blob/main/docs/licenses/PriorLabs-TabPFN-v2-LICENSE.txt). Hollmann et al., *Nature* 2025, doi:10.1038/s41586-024-08328-6.\n\nTinker was explored as an optional stretch (two LoRA training clients created as a capability probe), but nothing was trained or sampled with it, so it is **not** claimed as a result here.\n\nScore more days as they arrive. One scored day is a start, not a track record. Retrain through a full winter once one exists. And keep the rule unchanged: commit the forecast before the day, publish the score regardless of what it says.\n\nIf you're somewhere with bad air and a nearby OpenAQ sensor, the repo should work for you with a one-line change, and if it doesn't, that's a bug I'd like to hear about.\n\n**Repo:** [github.com/Abeera81/clear-hour](https://github.com/Abeera81/clear-hour) · **Live demo:** [abeera81.github.io/clear-hour](https://abeera81.github.io/clear-hour/)\n\nBuilt solo over the challenge window, with GitHub Copilot CLI as co-author. The repo's `AGENTS.md`, `MEMORY.md`, and Conventional Commit history show the process, not just the result.\n\n**Built with PriorLabs-TabPFN.** Weather and air-quality model data via [Open-Meteo.com](https://open-meteo.com/) (CC BY 4.0) / Copernicus Atmosphere Monitoring Service. Air-quality measurements: AirGradient sensors operated via The Urban Unit (CC BY 4.0), retrieved from [OpenAQ](https://openaq.org/); uncalibrated, and not an endorsement by The Urban Unit.", "url": "https://wpnews.pro/news/clear-hour-i-committed-an-air-quality-forecast-to-git-before-the-day-started-it", "canonical_source": "https://dev.to/abeera_lodhi/clear-hour-i-committed-an-air-quality-forecast-to-git-before-the-day-started-then-scored-it-in-p9n", "published_at": "2026-10-12 02:51:15+00:00", "updated_at": "2026-10-12 03:07:25.940149+00:00", "lang": "en", "topics": ["machine-learning", "ai-tools", "ai-research"], "entities": ["Clear Hour", "TabPFN v2", "CAMS", "OpenAQ", "Islamabad", "Rawalpindi", "GitHub", "Abeera81"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/clear-hour-i-committed-an-air-quality-forecast-to-git-before-the-day-started-it", "markdown": "https://wpnews.pro/news/clear-hour-i-committed-an-air-quality-forecast-to-git-before-the-day-started-it.md", "text": "https://wpnews.pro/news/clear-hour-i-committed-an-air-quality-forecast-to-git-before-the-day-started-it.txt", "jsonld": "https://wpnews.pro/news/clear-hour-i-committed-an-air-quality-forecast-to-git-before-the-day-started-it.jsonld"}}