A CSV can be completely happy with a temperature in the wrong unit.
It can also accept the same observation twice, a future year in a historical training file, and NaN where a measurement should be. The file opens. The downstream calculation gets a convincing-looking input. Nobody has checked what that input means yet. 🙃
Project ProtoStar's launch put scientific-data reuse back on my reading list.[1] I wanted something smaller than another “make data AI-ready” checklist, so I built a standard-library validator with deliberately broken climate CSVs.
The valid fixture passes. Seven corrupted or incomplete variants produce errors, including a unit mismatch, a duplicate key and a row crossing the declared training cutoff. These are synthetic teaching fixtures and my own contract, not ProtoStar's official dataset or submission format.
Original illustration of the checks implemented below. Passing them does not establish scientific correctness.
My tiny format has four columns:
city,year,annual_mean_temperature,unit
Example City,2017,12.5,degC
Example City,2018,13.0,degC
The contract says one row describes one city in one year. The key is (city, year). Years must be four digits, fall within a declared teaching window starting in 1900, and not exceed the training cutoff. Temperature must be numeric, finite and between −60 and 60 degrees Celsius. Every row must explicitly say degC.
The temperature range is an authored guardrail for this example. It is not a universal climate validity rule. A real dataset needs bounds justified by its geography, aggregation method and measurement definitions.
I also made the header strict and rejected extra or missing fields. That choice catches accidental format drift early. A production schema can allow optional fields, but those fields should be intentionally optional rather than silently ignored.
The complete validator and fixtures are one Python file. Save it as contract.py, then run:
python3 contract.py
It uses csv, io, json, math and re from the standard library. There is no model, API key or data download. The script runs the valid input and seven variants, checks their expected acceptance or rejection, and prints every issue with its line number.
Some of the executed output:
valid: []
duplicate: line 4: duplicate city-year
future_training_row: line 4: year outside training window 1900..2018
nonfinite: line 2: temperature must be finite
The actual script prints this information as JSON. Empty issues mean “no declared rule was violated,” not “this dataset is correct.”
Here is the key-checking portion of the implementation:
year_text = row["year"]
if not re.fullmatch(r"[0-9]{4}", year_text):
fail("year must have four digits")
else:
year = int(year_text)
if not 1900 <= year <= cutoff:
fail(f"year outside training window 1900..{cutoff}")
key = (city, year)
if key in seen:
fail("duplicate city-year")
seen.add(key)
seen is a set created before reading the rows. fail appends an issue prefixed with the current CSV line number. The full linked file includes those definitions and the remaining checks; the extract above belongs inside its row loop.
Units: a value such as 55 might be plausible in Fahrenheit but very unusual as an annual mean in Celsius. A broad range check alone cannot reliably decide which interpretation was intended. My contract rejects degF; it does not silently convert it.
If conversion is needed, I would put it in a separate explicit transformation and retain the source unit. Otherwise a convenience function can conceal which values were transformed, or apply a conversion twice.
Duplicate keys: two rows for Example City in 2018 might be a duplicate import, a corrected observation, or two stations accidentally collapsed into one city name. Automatically keeping the last row would make that scientific decision invisible. The validator raises the question instead.
Time: a 2019 row can be a perfectly reasonable climate observation and still be the wrong input to training that must stop at 2018. That is a role violation, not necessarily a bad measurement. A separate target file needs a different contract. Applying the training rule to every file would reject legitimate prediction targets.
Finite values: float("NaN") succeeds in Python. “Convertible to float” therefore does not imply usable numeric input. math.isfinite catches both NaN and infinity before they travel into means, losses or plots.
The remaining fixtures cover an empty dataset, a year written as 2017.0, and an extra column value. These are intentionally small examples so the failing line is obvious.
Suppose I label Fahrenheit values as degC. A moderate value might pass every implemented rule. The validator cannot read the instrument's intentions.
It also does not resolve city aliases, identify station changes, establish measurement provenance, check complete yearly coverage, or prove that annual means were aggregated correctly. A syntactically valid year does not reveal when a source was actually available. A retrospective correction can still create leakage even when its nominal observation year is old enough.
That is where a data card and source history matter. I would attach the observation definition, units, geographic identity, missing-value policy, source version and availability cutoff to the artifact. The validator enforces the portion that can be made executable; documentation carries the rest.
For a larger pipeline, I would also return structured issue codes instead of relying on message text, and choose explicit rules for case, whitespace and city identifiers. The example strips surrounding city whitespace but otherwise treats names literally. Two spellings can therefore evade its duplicate check. That limitation belongs next to the key definition, not buried in a model evaluation.
ProtoStar's first announced challenge concerns climate-risk forecasts, with historical records through 2018 and later prediction targets. Its challenge window opens on November 2.[1] That temporal split motivated my teaching cutoff. I have not entered the challenge or evaluated forecasts on its data.
The broader lesson is useful before any training run: an accessible file and a reusable scientific artifact are different milestones. A dozen lines of explicit assumptions can prevent hours of experimenting on an input that means something else.
The figure, fixtures and validator are original teaching material. No official challenge data or third-party images are reproduced.
AI was used to draft the article.