I asked ChatGPT to analyze a dataset of global health metrics. It came back with confident conclusions: “Countries with higher GDP have longer life expectancy.” “Population growth correlates with lower life spans.”
They sounded right. Some were. But I had no way to tell which, because what I got was a story, not evidence. No p-value. No sample size check. No way to separate a real pattern from a coincidence that happened to sound plausible.
That’s the part nobody mentions when they say “just ask the AI to analyze your data.” An LLM has never seen your dataset the way a statistical test has. It’s pattern-matching against the shape of language, not testing a hypothesis against the shape of your numbers.
So I built HypoLab: an agentic pipeline where an LLM is allowed to propose ideas, but classical statistics get the final vote.
This is the question I keep getting, so let me answer it directly: why not just trust the LLM’s summary, or a simple correlation heatmap?
Because “these two things look related” and “these two things are statistically related, at a known confidence level, accounting for sample size” are different claims — and only one of them is falsifiable. A hypothesis test forces a specific, checkable structure onto an idea: a null hypothesis, an alternative, a test statistic, a p-value. It can fail. That’s the entire point. An LLM’s prose summary can’t fail in a way you can check — it just sounds more or less convincing.
Hypothesis testing is also the same mechanism science and business analytics have used for a century to decide whether a pattern is worth acting on before you act on it. HypoLab doesn’t invent that idea. It just makes an LLM’s output pass through it before you see it, instead of trusting the output on vibes.
HypoLab runs a strict four-stage loop:
-
Profile — ingests a CSV, auto-detects column types, correlations, outliers, missing data
-
Propose — an LLM agent (or a data-driven fallback, more below) generates 3–5 falsifiable hypotheses, each with a null/alternative hypothesis, the statistical test needed, the variables involved, and a one-line rationale
-
Verify — every hypothesis runs against the real data via scipy and statsmodels: Pearson correlation, one-way ANOVA, Chi-square, independent t-test, Granger causality, or OLS regression. If p > 0.05, the hypothesis dies. No appeals.
-
Support — hypotheses that survive get matched against related literature via the arXiv API
HypoLab doesn’t force you into a paid API. It auto-selects the best available backend:
🟢 Groq (Llama 3.3–70B) — when GROQ_API_KEY is set. Fast, full LLM reasoning, free tier available.
🟡 Ollama (Llama 3.1–8B) — when USE_OLLAMA=true. Local, private, runs on your own machine.
🟠 Smart Analysis — when nothing is configured. Rule-based, reads your actual correlation matrix and column types to generate intelligent hypotheses.
The third tier is the one I’m most protective of. It’s not a templated stub — it looks at your strongest correlation pair, your variance, your column types, and proposes real tests from that. HypoLab runs meaningfully with zero API keys and zero cost.
I ran HypoLab against the Gapminder dataset — 1,704 rows spanning 50 years of health and economic data across 142 countries — and independently re-verified every statistic below directly against the raw data before publishing this:
A demo that only shows the dataset you tuned it against doesn’t prove much. Before writing this, I ran HypoLab against datasets I hadn’t touched before, specifically looking for where it would break.
**Palmer Penguins **(344 rows, 3 species) came back clean: 5 hypotheses, all 5 significant, across four different test types — Pearson, ANOVA, regression, chi-square.
Iris (150 rows, the classic) did the same: 4 hypotheses, 4 significant.
Titanic is where it got interesting. Two of five hypotheses HypoLab generated were ANOVA(PassengerId, Name) and chi-square(Name, Sex) — technically valid test types, meaningless comparisons, because Name is nearly a unique value per row and PassengerId is a row index that happens to be stored as an integer. The profiler currently treats "not numeric" as "categorical" without checking whether a column is actually a meaningful group label or just an identifier wearing a different data type. The statistical layer caught one of the two outright (not enough data per group to even run the test) and correctly returned "not significant" on the other. Nothing false got reported as true — but two of five hypothesis slots got wasted on a comparison no analyst would ever propose.
I also noticed something else while testing: the free Smart Analysis tier rarely proposes Granger causality hypotheses in practice. The code does have the branch, but Granger causality requires time-series structure — sequential observations per entity — and most flat CSVs don’t have that. So the branch exists, but it only fires when your data actually looks like a time series. Not broken. Just a reminder that the sophistication of your hypotheses is bounded by the structure of your data.
The real question isn’t whether HypoLab works on clean demo data. It’s what happens when the hypothesis generator proposes something wrong.
I built a synthetic benchmark to check: a dataset with 3 real, engineered relationships sitting next to 3 columns of pure, structureless noise. I ran the pipeline blind. It proposed hypotheses touching both the signal and the noise — because it doesn’t know in advance which is which.
The statistical layer accepted all 3 real relationships and rejected the noise, every time.
This isn’t a demo I ran once for this article. It’s test_ground_truth.py, one of 40 tests that run on every commit — verified again, live, before I wrote this piece. The LLM can propose anything. The math decides what survives.
Two fixes are queued now, both found by actually testing against messier data instead of just the curated demo:
• Filter high-cardinality, ID-like columns (Name, Ticket-style identifiers) out of hypothesis candidates before they waste a slot on a meaningless comparison — the Titanic result above is the concrete case.
• Add cardinality checks to the profiler so “categorical” means “meaningful groups,” not just “not a number.”
Beyond those, the longer roadmap — none of it built yet, all of it genuinely planned:
HypoLab is MIT-licensed and pip-installable:
GitHub Repository: github.com/gauravnepal/hypolab
git clone https://github.com/gauravnepal/hypolab.gitcd hypolab && pip install -e .streamlit run app.py
HypoLab: LLM proposes. Math verifies.
Gaurav Nepal
Machine Learning · Data Engineering · Agentic AI
Explore more projects on GitHub: github.com/gauravnepal
LLM Proposes. Math Verifies: Inside HypoLab, My Answer to Hallucinated Data Insights was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.