Stop Writing .isnull() — Audit Your Dataset in One Line with OMR

Every time I start a new data project, I write the same boilerplate:

python
df.isnull().sum()
df.duplicated().sum()
df.dtypes
df.describe()
df[df['age'] < 0]
After doing this for the hundredth time, I built OMR (Omni Data Refinement) — a Python library that replaces all of that with a single call.

What is OMR?
OMR is an open-source Python framework for dataset quality, validation, profiling, and monitoring. Think of it as a health check for your data — before you do any ML, EDA, or transformation, you should know how healthy your data actually is.

Installation
bash
pip install omni-data-refinement
Only needs pandas, numpy, and rich. No cloud, no LLMs.

The Core Feature: Health Score
python
import pandas as pd
from omr import Dataset
df = pd.read_csv("your_data.csv")
report = Dataset(df).health()
print(report.score)  # e.g. 87/100
You get a 0-100 quality score covering 5 dimensions:

Pillar  What it checks
Completeness    Missing values
Uniqueness  Duplicates
Consistency Type mismatches
Validity    Value ranges and format rules
Conformity  Schema adherence
Auto-Cleaning
python
dataset = Dataset(df)
dataset.clean()
dataset.explain_changes()  # See exactly what was fixed
OMR auto-resolves missing values, duplicates, and type mismatches — and gives you a full transformation log so nothing is a black box.

Schema Validation
python
from omr import schemas
schema = {
    "age":    schemas.PositiveInteger(max=120),
    "salary": schemas.PositiveFloat(min=10000),
    "status": schemas.OneOf("active", "inactive"),
    "email":  schemas.Email()
}
dataset.validate(schema)
Drift Detection
Compare your current dataset against production or a previous version:

python
prod_dataset = Dataset(pd.read_csv("prod_data.csv"))
dataset.compare(prod_dataset)
Uses PSI, KS Test, and JS Divergence under the hood.

Statistical Analysis
python
dataset.analyze()
Column Profiling (replaces .describe())
python
dataset.profile()
Unlike .describe() which only covers numeric columns, OMR profiles every column — numeric, categorical, and boolean — in a clean terminal table.

The Fluent API
Chain everything together:

python
clean_df = (Dataset(df)
    .health()
    .clean()
    .analyze()
    .export())  # Exports HTML, Markdown, or JSON report
Why I Built This
The goal is for OMR to become the first thing you run after pd.read_csv() — not a replacement for Pandas, but the quality layer on top of it.

Links
PyPI: pip install omni-data-refinement
GitHub: https://github.com/Omar-Alshafai2/omni-data-refinement
Docs: https://Omar-Alshafai2.github.io/omni-data-refinement/
Examples: https://Omar-Alshafai2.github.io/omni-data-refinement/examples/

What data quality problems do you run into most? I would love to know what to build next.