krAnand 1
I built a small CLI that profiles physical-AI and lab datasets on the Hugging Face Hub without down them. It reads only the bytes that hold metadata — headers, indexes, footers — through HTTP range requests, and reports streams, message counts, rates, jitter, gaps, episodes, NaN counts, and problems like truncated or unindexed recordings.
Formats: MCAP, ROS 1 .bag, ROS 2 .db3, MATLAB .mat, Parquet, and LeRobot datasets as a whole — state and actions from the Parquet tables, each camera from its MP4. v0.4.0 reads the MP4 moov index (codec, resolution, per-frame timestamps, keyframes); frames are never decoded.
On large files that’s usually well under 1%: a 5.8 GB ROS 1 bag is profiled from 533 KB, a 493 MB camera video from 444 KB.
Write what your model needs in a YAML file and get a pass / fail / unknown verdict — unknowns are never guessed passes:
pip install phys-ai-dataprofiler
phys-ai-dataprofiler check lerobot/pusht -r preset:teleop-imitation
phys-ai-dataprofiler compare org/a org/b -r req.yaml
PyPI: [Client Challenge](https://pypi.org/project/phys-ai-dataprofiler/)
GitHub: [GitHub - akrishni/phys-ai-dataprofiler · GitHub](https://github.com/akrishni/phys-ai-dataprofiler)
Feedback welcome — especially datasets it gets wrong, and requirements presets you’d want built in.
nootxlm 2
The pass-before-you-download shape is exactly right, and one check fits it while the timestamps are already in hand: cross-stream alignment as its own verdict — how much the streams’ time ranges overlap, and the largest gap between a camera frame and its nearest state or action sample. Every stream can pass on rate, jitter, and gaps while the set still trains badly because the camera sits a fixed offset behind the actions; a shifted-stream fixture proves the check, since shifting one stream leaves every per-stream number unchanged and flips only the alignment verdict.