Introducing Calibra: Open-source Robot Dataset Observability

Calibra, an open-source toolkit for robot dataset observability, has been released to help audit datasets, identify quality issues, and estimate training outcomes before training. The first public release includes a Robot Dataset Health Check interactive Space for auditing LeRobot datasets, a benchmark of 30 public LeRobot datasets with health scores and diagnostics, and a Python API/CLI for quality-aware coreset selection and training outcome prediction.

Hey everyone I’d like to share Calibra which is an open-source toolkit for robot dataset observability. I realized that while robot learning has excellent tooling for training policies, it’s still difficult to understand the quality of a dataset before training. Calibra aims to help by auditing datasets, identifying potential quality issues, building quality-aware coresets, and estimating training outcomes. As part of the first public release, we’ve launched: Robot Dataset Health Check — an interactive Space for auditing LeRobot datasets in seconds. A benchmark of 30 public LeRobot datasets with health scores, quality diagnostics, and community comparisons. The full toolkit also includes quality-aware coreset selection, training outcome prediction, and a Python API/CLI. Try it: Robot Dataset Health Check - a Hugging Face Space by omert27 https://huggingface.co/spaces/omert27/robot-dataset-health-check Source code: GitHub - omertt27/Calibra: Dataset observability and coreset selection for robotics imitation learning · GitHub https://github.com/omertt27/Calibra I’d really appreciate feedback from the Hugging Face and robotics communities. Some questions I’d love your thoughts on: Which dataset quality metrics would be most useful for your workflow? Are there important diagnostics that are missing? If you work with LeRobot datasets, would you use something like this before training? Thanks for taking a look, i would be appreciated any feedbacks.