{"slug": "open-source-tool-for-detecting-issues-in-robot-learning-datasets", "title": "Open-source tool for detecting issues in robot-learning datasets", "summary": "Developer omert27 released Calibra, an open-source toolkit for detecting issues in robot-learning datasets, designed to catch corrupted or invalid data before training. The tool targets users of LeRobot and imitation learning, and community feedback highlighted the need for benchmarks measuring false-positive rates on clean datasets.", "body_md": "**Open-source tool for robot dataset issues**\n\nI’ve been working on **Calibra**, an open-source toolkit for analyzing robot-learning datasets.\n\nWhile working with demo data, I noticed a common issue: we often waste compute training on data that shouldn’t be there in the first place.\n\nDatasets can include:\n\nSo I built Calibra to help catch these issues before training.\n\nWould love to hear thoughts from anyone working with **LeRobot or imitation learning.**\n\nGitHub: [GitHub - omertt27/Calibra: Dataset observability and coreset selection for robotics imitation learning · GitHub](https://github.com/omertt27/Calibra)\n\nI would appreciate any feedbacks.\n\n \n \nThis is a useful problem. One thing I’d want to see is a clear split between corrupted data and unusual but valid demonstrations, since rare movements can still be valuable in real-world data. A few visual examples for each failure type would make the tool much easier to trust.\n\n \n \n[Sattyam](https://discuss.huggingface.co/u/Sattyam)\n5\n \nThe corrupted-versus-unusual-but-valid distinction raised upthread is the whole problem, and it has a measurable form: what is Calibra’s false-positive rate on a dataset you know is clean?\n\nConcretely. Take a LeRobot dataset you are confident in, run the detector, and publish the per-check flag rate. Jittery motion and calibration drift will both flag on some genuinely good demonstrations, because a human demonstrator is jittery and a real rig does drift. That rate is not a bug to tune to zero; it is the number a user needs in order to decide what to do with 40 flags on their own data.\n\nThe reason this matters more here than in most tooling: a detector at the dataset layer sits upstream of every policy trained on it. If it silently removes 8 percent of valid demonstrations on a particular task family, the downstream success rate moves and nobody attributes it to the cleaner.\n\nSame discipline we hold ourselves to one layer down, at policy evaluation: every detection rate gets published beside its firing rate on known-benign input, and if the firings concentrate on a subset rather than spreading, that is a signal about the detector, not the data.\n\n \n \n[omert27](https://discuss.huggingface.co/u/omert27)\n6\n \nThanks for the feedback Sattyam, i will implement these benchmarks.", "url": "https://wpnews.pro/news/open-source-tool-for-detecting-issues-in-robot-learning-datasets", "canonical_source": "https://discuss.huggingface.co/t/open-source-tool-for-detecting-issues-in-robot-learning-datasets/178602#post_6", "published_at": "2026-09-06 13:12:24+00:00", "updated_at": "2026-09-07 02:03:12.668908+00:00", "lang": "en", "topics": ["robotics", "ai-tools", "machine-learning"], "entities": ["Calibra", "omert27", "LeRobot", "Sattyam"], "alternates": {"html": "https://wpnews.pro/news/open-source-tool-for-detecting-issues-in-robot-learning-datasets", "markdown": "https://wpnews.pro/news/open-source-tool-for-detecting-issues-in-robot-learning-datasets.md", "text": "https://wpnews.pro/news/open-source-tool-for-detecting-issues-in-robot-learning-datasets.txt", "jsonld": "https://wpnews.pro/news/open-source-tool-for-detecting-issues-in-robot-learning-datasets.jsonld"}}