# Why is there almost no manipulation data for agriculture? > Source: > Published: 2026-07-28 10:29:29+00:00 Hmm… From what I could find, it seems that the relevant work is scattered across several fields and is therefore hard to discover: My read is that this is **not simply an absence of interest**. People are collecting agricultural manipulation demonstrations and testing learned policies, but the public work is fragmented into small, crop-specific, hardware-specific projects rather than forming an agricultural equivalent of DROID or Open X-Embodiment. The second part of your question is also slightly different from “no current format can represent this.” [LeRobotDataset v3](https://huggingface.co/docs/lerobot/en/lerobot-dataset-v3) can already store synchronized video, state, action, timestamps and custom features. The missing piece appears to be a broadly used convention that links: I would therefore use LeRobot as the **trajectory layer**, rather than trying to force the entire farm-to-packhouse history into one episode table. A reasonable default design would be: ``` immutable raw recording ↓ LeRobot trajectory dataset │ └── harvest_event_id / target_id ↓ linked event and outcome table ├── harvest / container / packing events ├── inspection time ├── measured quality ├── inspection method and scale ├── missing-data reason └── causal attribution, if any, kept separate ``` Before collecting at scale, I think the highest-value route would be to take a **one-crop, one-task pilot schema** to the [LeRobot Discord](https://discord.com/invite/q8Dzzpym3f). That is probably the best place to check action representation, custom features, converter design and whether the resulting dataset will work with current policies. The [Hugging Science community](https://huggingface.co/hugging-science) also seems relevant, but for a different reason: it explicitly focuses on cross-disciplinary collaboration, data fragmentation, standardized formats and shared evaluation. That may be a better place to find plant science, postharvest, data-curation or benchmark collaborators. The most important design question to settle first is probably: What identity can remain attached to the harvest event through the packhouse? | What remains traceable? | What the later label can safely mean | |---|---| | Individual fruit | The later observation can be joined directly to one harvest event | | Tray, bin or small lot | The later result should remain an aggregate outcome, not be assigned to one grasp | | Only a large mixed lot | Useful for operational statistics, but weak supervision for individual trajectories | | No persistent linkage | Start with immediate outcomes, or modify the tagging workflow before collecting delayed labels | Everything else—including whether weather belongs per-frame or per-episode—depends somewhat on that identity boundary. Closest examples I could findIf I were starting from your position, my default path would be: The combination of farm and packhouse access is particularly valuable because it could capture a linkage that most current robot datasets cannot: not merely whether the robot completed the motion, but what happened to the biological product afterward. Even a modest dataset that preserves that linkage cleanly could be more reusable than a much larger collection of trajectories with only a binary success label.