{"slug": "converting-libero-hdf5-to-lerobot-format-how-to-map-79-dim-state-to-8-dim", "title": "Converting LIBERO HDF5 to LeRobot format: How to map 79-dim state to 8-dim?", "summary": "A developer seeking to convert LIBERO HDF5 datasets to LeRobot format asked how to map the 79-dimensional state vector to LeRobot's 8-dimensional observation.state. The response clarifies that the 79-D vector is a flattened MuJoCo simulator state, while LeRobot's 8-D state is constructed from 3-D end-effector position, 3-D axis-angle orientation, and 2-D gripper qpos, advising against selecting indices from the 79-D vector and recommending checking for existing proprioceptive fields or reconstructing observations via the LIBERO environment.", "body_md": "Hmm… I think you’d probably get a more definitive answer by asking in the LeRobot Discord (linked from [lerobot (LeRobot)](https://huggingface.co/lerobot)), but for now:\n\nI think the main distinction here is that the 79-D `states` vector and LeRobot’s 8-D `observation.state` are probably **two different kinds of state**, rather than the latter being a subset or dimensionality reduction of the former.\n\nIn the standard LIBERO data path, the HDF5 `states` array is used as a flattened MuJoCo simulator state. LIBERO’s own [`create_dataset.py`](https://github.com/Lifelong-Robot-Learning/LIBERO/blob/master/scripts/create_dataset.py) restores that simulator state, replays the trajectory, and separately constructs proprioceptive observations such as:\n\n```\nee_state = np.hstack(\n    (\n        obs[\"robot0_eef_pos\"],                     # 3\n        T.quat2axisangle(obs[\"robot0_eef_quat\"]), # 3\n    )\n)\n\ngripper_state = obs[\"robot0_gripper_qpos\"]         # 2\n```\n\nThat gives:\n\n```\n3 (EEF position)\n+ 3 (EEF orientation as axis-angle)\n+ 2 (gripper qpos)\n= 8\n```\n\nThis is also the representation currently documented by [LeRobot’s LIBERO integration](https://huggingface.co/docs/lerobot/main/libero): `observation.state` is 8-D EEF position + axis-angle orientation + gripper qpos.\n\nSo, **if your custom HDF5 follows the usual LIBERO layout, I would not start by selecting eight indices out of the 79-D vector.** I would first check whether the HDF5 already contains the proprio fields used to construct the 8-D state.\n\nA practical decision tree would be:\n\n```\nDoes demo_0/obs contain ee_states (6) and gripper_states (2)?\n│\n├─ Yes\n│   └─ observation.state = concat(ee_states, gripper_states)\n│\n└─ No\n    │\n    ├─ Do you have eef_pos + eef_quat + gripper_qpos?\n    │   └─ Build 3 + quat→axis-angle(3) + 2\n    │\n    └─ Only the simulator states are available\n        │\n        ├─ Can the LIBERO environment be reconstructed?\n        │   └─ Restore each simulator state and regenerate observations\n        │\n        └─ The 79-D layout is custom / unknown\n            └─ Check the exporter/schema first; don't guess fixed indices\n```\n\nThe [`any4lerobot/libero2lerobot`](https://github.com/Tavish9/any4lerobot/tree/main/libero2lerobot) converter is a useful concrete example here: its LeRobot state is 8-D, while it also preserves richer native LIBERO state information separately. In other words, making a training-compatible `observation.state` does **not** require throwing the original simulator/robot state away.\n\n`features` / `meta/info.json`\nI would treat the feature schema as the **output contract of the conversion**, not as the thing that determines how the 79-D source vector should be interpreted.\n\nConceptually:\n\n```\nsource HDF5\n    ↓\ninterpret source semantics\n    ↓\nconstruct observation.state / action / images\n    ↓\ndeclare those outputs in LeRobot `features`\n    ↓\nwrite the LeRobot dataset\n```\n\nWith current LeRobot v3, I would normally define the semantic features when creating the dataset rather than manually constructing the whole `meta/info.json` first.\n\nA simplified skeleton is roughly:\n\n```\nfeatures = {\n    \"observation.state\": {\n        \"dtype\": \"float32\",\n        \"shape\": (8,),\n        \"names\": {\n            \"motors\": [\n                \"x\", \"y\", \"z\",\n                \"axis_angle1\", \"axis_angle2\", \"axis_angle3\",\n                \"gripper_left\", \"gripper_right\",\n            ]\n        },\n    },\n\n    \"action\": {\n        \"dtype\": \"float32\",\n        \"shape\": (7,),\n        # Add names that match your actual action semantics.\n    },\n\n    \"observation.images.image\": {\n        \"dtype\": \"video\",   # or \"image\", depending on your choice\n        \"shape\": (HEIGHT, WIDTH, 3),\n        \"names\": [\"height\", \"width\", \"rgb\"],\n    },\n\n    \"observation.images.image2\": {\n        \"dtype\": \"video\",\n        \"shape\": (HEIGHT, WIDTH, 3),\n        \"names\": [\"height\", \"width\", \"rgb\"],\n    },\n}\n\ndataset = LeRobotDataset.create(\n    repo_id=\"your-name/your-dataset\",\n    fps=SOURCE_FPS,\n    features=features,\n    # root=...,\n)\n\nfor episode in episodes:\n    for frame in episode:\n        dataset.add_frame(\n            {\n                \"observation.state\": state8,\n                \"action\": action7,\n                \"observation.images.image\": main_image,\n                \"observation.images.image2\": wrist_image,\n                \"task\": task_description,\n            }\n        )\n\n    dataset.save_episode()\n\ndataset.finalize()\n```\n\nThe exact camera keys and shapes should match **your** source data and the policy you intend to train.\n\nCurrent LeRobot’s writer automatically handles standard bookkeeping such as frame indices and timestamps; `add_frame()` computes timestamp from `frame_index / fps`. Also, with v3 you should call [` finalize()`](https://github.com/huggingface/lerobot/blob/main/docs/source/lerobot-dataset-v3.mdx) after recording, because it closes/flushed the incremental Parquet writers and finalizes metadata.\n\nSo I would not copy another converter’s `FEATURES` constant verbatim. Use it as a template, but derive:\n\nBefore converting everything, I would do only **one or two episodes**, call `finalize()`, reload the resulting `LeRobotDataset`, and inspect a few samples. That usually catches schema mistakes much more cheaply than converting the whole dataset first.\n\nFor example, verify at least:\n\n``` php\nobservation.state       -> (8,), float32\naction                  -> (7,), float32\nobservation.images.*    -> expected camera keys/shapes\ntimestamp               -> consistent with the source FPS\ntask                    -> present\n```\n\nIf you are targeting a particular pretrained LeRobot policy/checkpoint rather than training completely from scratch, I would additionally verify that its expected input feature **names as well as shapes** match your dataset.\n\nSo for your specific three questions, my current answer would be:\n\n**State mapping:** probably do **not** select eight elements from the 79-D simulator state. If `obs/ee_states` and `obs/gripper_states` exist, concatenating those is the most direct candidate. Otherwise reconstruct those observations from the raw EEF/gripper fields or from the simulator.\n\n**Features configuration:** define `features` according to the **converted semantic outputs**, e.g. an 8-D float `observation.state`, 7-D action, and your actual image features. Let the current LeRobot writer generate the corresponding v3 metadata rather than treating `meta/info.json` as the starting point.\n\n**Known pattern/script:** [`any4lerobot/libero2lerobot`](https://github.com/Tavish9/any4lerobot/tree/main/libero2lerobot) is probably the closest reference implementation for LIBERO-style HDF5 → LeRobot v3. I would compare its source mapping with your HDF5 keys, rather than just copying its `FEATURES` constant.\n\nIf your file already has `obs/ee_states` and `obs/gripper_states`, you may actually be quite close—the 79-D vector might not need to participate in the LeRobot policy state conversion at all.", "url": "https://wpnews.pro/news/converting-libero-hdf5-to-lerobot-format-how-to-map-79-dim-state-to-8-dim", "canonical_source": "https://discuss.huggingface.co/t/converting-libero-hdf5-to-lerobot-format-how-to-map-79-dim-state-to-8-dim/180039#post_2", "published_at": "2026-09-08 01:27:52+00:00", "updated_at": "2026-09-08 01:30:40.606716+00:00", "lang": "en", "topics": ["robotics", "machine-learning", "ai-research"], "entities": ["LeRobot", "LIBERO", "Hugging Face", "MuJoCo"], "alternates": {"html": "https://wpnews.pro/news/converting-libero-hdf5-to-lerobot-format-how-to-map-79-dim-state-to-8-dim", "markdown": "https://wpnews.pro/news/converting-libero-hdf5-to-lerobot-format-how-to-map-79-dim-state-to-8-dim.md", "text": "https://wpnews.pro/news/converting-libero-hdf5-to-lerobot-format-how-to-map-79-dim-state-to-8-dim.txt", "jsonld": "https://wpnews.pro/news/converting-libero-hdf5-to-lerobot-format-how-to-map-79-dim-state-to-8-dim.jsonld"}}