# Converting LIBERO HDF5 to LeRobot format: How to map 79-dim state to 8-dim? > Source: > Published: 2026-09-08 01:27:52+00:00 Hmm… I think you’d probably get a more definitive answer by asking in the LeRobot Discord (linked from [lerobot (LeRobot)](https://huggingface.co/lerobot)), but for now: I think the main distinction here is that the 79-D `states` vector and LeRobot’s 8-D `observation.state` are probably **two different kinds of state**, rather than the latter being a subset or dimensionality reduction of the former. In the standard LIBERO data path, the HDF5 `states` array is used as a flattened MuJoCo simulator state. LIBERO’s own [`create_dataset.py`](https://github.com/Lifelong-Robot-Learning/LIBERO/blob/master/scripts/create_dataset.py) restores that simulator state, replays the trajectory, and separately constructs proprioceptive observations such as: ``` ee_state = np.hstack( ( obs["robot0_eef_pos"], # 3 T.quat2axisangle(obs["robot0_eef_quat"]), # 3 ) ) gripper_state = obs["robot0_gripper_qpos"] # 2 ``` That gives: ``` 3 (EEF position) + 3 (EEF orientation as axis-angle) + 2 (gripper qpos) = 8 ``` This is also the representation currently documented by [LeRobot’s LIBERO integration](https://huggingface.co/docs/lerobot/main/libero): `observation.state` is 8-D EEF position + axis-angle orientation + gripper qpos. So, **if your custom HDF5 follows the usual LIBERO layout, I would not start by selecting eight indices out of the 79-D vector.** I would first check whether the HDF5 already contains the proprio fields used to construct the 8-D state. A practical decision tree would be: ``` Does demo_0/obs contain ee_states (6) and gripper_states (2)? │ ├─ Yes │ └─ observation.state = concat(ee_states, gripper_states) │ └─ No │ ├─ Do you have eef_pos + eef_quat + gripper_qpos? │ └─ Build 3 + quat→axis-angle(3) + 2 │ └─ Only the simulator states are available │ ├─ Can the LIBERO environment be reconstructed? │ └─ Restore each simulator state and regenerate observations │ └─ The 79-D layout is custom / unknown └─ Check the exporter/schema first; don't guess fixed indices ``` The [`any4lerobot/libero2lerobot`](https://github.com/Tavish9/any4lerobot/tree/main/libero2lerobot) converter is a useful concrete example here: its LeRobot state is 8-D, while it also preserves richer native LIBERO state information separately. In other words, making a training-compatible `observation.state` does **not** require throwing the original simulator/robot state away. `features` / `meta/info.json` I would treat the feature schema as the **output contract of the conversion**, not as the thing that determines how the 79-D source vector should be interpreted. Conceptually: ``` source HDF5 ↓ interpret source semantics ↓ construct observation.state / action / images ↓ declare those outputs in LeRobot `features` ↓ write the LeRobot dataset ``` With current LeRobot v3, I would normally define the semantic features when creating the dataset rather than manually constructing the whole `meta/info.json` first. A simplified skeleton is roughly: ``` features = { "observation.state": { "dtype": "float32", "shape": (8,), "names": { "motors": [ "x", "y", "z", "axis_angle1", "axis_angle2", "axis_angle3", "gripper_left", "gripper_right", ] }, }, "action": { "dtype": "float32", "shape": (7,), # Add names that match your actual action semantics. }, "observation.images.image": { "dtype": "video", # or "image", depending on your choice "shape": (HEIGHT, WIDTH, 3), "names": ["height", "width", "rgb"], }, "observation.images.image2": { "dtype": "video", "shape": (HEIGHT, WIDTH, 3), "names": ["height", "width", "rgb"], }, } dataset = LeRobotDataset.create( repo_id="your-name/your-dataset", fps=SOURCE_FPS, features=features, # root=..., ) for episode in episodes: for frame in episode: dataset.add_frame( { "observation.state": state8, "action": action7, "observation.images.image": main_image, "observation.images.image2": wrist_image, "task": task_description, } ) dataset.save_episode() dataset.finalize() ``` The exact camera keys and shapes should match **your** source data and the policy you intend to train. Current LeRobot’s writer automatically handles standard bookkeeping such as frame indices and timestamps; `add_frame()` computes timestamp from `frame_index / fps`. Also, with v3 you should call [` finalize()`](https://github.com/huggingface/lerobot/blob/main/docs/source/lerobot-dataset-v3.mdx) after recording, because it closes/flushed the incremental Parquet writers and finalizes metadata. So I would not copy another converter’s `FEATURES` constant verbatim. Use it as a template, but derive: Before converting everything, I would do only **one or two episodes**, call `finalize()`, reload the resulting `LeRobotDataset`, and inspect a few samples. That usually catches schema mistakes much more cheaply than converting the whole dataset first. For example, verify at least: ``` php observation.state -> (8,), float32 action -> (7,), float32 observation.images.* -> expected camera keys/shapes timestamp -> consistent with the source FPS task -> present ``` If you are targeting a particular pretrained LeRobot policy/checkpoint rather than training completely from scratch, I would additionally verify that its expected input feature **names as well as shapes** match your dataset. So for your specific three questions, my current answer would be: **State mapping:** probably do **not** select eight elements from the 79-D simulator state. If `obs/ee_states` and `obs/gripper_states` exist, concatenating those is the most direct candidate. Otherwise reconstruct those observations from the raw EEF/gripper fields or from the simulator. **Features configuration:** define `features` according to the **converted semantic outputs**, e.g. an 8-D float `observation.state`, 7-D action, and your actual image features. Let the current LeRobot writer generate the corresponding v3 metadata rather than treating `meta/info.json` as the starting point. **Known pattern/script:** [`any4lerobot/libero2lerobot`](https://github.com/Tavish9/any4lerobot/tree/main/libero2lerobot) is probably the closest reference implementation for LIBERO-style HDF5 → LeRobot v3. I would compare its source mapping with your HDF5 keys, rather than just copying its `FEATURES` constant. If your file already has `obs/ee_states` and `obs/gripper_states`, you may actually be quite close—the 79-D vector might not need to participate in the LeRobot policy state conversion at all.