Building an Imitation Learning Pipeline for Robotic Manipulation

A developer detailed the construction of an end-to-end imitation learning pipeline for robotic manipulation, covering dataset loading, normalization, model architecture, and training. The tutorial emphasizes action chunking to reduce compounding error and the importance of state normalization, providing code examples for a PyTorch-based policy with a CNN vision encoder.

You've collected demonstrations — now it's time to turn them into a working policy. This tutorial walks through building an end-to-end imitation learning pipeline: dataset loading, model architecture, training loop, and evaluation, using the demonstration format from the previous tutorial. php Raw Episodes -- Dataset Loader -- Preprocessing -- Policy Model | v Training Loop | v Checkpoint -- Rollout/Eval Wrap your recorded episodes in a PyTorch Dataset that returns observation, action pairs, optionally as short action chunks rather than single steps — chunking predicting several future actions at once is standard in modern imitation learning and reduces compounding error. python import torch from torch.utils.data import Dataset import h5py import glob class DemoDataset Dataset : def init self, data dir, chunk size=8 : self.files = glob.glob f"{data dir}/ .hdf5" self.chunk size = chunk size self.index = self. build index def build index self : index = for file idx, f in enumerate self.files : with h5py.File f, "r" as h: length = h "actions" .shape 0 for t in range length - self.chunk size : index.append file idx, t return index def len self : return len self.index def getitem self, idx : file idx, t = self.index idx with h5py.File self.files file idx , "r" as h: image = h "obs/front rgb" t state = h "obs/joint positions" t action chunk = h "actions" t:t + self.chunk size return { "image": torch.from numpy image .permute 2, 0, 1 .float / 255.0, "state": torch.from numpy state .float , "action chunk": torch.from numpy action chunk .float , } Action and state normalization is one of the most impactful — and most skipped — steps. Compute statistics over your full dataset and normalize to zero mean / unit variance or a fixed range . python import numpy as np def compute normalization stats dataset : all actions = np.concatenate dataset i "action chunk" .numpy for i in range len dataset return { "action mean": all actions.mean axis=0 , "action std": all actions.std axis=0 + 1e-6, } def normalize action action, stats : return action - stats "action mean" / stats "action std" def denormalize action action, stats : return action stats "action std" + stats "action mean" Save these stats alongside your checkpoint — you'll need the exact same normalization at inference time. A common and effective baseline is a CNN vision encoder feeding into an MLP or transformer head that predicts an action chunk. Here's a simple version: python import torch.nn as nn class VisionEncoder nn.Module : def init self, out dim=256 : super . init self.net = nn.Sequential nn.Conv2d 3, 32, 5, stride=2 , nn.ReLU , nn.Conv2d 32, 64, 5, stride=2 , nn.ReLU , nn.Conv2d 64, 128, 3, stride=2 , nn.ReLU , nn.AdaptiveAvgPool2d 1 , self.fc = nn.Linear 128, out dim def forward self, x : x = self.net x .flatten 1 return self.fc x class Policy nn.Module : def init self, state dim, action dim, chunk size, hidden dim=256 : super . init self.vision = VisionEncoder out dim=hidden dim self.state proj = nn.Linear state dim, hidden dim self.head = nn.Sequential nn.Linear hidden dim 2, hidden dim , nn.ReLU , nn.Linear hidden dim, action dim chunk size , self.chunk size = chunk size self.action dim = action dim def forward self, image, state : v = self.vision image s = self.state proj state combined = torch.cat v, s , dim=-1 out = self.head combined return out.view -1, self.chunk size, self.action dim For more capable policies, consider swapping the head for a small transformer decoder or a diffusion head as in Diffusion Policy — but this MLP baseline is a good place to validate your whole pipeline before adding complexity. Standard behavior cloning uses MSE loss between predicted and demonstrated actions: python def train model, dataloader, optimizer, epochs=100, device="cuda" : model.to device model.train for epoch in range epochs : total loss = 0.0 for batch in dataloader: image = batch "image" .to device state = batch "state" .to device target = batch "action chunk" .to device pred = model image, state loss = nn.functional.mse loss pred, target optimizer.zero grad loss.backward optimizer.step total loss += loss.item print f"Epoch {epoch}: loss={total loss / len dataloader :.4f}" Loss curves only tell part of the story — the real test is closed-loop rollout on the robot or in simulation . Structure your eval loop to reuse the exact same normalization and observation pipeline as training: python def rollout env, model, stats, device="cuda", max steps=200 : obs = env.reset model.eval for step in range max steps : image = preprocess image obs "front rgb" .to device state = torch.from numpy obs "joint positions" .float .unsqueeze 0 .to device with torch.no grad : action chunk = model image, state 0 .cpu .numpy action = denormalize action action chunk 0 , stats obs, , done, info = env.step action if done: break return info.get "success", False Executing only the first action of a predicted chunk, then re-planning, is a common and robust strategy receding-horizon control — it limits drift from small model errors. | Symptom | Likely Cause | Fix | |---|---|---| | Training loss low, rollout fails | Distribution shift / compounding error | Add action chunking, DAgger-style correction data, or more diverse demos | | Policy freezes near objects | Visual overfitting to background | Augment with random crops/color jitter, add more scene variation | | Jerky robot motion | No temporal smoothing | Smooth action chunk execution, reduce control frequency mismatch | | Great on training tasks, bad on new positions | Insufficient position diversity in demos | Collect demos with randomized object placement | This baseline pipeline gets you to a working policy quickly. From here, natural extensions include diffusion-based action heads, transformer backbones e.g., ACT-style architectures , and multi-task training across several manipulation tasks. Website: www.v-modal.com https://www.v-modal.com SDK Flutter: https://github.com/v-modal/vmodal sdk flutter https://github.com/v-modal/vmodal sdk flutter SDK Android: https://github.com/v-modal/vmodal sdk android https://github.com/v-modal/vmodal sdk android Discord: https://discord.gg/K72z28KUx https://discord.gg/K72z28KUx