From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot AWS has released Strands Robots, an open-source SDK that integrates Hugging Face's LeRobot stack into a single agent loop, enabling users to go from a Hub dataset to a physical robot with minimal code changes. The SDK provides AgentTools for recording demonstrations, running policies, and deploying across fleets, supporting simulation and hardware interchangeably. This simplifies the robotics workflow by unifying tools that previously required separate software. Robotics • 5B • Updated • 1.39k • 14 From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot Enterprise Article /blog A walkthrough of the LeRobot integration in Strands Robots - one agent loop, from a Hub dataset to a physical robot, with sim-to-real datasets in the same on-disk format and policies you swap with a string. You have a robot, a folder of demonstration data on the Hugging Face Hub https://huggingface.co , and a new task you want it to learn. Today that takes five separate tools: one to record new demonstrations, another to train, a third to test in simulation, custom code to deploy on hardware, and yet another to coordinate when you have more than one robot. The pieces work on their own. They don't talk to each other. Strands Robots https://github.com/strands-labs/robots is an open source SDK from AWS Apache 2.0 https://www.apache.org/licenses/LICENSE-2.0 that exposes robot abstractions, simulation, and the LeRobot https://github.com/huggingface/lerobot stack as AgentTools that you compose into a single Strands agent. The integration is deliberately thin: LeRobot's own scripts handle hardware recording and calibration, and the Strands AgentTools come in for the parts an agent actually orchestrates. The simulation tool records LeRobotDatasets in the same format LeRobot writes on hardware. GR00T https://github.com/NVIDIA/Isaac-GR00T and LerobotLocal https://strands-labs.github.io/robots/policies/lerobot-local/ serve policy inference behind a common interface, and MolmoAct2 checkpoints run through the LerobotLocal path. A peer mesh fans the agent out to remote robots. The dataset format stays exactly as LeRobot wrote it; the agent loop is the glue. This post walks you through five steps inside a single agent: build the agent over the LeRobot AgentTools, record a demonstration as a LeRobotDataset in simulation, run a policy on the same robot, deploy the same agent code to a physical SO-101 https://github.com/TheRobotStudio/SO-ARM100 with one keyword argument change, and broadcast commands across a fleet over the Zenoh https://zenoh.io mesh. At the end, you can clone the working sample application from GitHub and run it on your laptop in simulation. No hardware, no GPU, no Hugging Face credentials needed for the default path. The runnable companion to this post lives at examples/lerobot/hub to hardware.py https://github.com/strands-labs/robots/blob/main/examples/lerobot/hub to hardware.py and . The notebook is sim-only and Mock-policy by default. https://github.com/strands-labs/robots/blob/main/examples/lerobot/hub to hardware.ipynb hub to hardware.ipynb What you'll build The Strands Robots SDK exposes the LeRobot stack as AgentTools that you compose into one Strands agent. The example agent in this post does four things: record new demonstrations in simulation, push the result to the Hub as a LeRobotDataset, run a policy in simulation against that same format, and deploy the same agent code to a physical robot with one keyword argument change. When you have more than one robot, the agent can coordinate the whole fleet through a built-in peer mesh. For hardware recording and calibration, LeRobot's own CLIs lerobot-record , lerobot-calibrate handle the bring-up; the agent picks up from there. Figure 1. Robot "so100" defaults to a MuJoCo-backed simulation; mode="real" returns a hardware robot driven by LeRobot. Both modes share the same DatasetRecorder and the same policy providers, so a dataset captured in sim and a dataset captured on hardware use the same on-disk LeRobotDataset format. Two design choices make this work. First, Robot "so100" https://strands-labs.github.io/robots/getting-started/robot-factory/ returns a simulation by default no hardware, no risk , and mode="real" returns a hardware-backed robot driven by LeRobot. The agent code is identical across both modes. Second, the DatasetRecorder that writes a LeRobotDataset is shared between the simulation path https://strands-labs.github.io/robots/simulation/overview/ and LeRobot's own hardware recording, so a dataset captured in MuJoCo https://mujoco.org and one captured from a physical SO-101 https://github.com/TheRobotStudio/SO-ARM100 are in the same format. The whole workflow in five lines of Python: python from strands robots import Robot from strands import Agent arm = Robot "so100" mode="sim" default - safe, no hardware agent = Agent tools= arm agent "Pick up the red cube" What follows is what's actually happening inside that call, step by step. Prerequisites Minimal default simulation path - Python 3.12+, on Linux or macOS Apple Silicon supported for the MuJoCo backend . - A Strands-compatible model provider for the agent's reasoning. Amazon Bedrock https://aws.amazon.com/bedrock/ with AWS credentials, the Anthropic API https://docs.anthropic.com , OpenAI, or Ollama https://ollama.com running locally. - Strands Robots installed with the install extras https://strands-labs.github.io/robots/getting-started/installation/ : uv pip install "strands-robots sim-mujoco,lerobot,mesh " That's it. The example in this post runs end-to-end on a laptop with these three. Advanced hardware deployment, real policies, Hub push - A Hugging Face account and token with write permission, for pushing datasets and pulling policy checkpoints from the Hub. - For the hardware path: an SO-101 follower and leader pair, or any other LeRobot-supported robot. Both devices need calibration files under ~/.cache/huggingface/lerobot/calibration/ . - For local GR00T inference: an NVIDIA GPU with at least 16 GB of video memory and Docker installed. The post uses the gr00t inference tool's lifecycle="full" action, which pulls the image, downloads a checkpoint, and starts the container in one call. Step 1 - Set up the example Install Strands Robots and get the example files: uv pip install "strands-robots sim-mujoco,lerobot,mesh " git clone https://github.com/strands-labs/robots.git cd robots Export your Hugging Face token if you want the agent to push datasets or pull policies from the Hub. This is optional for the default simulation path in this post; the example runs end-to-end with the Mock policy and writes the dataset to your local cache without needing Hub access. export HF TOKEN=hf ... The runnable example lives at examples/lerobot/hub to hardware.py Python script and hub to hardware.ipynb notebook , in the strands-labs/robots repository alongside the MuJoCo and LIBERO examples. The notebook is the recommended starting point: open it in JupyterLab https://jupyterlab.readthedocs.io and run cells top-to-bottom in simulation mode without any hardware connected. Step 2 - Record demonstrations and push to the Hub The simulation tool records LeRobotDatasets https://strands-labs.github.io/robots/recording/ in the same format LeRobot writes on hardware. No hardware required. The Simulation tool's start recording action writes through the same DatasetRecorder class: same parquet schema for joint states and actions, same per-camera MP4 layout. The agent prompt is almost identical: python from strands import Agent from strands robots import Robot robot = Robot "so100" mode="sim" by default agent = Agent tools= robot agent "Record a demonstration of 'pick the red cube and place it in the box' " "using the Mock policy provider at FPS 30. Write the dataset to " "my user/cube picking sim and push to the Hub when done." Figure 2. The recording scene in MuJoCo simulation: the SO-100 arm reaching toward a red cube on the ground plane, captured to a LeRobotDataset. No hardware, no GPU, no Hugging Face credentials needed for this default path. The Mock policy is intentional: it generates placeholder joint actions so the workflow runs end-to-end without a trained checkpoint. The robot moves through random motions rather than completing the grasp, and the recording is structurally complete valid joint states, valid camera frames, a well-formed LeRobotDataset episode , but the demonstration itself isn't useful as training data. Step 3 below swaps in GR00T or LerobotLocal for real grasping behavior. To see actual cube-picking in this step, run --policy lerobot local --checkpoint allenai/MolmoAct2-SO100 101 a MolmoAct2 checkpoint https://huggingface.co/allenai/MolmoAct2-SO100 101 , auto-detected from its config.json and routed through the LerobotLocal path ; the prompt, dataset format, and agent code stay the same. The proof is what happens next. LeRobot's own dataset loader reads the sim-recorded data with no Strands-specific code path: python from lerobot.datasets.lerobot dataset import LeRobotDataset dataset = LeRobotDataset "my user/cube picking sim" print dataset.features {'observation.state': Sequence ... , 'observation.images.front': VideoFrame ... , 'action': Sequence ... , 'episode index': Value ... , 'frame index': Value ... , ...} This features dict is identical in shape to any LeRobot dataset on the Hub: same column names, same parquet+MP4 layout, same loader path. Training scripts that consume hardware-recorded data consume the sim-recorded data without modification. Datasets pushed from sim sit alongside hardware recordings in the same Hub repository if you want them to. A single episode from a recorded LeRobotDataset, played back from the per-camera MP4 the recorder wrote, the same on-disk video a training script reads. Recording on hardware To record demonstrations on a physical SO-101 instead of simulation, use LeRobot's record CLI directly. The Strands integration doesn't wrap that command as an AgentTool because LeRobot already does the job cleanly: lerobot-calibrate --robot.type=so101 follower --robot.id=my follower lerobot-calibrate --robot.type=so101 leader --robot.id=my leader lerobot-record \ --robot.type=so101 follower --robot.id=my follower \ --teleop.type=so101 leader --teleop.id=my leader \ --dataset.repo id=my user/cube picking \ --dataset.single task='Pick up the red cube and place it in the box' \ --dataset.num episodes=25 \ --dataset.push to hub=true The dataset that lands on the Hub from this command is in the same format as the simulation recording. To fine-tune a policy on it, run LeRobot's training CLI lerobot-train ; training itself is out of scope for this post and follows the standard LeRobot workflow. From Step 3 onward, the agent picks up either the original or a fine-tuned checkpoint interchangeably. For full SO-101 hardware setup, calibration walkthroughs, and troubleshooting, see the README in the example folder https://github.com/strands-labs/robots/blob/main/examples/lerobot/README.md . Step 3 - Run a policy in simulation With the dataset on the Hub, the next step is to run a policy. The example uses the Robot factory in its default sim mode, then attaches gr00t inference so the agent can manage the inference container: python from strands import Agent from strands robots import Robot, gr00t inference robot = Robot "so100" mode="sim" by default agent = Agent tools= robot, gr00t inference agent "Start GR00T inference on port 5555 with the cube-picking checkpoint " "from my user/cube-picker. Then ask the robot to pick up the red cube." Under the hood, the agent runs gr00t inference action="lifecycle", lifecycle="full", ... to pull the GR00T container image https://strands-labs.github.io/robots/policies/groot/ , download the checkpoint from the Hub, and start the inference service. It then runs a run policy action on the simulated robot with policy provider="groot" , passing the GR00T service's host and port in the policy config dict the container is reachable on port 5555 . The simulation steps with the policy's action chunks, and a render of the result is available via Simulation.render . Figure 3. With a trained policy a GR00T or MolmoAct2 checkpoint , the agent drives the SO-100 to grasp the red cube in simulation, the behavior the Mock policy stands in for. For developers who prefer in-process inference no container, no ZeroMQ ZMQ , swap gr00t inference for a LerobotLocalPolicy instance loaded from a Hub repository. The provider routes any model ID under the lerobot/ organization to the in-process path: python from strands robots.policies import create policy policy = create policy "lerobot/act aloha sim transfer cube human" LerobotLocalPolicy supports ACT https://tonyzhaozh.github.io/aloha/ , Diffusion Policy https://diffusion-policy.cs.columbia.edu , SmolVLA https://huggingface.co/blog/smolvla , π0 https://www.pi.website/blog/pi0 , and π0.5 https://www.pi.website/blog/pi05 , anything LeRobot's own policy registry can resolve from a config.json . Real-Time Chunking https://www.pi.website/research/real time chunking turns on automatically for flow-matching policies that ship an rtc config π0, SmolVLA . NVIDIA's recently released Cosmos 3 https://huggingface.co/nvidia/Cosmos3-Nano is also available as a policy provider behind the same interface, so the agent code stays the same whichever provider you point it at. Note: LerobotLocalPolicy loads Hugging Face models with trust remote code=True. Set STRANDS TRUST REMOTE CODE=1 to opt in, and only load checkpoints from organizations you trust. Step 4 - Deploy the policy to physical hardware This is the same code as Step 3, with one keyword argument changed. The Robot factory returns a hardware-backed robot driven by LeRobot's make robot from config : robot = Robot "so100", mode="real", port="/dev/ttyACM0", data config="so100 dualcam", cameras={ "front": {"type": "opencv", "index or path": "/dev/video0", "fps": 30}, "wrist": {"type": "opencv", "index or path": "/dev/video2", "fps": 30}, }, agent = Agent tools= robot, gr00t inference agent "Start GR00T inference on port 5555 with the cube-picking checkpoint " "from my user/cube-picker. Then ask the robot to pick up the red cube." The same agent prompt now runs against a physical arm. The hardware path uses LeRobot's robot abstraction for joint commands and camera reads, and the GR00T container reachable on port 5555 generates the action chunks. Before this runs against your SO-101, calibration for both follower and leader has to be in place. Run LeRobot's calibration command lerobot-calibrate once per device; the files land under ~/.cache/huggingface/lerobot/calibration/ and any Strands code path that touches the hardware reads them from there. If a calibration is missing, the agent surfaces the error from the LeRobot driver layer. Step 5 - Coordinate multiple robots with the mesh Up to now we've driven one robot at a time. The mesh https://strands-labs.github.io/robots/mesh/ is how Strands Robots handles more than one. Picture a leader arm on your desk teleoperating a follower arm in another room, or five SO-101s running the same warehouse task in parallel, or a humanoid coordinating with a mobile base. All of those are mesh patterns. The mesh is built on Zenoh, an open source peer-to-peer protocol, and you don't manage IP addresses, write discovery code, or pick a broker; new robots show up on the mesh the moment they come up, and the agent can talk to all of them at once. Every Robot and every Simulation joins a Zenoh peer mesh automatically. The robot mesh tool gives the agent a vocabulary for fleet operations such as discovery, structured commands, broadcasts, and emergency stop: agent = Agent tools= robot mesh agent "List every robot and simulation on the mesh. " "Then send 'go to home pose' to each one in parallel." The agent calls robot mesh action="peers" to enumerate locals and discovered peers, then robot mesh action="broadcast", ... to send the structured command to every peer with a timeout. Add the mesh-iot extra to route this traffic over AWS IoT Core for cross-network fleets. The robot mesh tool's action reference in the project documentation covers the full vocabulary: subscribe, watch, inbox, and structured peer-to-peer commands. By default, every physically-actuating mesh action pauses for a human approval interrupt before it runs: the fleet-wide broadcast and emergency stop, plus the single-peer tell, send, and stop. You can tune this set with the STRANDS MESH HITL ACTIONS environment variable set it to all, none, or a comma-separated subset . The first time you run this example, you'll see a robot mesh-broadcast-approval prompt in your terminal; type y or yes / approve to authorize the broadcast. The approval is delivered out-of-band of the LLM's tool arguments, so a prompt-injection attempt that tries to slip an approval flag into the command body cannot bypass the gate. The transport scales without touching agent code. The built-in Zenoh mesh is the automatic fallback: on the LAN, Zenoh multicast handles peer discovery with no broker, and adding the mesh-iot extra routes traffic through AWS IoT Core https://aws.amazon.com/iot-core/ MQTT5 with mTLS for cloud fleets, with a BridgeTransport that fans LAN and cloud behind one API select it with STRANDS MESH BACKEND=bridge . For production fleets, Device Connect, a device-aware networking layer developed in collaboration with Arm, handles discovery, presence, structured RPC, event routing, and safety. The same robot mesh tool dispatches through Device Connect when it is available and falls back to the built-in Zenoh mesh otherwise, so the agent code in this post is unchanged either way. See the Device Connect documentation https://strands-labs.github.io/robots/device-connect/ for setup and current availability. Try it using the sample application The full sample is on GitHub at strands-labs/robots https://github.com/strands-labs/robots in the examples/lerobot/ folder. It packages all five steps into a single CLI script hub to hardware.py and a notebook hub to hardware.ipynb . The CLI defaults run end-to-end in simulation with the Mock policy. No GPU, no Docker, no Hugging Face credentials needed. uv pip install "strands-robots sim-mujoco,lerobot,mesh " git clone https://github.com/strands-labs/robots.git cd robots export STRANDS MESH LOCAL DEV=1 python examples/lerobot/hub to hardware.py The recorded dataset lands at ~/.cache/huggingface/lerobot/local/strands-cube-pick/ . To push to the Hugging Face Hub instead of keeping it local, pass --hf-user