Train a Dual-Arm Nero Reach Task in Isaac Lab

NVIDIA Isaac Lab now supports training reinforcement learning policies for the AgileX Dual-Arm Nero Manipulator, extending the SO-ARM101 implementation. The open-source project provides robot configuration files, URDF mesh conversion, and training scripts to enable dual-arm robotic manipulation tasks.

Robotic manipulation remains one of the most important research directions in embodied AI. While traditional kinematics-based controllers provide stable motion execution, they often struggle in unstructured environments where adaptability is required. Recent advances in Reinforcement Learning RL have enabled robotic arms to learn task-oriented behaviors directly from interaction, making it possible to achieve robust control policies without manually designing every motion strategy. In this project, we extend the original SO-ARM101 Isaac Lab implementation by integrating the AgileX Dual-Arm Nero Manipulator, allowing developers to quickly train and validate dual-arm RL policies using NVIDIA Isaac Lab. ReposityOpen-source implementation: https://github.com/smalleha/isaac so arm101.git https://github.com/smalleha/isaac so arm101.git ├── robots│ ├── dual nero│ │ ├── dual nero.py│ │ ├── init .py│ │ ├── meshes│ │ └── urdf│ │ └── dual nero.urdf├── scripts│ ├── rl games│ │ ├── play.py│ │ └── train.py│ └── zero agent.py├── tasks│ ├── init .py│ └── reach│ ├── agents│ ├── dual nero joint pos env cfg.py│ ├── dual nero reach env cfg.py│ ├── init .py│ └── mdp└── ui extension example.py Key additions include: Before importing the robot into Isaac Lab, mesh references inside the URDF should be converted to relative paths. For example: <link name="base link" <inertial <origin rpy="0 0 0" xyz="-0.0592395620981769 -0.068642440505388 0.0562764736144042"/ <mass value="4.46524857458863"/ <inertia ixx="0.0608289280191989" ixy="-2.54649959130438E-06" ixz="1.11851948046933E-07" iyy="0.0218722514454004" iyz="-0.000689477252402357" izz="0.0680524540174318"/ </inertial <visual <origin rpy="0 0 0" xyz="0 0 0"/ <geometry <mesh filename="../meshes/base link.dae"/ </geometry <material name="" <color rgba="0.776470588235294 0.756862745098039 0.737254901960784 1"/ </material </visual <collision <origin rpy="0 0 0" xyz="0 0 0"/ <geometry <mesh filename="../meshes/base link.dae"/ </geometry </collision </link This ensures that the asset loader can correctly locate mesh resources during simulation. 3.2 Creating the Robot ConfigurationNext, create a robot configuration file: isaac so arm101/robots/dual nero/dual nero.py This file defines: - Robot articulation properties - Joint stiffness and damping - Actuator configuration - Initial joint states - Gripper settings The resulting DUAL NERO CFG object becomes the robot asset used by Isaac Lab during training. python from pathlib import Pathimport isaaclab.sim as sim utilsfrom isaaclab.actuators import ImplicitActuatorCfgfrom isaaclab.assets.articulation import ArticulationCfgTEMPLATE ASSETS DATA DIR = Path file .resolve .parentDUAL NERO CFG = ArticulationCfg spawn=sim utils.UrdfFileCfg fix base=True, merge fixed joints=False, replace cylinders with capsules=True, asset path=f"{TEMPLATE ASSETS DATA DIR}/urdf/dual nero.urdf", activate contact sensors=False, set as false while waiting for capsule implementation rigid props=sim utils.RigidBodyPropertiesCfg disable gravity=False, max depenetration velocity=5.0, , articulation props=sim utils.ArticulationRootPropertiesCfg enabled self collisions=True, solver position iteration count=8, solver velocity iteration count=0, , joint drive=sim utils.UrdfConverterCfg.JointDriveCfg gains=sim utils.UrdfConverterCfg.JointDriveCfg.PDGainsCfg stiffness=0, damping=0 , , init state=ArticulationCfg.InitialStateCfg rot= 1.0, 0.0, 0.0, 0.0 , joint pos={ "left joint. ": 0.0, "right joint. ": 0.0, "left gripper joint. ": 0.0, "right gripper joint. ": 0.0, }, Set initial joint velocities to zero joint vel={". ": 0.0}, , actuators={ "arm": ImplicitActuatorCfg joint names expr= "left joint. ", "right joint. " , effort limit=25.0, velocity limit=1.5, stiffness={ "left joint1": 200.0, "left joint2": 170.0, "left joint3": 120.0, "left joint4": 80.0, "left joint5": 50.0, "left joint6": 20.0, "left joint7": 10.0, "right joint1": 200.0, "right joint2": 170.0, "right joint3": 120.0, "right joint4": 80.0, "right joint5": 50.0, "right joint6": 20.0, "right joint7": 10.0 }, damping={ "left joint1": 100.0, "left joint2": 60.0, "left joint3": 70.0, "left joint4": 24.0, "left joint5": 20.0, "left joint6": 10.0, "left joint7": 5, "right joint1": 100.0, "right joint2": 60.0, "right joint3": 70.0, "right joint4": 24.0, "right joint5": 20.0, "right joint6": 10.0, "right joint7": 5, }, , "gripper": ImplicitActuatorCfg joint names expr= "left gripper joint. ","right gripper joint. " , effort limit sim=22, Increased from 1.9 to 2.5 for stronger grip velocity limit sim=1.5, stiffness=800.0, Increased from 25.0 to 60.0 for more reliable closing damping=20.0, Increased from 10.0 to 20.0 for stability , }, soft joint pos limit factor=0.9, Create an init .py file inside the dual nero directory so that Python recognizes it as a package. Two environment configuration files are required: tasks/reach/├── dual nero joint pos env cfg.py└── dual nero reach env cfg.py 4.1 Joint Position Environment Configuration dual nero joint pos env cfg.py specifies: - Controlled joints - End-effector links - Action spaces - Command targets python import mathimport isaaclab tasks.manager based.manipulation.reach.mdp as mdpfrom isaaclab.utils import configclassfrom isaac so arm101.robots import DUAL NERO CFG noqa: F401 from isaac so arm101.tasks.reach.dual nero reach env cfg import Dual NeroReachEnvCfgfrom isaaclab.assets.articulation import ArticulationCfg@configclassclass Dual Nero ReachEnvCfg Dual NeroReachEnvCfg : def post init self : post init of parent super . post init switch robot to OpenArm self.scene.robot = DUAL NERO CFG.replace prim path="{ENV REGEX NS}/Robot", override rewards self.rewards.left end effector position tracking.params "asset cfg" .body names = "left gripper base" self.rewards.left end effector position tracking fine grained.params "asset cfg" .body names = "left gripper base" self.rewards.left end effector orientation tracking.params "asset cfg" .body names = "left gripper base" self.rewards.right end effector position tracking.params "asset cfg" .body names = "right gripper base" self.rewards.right end effector position tracking fine grained.params "asset cfg" .body names = "right gripper base" self.rewards.right end effector orientation tracking.params "asset cfg" .body names = "right gripper base" override actions self.actions.left arm action = mdp.JointPositionActionCfg asset name="robot", joint names= "left joint. ", , scale=0.5, use default offset=True, self.actions.right arm action = mdp.JointPositionActionCfg asset name="robot", joint names= "right joint. ", , scale=0.5, use default offset=True, override command generator body end-effector is along z-direction self.commands.left ee pose.body name = "left gripper base" self.commands.right ee pose.body name = "right gripper base"@configclass class Dual Nero ReachEnvCfg PLAY Dual Nero ReachEnvCfg : def post init self : post init of parent super . post init make a smaller scene for play self.scene.num envs = 50 self.scene.env spacing = 2.5 disable randomization for play self.observations.policy.enable corruption = False 4.2 Reach Task Definition dual nero reach env cfg.py contains the full RL environment definition. This includes: 1.Scene Configuration - Ground plane - Lighting - Robot asset 2.Command Generation 3.Observation Space 4.Reward Design 5.Curriculum Learning 6.Episode Settings python import mathfrom dataclasses import MISSINGimport isaaclab.sim as sim utilsfrom isaaclab.assets import ArticulationCfg, AssetBaseCfgfrom isaaclab.envs import ManagerBasedRLEnvCfgfrom isaaclab.managers import ActionTermCfg as ActionTermfrom isaaclab.managers import CurriculumTermCfg as CurrTermfrom isaaclab.managers import EventTermCfg as EventTermfrom isaaclab.managers import ObservationGroupCfg as ObsGroupfrom isaaclab.managers import ObservationTermCfg as ObsTermfrom isaaclab.managers import RewardTermCfg as RewTermfrom isaaclab.managers import SceneEntityCfgfrom isaaclab.managers import TerminationTermCfg as DoneTermfrom isaaclab.scene import InteractiveSceneCfgfrom isaaclab.utils import configclassfrom isaaclab.utils.noise import AdditiveUniformNoiseCfg as Unoiseimport isaaclab tasks.manager based.manipulation.reach.mdp as mdp Scene definition @configclassclass Dual NeroReachSceneCfg InteractiveSceneCfg : """Configuration for the scene with a robotic arm.""" world ground = AssetBaseCfg prim path="/World/ground", spawn=sim utils.GroundPlaneCfg , init state=AssetBaseCfg.InitialStateCfg pos= 0.0, 0.0, 0 , robots robot: ArticulationCfg = MISSING lights light = AssetBaseCfg prim path="/World/light", spawn=sim utils.DomeLightCfg color= 0.75, 0.75, 0.75 , intensity=2500.0 , MDP settings @configclassclass CommandsCfg: """Command terms for the MDP.""" left ee pose = mdp.UniformPoseCommandCfg asset name="robot", body name=MISSING, resampling time range= 4.0, 4.0 , debug vis=True, ranges=mdp.UniformPoseCommandCfg.Ranges pos x= 0.5, 0.5 , pos y= 0.15, 0.25 , pos z= 0.3, 0.5 , roll= -math.pi / 6, math.pi / 6 , pitch= 3 math.pi / 2, 3 math.pi / 2 , yaw= 8 math.pi / 9, 10 math.pi / 9 , , right ee pose = mdp.UniformPoseCommandCfg asset name="robot", body name=MISSING, resampling time range= 4.0, 4.0 , debug vis=True, ranges=mdp.UniformPoseCommandCfg.Ranges pos x= 0.5, 0.5 , pos y= -0.25, -0.15 , pos z= 0.3, 0.5 , roll= -math.pi / 6, math.pi / 6 , pitch= 3 math.pi / 2, 3 math.pi / 2 , yaw= 8 math.pi / 9, 10 math.pi / 9 , , @configclassclass ActionsCfg: """Action specifications for the MDP.""" left arm action: ActionTerm = MISSING right arm action: ActionTerm = MISSING@configclassclass ObservationsCfg: """Observation specifications for the MDP.""" @configclass class PolicyCfg ObsGroup : """Observations for policy group.""" observation terms order preserved left joint pos = ObsTerm func=mdp.joint pos rel, params={ "asset cfg": SceneEntityCfg "robot", joint names= "left joint. ", , }, noise=Unoise n min=-0.01, n max=0.01 , right joint pos = ObsTerm func=mdp.joint pos rel, params={ "asset cfg": SceneEntityCfg "robot", joint names= "right joint. ", , }, noise=Unoise n min=-0.01, n max=0.01 , left joint vel = ObsTerm func=mdp.joint vel rel, params={ "asset cfg": SceneEntityCfg "robot", joint names= "left joint. ", , }, noise=Unoise n min=-0.01, n max=0.01 , right joint vel = ObsTerm func=mdp.joint vel rel, params={ "asset cfg": SceneEntityCfg "robot", joint names= "right joint. ", , }, noise=Unoise n min=-0.01, n max=0.01 , left pose command = ObsTerm func=mdp.generated commands, params={"command name": "left ee pose"} right pose command = ObsTerm func=mdp.generated commands, params={"command name": "right ee pose"} left actions = ObsTerm func=mdp.last action, params={"action name": "left arm action"} right actions = ObsTerm func=mdp.last action, params={"action name": "right arm action"} def post init self : self.enable corruption = True self.concatenate terms = True observation groups policy: PolicyCfg = PolicyCfg @configclassclass EventCfg: """Configuration for events.""" reset robot joints = EventTerm func=mdp.reset joints by scale, mode="reset", params={ "position range": 0.5, 1.5 , "velocity range": 0.0, 0.0 , }, @configclassclass RewardsCfg: """Reward terms for the MDP.""" left end effector position tracking = RewTerm func=mdp.position command error, weight=-0.2, params={ "asset cfg": SceneEntityCfg "robot", body names=MISSING , "command name": "left ee pose", }, right end effector position tracking = RewTerm func=mdp.position command error, weight=-0.25, params={ "asset cfg": SceneEntityCfg "robot", body names=MISSING , "command name": "right ee pose", }, left end effector position tracking fine grained = RewTerm func=mdp.position command error tanh, weight=0.1, params={ "asset cfg": SceneEntityCfg "robot", body names=MISSING , "std": 0.1, "command name": "left ee pose", }, right end effector position tracking fine grained = RewTerm func=mdp.position command error tanh, weight=0.2, params={ "asset cfg": SceneEntityCfg "robot", body names=MISSING , "std": 0.1, "command name": "right ee pose", }, left end effector orientation tracking = RewTerm func=mdp.orientation command error, weight=-0.1, params={ "asset cfg": SceneEntityCfg "robot", body names=MISSING , "command name": "left ee pose", }, right end effector orientation tracking = RewTerm func=mdp.orientation command error, weight=-0.1, params={ "asset cfg": SceneEntityCfg "robot", body names=MISSING , "command name": "right ee pose", }, action penalty action rate = RewTerm func=mdp.action rate l2, weight=-0.0001 left joint vel = RewTerm func=mdp.joint vel l2, weight=-0.0001, params={ "asset cfg": SceneEntityCfg "robot", joint names= "left joint. ", , }, right joint vel = RewTerm func=mdp.joint vel l2, weight=-0.0001, params={ "asset cfg": SceneEntityCfg "robot", joint names= "right joint. ", , }, @configclassclass TerminationsCfg: """Termination terms for the MDP.""" time out = DoneTerm func=mdp.time out, time out=True @configclassclass CurriculumCfg: """Curriculum terms for the MDP.""" action rate = CurrTerm func=mdp.modify reward weight, params={"term name": "action rate", "weight": -0.005, "num steps": 4500}, left joint vel = CurrTerm func=mdp.modify reward weight, params={"term name": "left joint vel", "weight": -0.001, "num steps": 4500}, right joint vel = CurrTerm func=mdp.modify reward weight, params={"term name": "right joint vel", "weight": -0.001, "num steps": 4500}, Environment configuration @configclassclass Dual NeroReachEnvCfg ManagerBasedRLEnvCfg : """Configuration for the reach end-effector pose tracking environment.""" Scene settings scene: Dual NeroReachSceneCfg = Dual NeroReachSceneCfg num envs=4096, env spacing=2.5 Basic settings observations: ObservationsCfg = ObservationsCfg actions: ActionsCfg = ActionsCfg commands: CommandsCfg = CommandsCfg MDP settings rewards: RewardsCfg = RewardsCfg terminations: TerminationsCfg = TerminationsCfg events: EventCfg = EventCfg curriculum: CurriculumCfg = CurriculumCfg def post init self : """Post initialization.""" general settings self.decimation = 2 self.sim.render interval = self.decimation self.episode length s = 24.0 self.viewer.eye = 3.5, 3.5, 3.5 simulation settings self.sim.dt = 1.0 / 60.0\ \ \ 4.3 Registering the Environment Register the task inside:\ src/isaac so arm101/tasks/reach/\ init \ .py\ \ \ \ PYTHONgym.register id="Isaac-Dual-Nero-Reach-v0", entry point="isaaclab.envs:ManagerBasedRLEnv", kwargs={ "env cfg entry point":f"{\ name \ }.dual nero joint pos env cfg:Dual Nero ReachEnvCfg", "rsl rl cfg entry point": f"{agents.\ name \ }.rsl rl ppo cfg:ReachPPORunnerCfg", "rl games cfg entry point": f"{agents.\ name \ }:rl games ppo cfg.yaml", }, disable env checker=True, 5. Training the Reach PolicyStep 1.Activate the Isaac Lab Environment conda activate env isaaclab Step 2.Navigate to the project directory: cd isaac so arm101 Step 3.Train the whole project Option 1:RSL-RL - Train: uv run train \ --task Isaac-Dual-Nero-Reach-v0 \ --headless - Evaluate: uv run play \ --task Isaac-Dual-Nero-Reach-v0 - Training result: dual nero rsl rl.gif Option 2: RL-Games - Train: python3 scripts/rl games/train.py \ --task Isaac-Dual-Nero-Reach-v0 \ --headless - Evaluate: python3 scripts/rl games/play.py \ --task Isaac-Dual-Nero-Reach-v0 - Training result: dual nero rl games.gif Both frameworks successfully learn the dual-arm reaching task. However, in our experiments: - RL-Games converges faster - Motion trajectories appear smoother - Final reaching accuracy is generally higher For relatively complex robot morphologies such as dual-arm manipulators, RL-Games currently provides more stable performance and is recommended as the default training backend. FAQ Q1: Why is my Dual-Nero robot collapsing or shaking violently after loading the URDF? This is usually caused by incorrect actuator parameters or unrealistic inertial properties. Common causes include: - Joint stiffness set too high - Damping values too low - Incorrect mass distribution in the URDF - Self-collision configuration issues - Unstable simulation timestep Before starting RL training, verify that the robot can remain stable under gravity using only PD control. Quick check: If the robot cannot stand still without RL, the issue is likely in the robot model rather than the training algorithm. Q2: Why does the reward improve, but the robot never reaches the target accurately? A rising reward does not always indicate successful task completion. Typical reasons include: - Reward weights are unbalanced - Orientation rewards dominate position rewards - Action penalties are too strong - Command sampling range is too large - End-effector link is incorrectly configured In most reach tasks, incorrect reward shaping is the primary reason for poor final accuracy. Q3: How do I verify that the end-effector link is configured correctly? One of the most common mistakes in Isaac Lab reach tasks is assigning the wrong end-effector body. For Dual-Nero, the target link should be: left gripper baseright gripper base Symptoms of an incorrect configuration include: - Reward remains low - Robot moves randomly - Training appears to converge but fails visually - End-effector does not move toward the target marker Always verify the body name in Isaac Sim before launching large-scale training. Q4: Why does RL-Games perform better than RSL-RL for this task? Both frameworks are PPO-based, but their implementations differ. For large-scale manipulation environments: RL-Games generally scales better with thousands of parallel environments PPO updates are often more stable Training throughput is higher on modern GPUs For Dual-Nero reach experiments, RL-Games typically achieves smoother trajectories and faster convergence. However, results may vary depending on reward design and task complexity. Q5: My policy works in simulation but fails on the real robot. Why? This is the most common Sim-to-Real issue. Possible causes include: - Joint friction mismatch - Encoder noise - Latency differences - Payload variations - Inaccurate motor models - Unmodeled cable effects To improve transfer performance: - Apply domain randomization - Add observation noise - Randomize dynamics parameters - Validate trajectories at low speed first Successful simulation training is only the first step toward real-world deployment. 📷 Have Question?If you encounter any issues with environment installation, parameter configuration, or RL training, feel free to leave your questions for further discussion. Read more javascript:void 0