SO-ARM101: MoveIt in Isaac Sim with ROS2

Intro

In this tutorial we will learn how to integrate MoveIt in Isaac Sim with ROS2 on the SO-ARM101. The SO-ARM101 (or the older version SO-ARM100) is an open-source dual-arm robot created by TheRobotStudio along with HuggingFace’s LeRobot for robotics research & education. MoveIt is used for robot motion planning, manipulation, and control in ROS2 allowing to control the real robot from simulation. However this tutorial is “simulation only”. A Sim2Real will follow in the future!

🤖 Don’t have a SO-ARM? → Buy one with a Discount: LYCHEEAI5 !

There are official vendors who sell all required parts and already assembled kits such as WowRobo from where I bought mine. I was very happy with the low cost, the quick delivery and also the awesome support from the founder himself, Leo Xiao, so I naturally asked for a collaboration and now I’m excited to announce that WowRobo is sponsoring this project. They also provide my community with a 5% discount to make learning robotics even more accessible!

You can get your own SO-ARM101 (or SO-ARM100) here using code LYCHEEAI5:

💡

Make sure to “Add to cart” or choose “More payment options” in order to use the Discount Code!

Prerequisites

End Result (if you want to skip Step-By-Step Tutorial below or encounter issues)

# Clone the repository
git clone https://github.com/MuammerBay/SO-ARM101_MoveIt_IsaacSim.git
cd SO-ARM101_MoveIt_IsaacSim

# Source ROS 2
source /opt/ros/humble/setup.bash

# Update rosdep database
rosdep update

# Install ALL dependencies automatically
rosdep install --from-paths src --ignore-src -r -y

# Build the workspace
colcon build

# Source the workspace
source install/setup.bash

# Launch MoveIt demo
ros2 launch so_arm_moveit_config demo.launch.py
  • On a seperate terminal start Isaac Sim
# Source ROS 2
source /opt/ros/humble/setup.bash

# Run Isaac Sim with ros2 enabled
[ISAAC-SIM-4.5-FOLDER]/isaac-sim.selector.sh
  • Open the USD file (created from URDF)
  • Start the Simulation (press Play)
  • Move Robot in RViz

Step-By-Step Tutorial

ROS2 & MoveIt

  • We will follow the Tutorial from robot mania and apply it to our SO-ARM robot
      • [YT: robot mania] How to Use MoveIt with Isaac Sim: A Step-by-Step Guide
    1. All credits to YouTube robot maniaYouTube robot mania. I highly recommend his channel as he covers a lot of very interesting topics!
  • First make sure you always source your ROS2 Humble in every terminal
    • source /opt/ros/humble/setup.bash
  • We will create a ROS workspace
    • mkdir -p ~/so-arm_moveit_isaacsim_ws/src
cd ~/so-arm_moveit_isaacsim_ws
  • And get all the ROS2 Dependencies
    • sudo apt update
sudo apt install \
  ros-humble-moveit \
  ros-humble-ros2-control \
  ros-humble-ros2-controllers \
  ros-humble-gripper-controllers \
  ros-humble-topic-based-ros2-control
  • Prepare your MoveIt configuration & robot-description folder
    • cd ~/so-arm_moveit_isaacsim_ws/src
mkdir -p so_arm_moveit_config
mkdir -p so_arm_description
  • First we will copy the URDF file for the SO-ARM from my newly created repo Embed GitHubEmbed GitHub into the so_arm_description folder
  • Git clone the URDF repo.
    • 💡

    Make sure to add a point . to the end of the git clone command to include only the files and not the folder. This is important for the URDF file

    cd ~/so-arm_moveit_isaacsim_ws/src/so_arm_description
git clone https://github.com/MuammerBay/SO-ARM_ROS2_URDF.git .
  • Build your workspace
    • cd ~/so-arm_moveit_isaacsim_ws
colcon build
  • Launch the MoveIt Setup Assistant (see video above). # Make sure your workspace is sourced so it can find your robot_description package
    • source ~/so-arm_moveit_isaacsim_ws/install/setup.bash
ros2 launch moveit_setup_assistant setup_assistant.launch.py
  • After the Setup: rebuild
    • cd ~/so-arm_moveit_isaacsim_ws
colcon build
  • After Source and test (Before Isaac Sim Setup)
    • source install/setup.bash
ros2 launch so_arm_moveit_config demo.launch.py
  • Errors: Replace Integer with Float (see video above) in joint_limits.yaml
  • Controller adjustment moveit_controllers.yaml . Add these to arm_controller
    • action_ns: follow_joint_trajectory
default: true
  • After errors: rebuild again
    • colcon build
  • Now you can test MoveIt without Isaac Sim
    • ros2 launch so_arm_moveit_config demo.launch.py

Isaac Sim

  • Open a terminal and source ROS
    • source /opt/ros/humble/setup.bash
  • Open Isaac Sim with the selector
    • [ISAAC-SIM-4.5-FOLDER]/isaac-sim.selector.sh
  • Import URDF
  • For the Stiffness & Damping use these values (from the Official Repo)
    • stiffness= 17.8
    • damping= 0.60
  • Don’t forget to set the articulation root to the base mesh (xform)
  • Create a pre-configured Action Graph for Isaac Sim ROS2 Bridge
  • (follow video instructions) Tools → Robotics →ROS2 Omnigraphs → Joint States
    • Articulation to base mesh (xform)
    • /isaac_joint_states
    • /isaac_joint_command
  • In src/so_arm_moveit_config/config/so101_new_calib.ros2_control.xacro adjust this
    • 				<ros2_control name="${name}" type="system">
				  <hardware>
				      <!-- By default, set up controllers for simulation. This won't work on real hardware -->
				      <plugin>mock_components/GenericSystem</plugin>
				  </hardware>
  • To this:
    •         <ros2_control name="${name}" type="system">
            <hardware>
                <!-- By default, set up controllers for simulation. This won't work on real hardware -->
                <!-- plugin>mock_components/GenericSystem</plugin -->
                <plugin>topic_based_ros2_control/TopicBasedSystem</plugin>
                <param name="joint_states_topic">/isaac_joint_states</param>
                <param name="joint_commands_topic">/isaac_joint_command</param>
            </hardware>
  • One last time: rebuild
    • cd ~/so-arm_moveit_isaacsim_ws
colcon build
  • Open MoveIt in RViz
    • source install/setup.bash
ros2 launch so_arm_moveit_config demo.launch.py
  • Start the Simulation (press Play)
  • Move Robot in RViz