Course 3 - Synthetic Data Generation for Perception Model Training in Isaac Sim

Course 3 - Synthetic Data Generation for Perception Model Training in Isaac Sim

Image Credits

Image from my own synthetic dataset generated from this course.

This course walks through how to generate a synthetic dataset using Isaac Sim Replicator. The dataset uses domain randomization (a technique that randomizes simulation aspects) to better mimic the real world. The course follows this workflow to generate a synthetic dataset of a robot’s visual perception pipeline in a warehouse setting.

image
image
image

Module 1: Perception Models in Dynamic Robotic Tasks

No additional comments for this module.

Module 2: What is Synthetic Data Generation (SDG)?

‣
What exactly are we simulating in this module? How is it different from generating data from a statistical model?
‣
What is the difference between Replicator and NVIDIA Cosmos?

Module 3: Domain Randomization with Replicator

No additional comments for this module.

Module 4: Generating a Synthetic Dataset Using Replicator

Unfortunately, all commands in the course are for Linux. Below are two workarounds I found to execute the ./generate_data script in Windows.

‣
Via WSL2 (Ubuntu)
‣
Via Cygwin:
❗

In both workarounds, the script may end with an error about how the last python command in generate_data.sh file needs a valid input for --num_frames , even though we put something like 10 or 1000 as the argument. In my experience, the “NO_DISTRACTIONS” folder and the images inside still get generated, even with this error.

‣
What are the depth, instance_segmentation, object_detection, rgb, semantic, and semantic_segmentation folders generated?

Module 5: Fine-Tuning and Validating an AI Perception Model

This module is on hold.