Skip to main content
AmigoFish

From Human Demonstrations to Robot Training Data With RLWRLD and Reactor | Cosmos Labs

From NVIDIA Developer ·

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

How do you turn a video of someone working into training data a robot can learn from? Join NVIDIA, RLWRLD, and Reactor to explore a pipeline that connects human demonstrations to robot actions, visual observations, and task instructions—and see how to run a Cosmos simulation checkpoint on Reactor.

Follow the process from synchronized video capture and hand–object reconstruction to robot-specific motion retargeting and rendering in NVIDIA Isaac Sim. Explore how RLWRLD uses NVIDIA Cosmos 3 features in its RLHND hand-tracking system, then examine research demos of world-model-generated observations for garment packing and different robot configurations.

Reactor will demonstrate how to run the Cosmos 3 Nano Sim Bimanual checkpoint on its platform for real-time world-model inference.

Built for robotics developers and researchers, the session covers practical data-quality choices, what the current results demonstrate, and what remains to be evaluated.

What You'll Learn: - Follow the pipeline from synchronized human video to aligned robot actions, observations, and task instructions. - Understand how RLHND uses NVIDIA Cosmos 3 features for hand pose, contact, and force estimation, and how reconstructed motion is retargeted to a robot. - Compare Isaac Sim-rendered observations with experimental world-model-generated observations, including deformable objects and different robot configurations. - How to run interactive world simulation models on Reactor

Have questions about human demonstration data, hand tracking, retargeting, or observation synthesis? Add them to the live chat for the NVIDIA + RLWRLD + Reactor Q&A.

Access more NVIDIA Cosmos developer resources and join our developer community:

📄Read the RLWRLD technical blog → https://www.rlwrld.ai/en/insight/blog/116 ⬇️ Download Cosmos on Hugging Face → https://huggingface.co/collections/nvidia/cosmos3 📚 Explore Models & Datasets on GitHub → https://github.com/nvidia/Cosmos 👥 Join the Cosmos Community → https://discord.com/invite/nvidiaomniverse

Back to NVIDIA Developer · Report a problem