From Human Demonstrations to Robot Training Data With RLWRLD and Reactor | Cosmos Labs
From NVIDIA Developer ·

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
How do you turn a video of someone working into training data a robot can learn from? Join NVIDIA, RLWRLD, and Reactor to explore a pipeline that connects human demonstrations to robot actions, visual observations, and task instructions—and see how to run a Cosmos simulation checkpoint on Reactor.
Follow the process from synchronized video capture and hand–object reconstruction to robot-specific motion retargeting and rendering in NVIDIA Isaac Sim. Explore how RLWRLD uses NVIDIA Cosmos 3 features in its RLHND hand-tracking system, then examine research demos of world-model-generated observations for garment packing and different robot configurations.
Reactor will demonstrate how to run the Cosmos 3 Nano Sim Bimanual checkpoint on its platform for real-time world-model inference.
Built for robotics developers and researchers, the session covers practical data-quality choices, what the current results demonstrate, and what remains to be evaluated.
What You'll Learn: - Follow the pipeline from synchronized human video to aligned robot actions, observations, and task instructions. - Understand how RLHND uses NVIDIA Cosmos 3 features for hand pose, contact, and force estimation, and how reconstructed motion is retargeted to a robot. - Compare Isaac Sim-rendered observations with experimental world-model-generated observations, including deformable objects and different robot configurations. - How to run interactive world simulation models on Reactor
Have questions about human demonstration data, hand tracking, retargeting, or observation synthesis? Add them to the live chat for the NVIDIA + RLWRLD + Reactor Q&A.
Access more NVIDIA Cosmos developer resources and join our developer community:
📄Read the RLWRLD technical blog → https://www.rlwrld.ai/en/insight/blog/116 ⬇️ Download Cosmos on Hugging Face → https://huggingface.co/collections/nvidia/cosmos3 📚 Explore Models & Datasets on GitHub → https://github.com/nvidia/Cosmos 👥 Join the Cosmos Community → https://discord.com/invite/nvidiaomniverse