NVIDIA, Google DeepMind, and Disney Research collaborate to develop next-generation open-source Newton physics engine.
NVIDIA announced a portfolio of technologies to enhance humanoid robot development, including NVIDIA Isaac GR00T N1, an open and customizable foundation model for generalized humanoid reasoning and skills.
The other technologies include simulation frameworks and blueprints such as the NVIDIA Isaac GR00T Blueprint for generating synthetic data, as well as Newton, an open-source physics engine — under development with Google DeepMind and Disney Research — purpose-built for developing robots.
Available now, GR00T N1 is the first of a family of fully customizable models that NVIDIA will pretrain and release to worldwide robotics developers — accelerating the transformation of industries challenged by global labor shortages estimated at more than 50 million people.
GR00T N1 advances humanoid developer community
The GR00T N1 foundation model uses a dual-system architecture modeled after cognitive processing. “System 1” processes actions quickly, similar to reflexes or intuition, while “System 2” processes information slowly for careful decision-making.
Powered by a vision language model, System 2 reasons about its environment and the instructions it has received to plan actions. System 1 then translates these plans into precise, continuous robot movements. System 1 is trained on human demonstration data and a massive amount of synthetic data generated by the NVIDIA Omniverse platform.
GR00T N1 can easily generalize across common tasks — such as grasping, moving objects with one or both arms, and transferring items from one arm to another — or perform multistep tasks that require long context and combinations of general skills. These capabilities can be applied across use cases such as material handling, packaging and inspection.
Developers and researchers can post-train GR00T N1 with real or synthetic data for their specific humanoid robot or task.
In his GTC keynote, Huang demonstrated 1X’s humanoid robot autonomously performing domestic tidying tasks using a post-trained policy built on GR00T N1. The robot’s autonomous capabilities are the result of an AI training collaboration between 1X and NVIDIA.
Among the additional leading humanoid developers worldwide with early access to GR00T N1 are Agility Robotics, Boston Dynamics, Mentee Robotics and NEURA Robotics.
NVIDIA, Google DeepMind and Disney Research focus on physics
NVIDIA announced a collaboration with Google DeepMind and Disney Research to develop Newton, an open-source physics engine that lets robots learn how to handle complex tasks with greater precision.
Built on the NVIDIA Warp framework, Newton will be optimized for robot learning and compatible with simulation frameworks such as Google DeepMind’s MuJoCo and NVIDIA Isaac Lab. Additionally, the three companies plan to enable Newton to use Disney’s physics engine.
Google DeepMind and NVIDIA are collaborating to develop MuJoCo-Warp, which is expected to accelerate robotics machine learning workloads by more than 70x and will be available to developers through Google DeepMind’s MJX open-source library, as well as through Newton.
Disney Research will be one of the first to use Newton to advance its robotic character platform that powers next-generation entertainment robots, such as the expressive Star Wars-inspired BDX droids that joined Huang on stage during his GTC keynote.
NVIDIA and Disney Research, along with Intrinsic, announced an additional collaboration to build OpenUSD pipelines and best practices for robotics data workflows.
More data to advance robotics post-training
Large, diverse, high-quality datasets are critical for robot development but costly to capture. For humanoids, real-world human demonstration data is limited by a person’s 24-hour day.
NVIDIA Isaac GR00T Blueprint for synthetic manipulation motion generation helps address this challenge. Built on Omniverse and NVIDIA Cosmos Transfer world foundation models, the blueprint lets developers generate exponentially large amounts of synthetic motion data for manipulation tasks from a small number of human demonstrations.
Using the first components available for the blueprint, NVIDIA generated 780,000 synthetic trajectories — the equivalent of 6,500 hours, or nine continuous months, of human demonstration data — in just 11 hours. Then, combining the synthetic data with real data, NVIDIA improved GR00T N1’s performance by 40%, compared with using only real data.
To further equip the developer community with valuable training data, NVIDIA is releasing the GR00T N1 dataset as part of a larger open-source physical AI dataset — also announced at GTC and now available on Hugging Face.
Availability
NVIDIA GR00T N1 training data and task evaluation scenarios are now available for download from Hugging Face and GitHub. The NVIDIA Isaac GR00T Blueprint for synthetic manipulation motion generation is also now available as an interactive demo on build.nvidia.com or to download from GitHub.
The NVIDIA DGX Spark personal AI supercomputer, also announced today at GTC, provides developers a turnkey system to expand GR00T N1’s capabilities for new robots, tasks and environments without extensive custom programming.
The Newton physics engine is expected to be available later this year.
For more information, visit nvidia.com.
