Get Started with NuRec: For Robotics

NVIDIA Omniverse NuRec is integrated in Omniverse, and available to use with the Omniverse robotics simulation app Isaac Sim. To use NuRec in a robotics simulation platform, you can reconstruct a scene from mono camera data or stereo camera data and then use the generated assets in Isaac Sim. Follow the steps in this quickstart guide to use NuRec for robotics simulations.

Step 1: Set Up

Check that you have the right hardware and software setup to run NuRec.

Step 2: Generate Reconstructed Scenes

If you have existing data you want to use to reconstruct scenes for use in a simulation platform, follow the steps in the guides to robotics reconstructions with NuRec based on your data type:

• For Mono Camera Data

• For Stereo Camera Data

If you want to use existing datasets in a simulation platform, get the NVIDIA PhysicalAI Robotics NuRec Dataset from HuggingFace (you need a HuggingFace account for this).

Step 3: Render Reconstructions in a Simulation

NuRec rendering technology is integrated into Omniverse, and available on any simulation platform built with Omniverse Kit, like Isaac Sim. Render NuRec-reconstructed scenes in Isaac Sim by loading the generated USDZ file and using the OmniNuRecVolumeAPI schema.

Learn more and follow the steps in the guide to using NuRec Assets in Isaac Sim, Neural Volume Rendering.