Addressing the Robotics Sim-to-Real Gap with USDZ Export in Scaniverse
Niantic Spatial's meshes inherit their source splat's high fidelity, yielding smoother, truer surfaces than a standalone geometry scan - without the roughness that typically plagues reconstructed meshes.
The real world is the best simulation. While synthetically generated environments are important for robotics training, simulations that include real-world, geometrically accurate and aligned data are critical for training robots to operate safely and effectively in the complexities of the physical world.
That’s why today we’re launching USDZ export in Scaniverse: a direct path from a real-world capture to a simulation-ready environment.
Scaniverse can now turn a 360 video capture into a simulation-ready USDZ file for NVIDIA Isaac Sim by combining a Gaussian splat with an automatically generated and aligned mesh. It’s just the latest step in our mission to build real-world foundational models for physical AI. We’re already using one of our models – our depth model – to produce these reconstructions with much smoother, more accurate surfaces.
This update extends our real-world reconstruction workflow directly into the robotics simulation stack, giving developers the simplest path yet from a real-world 360 capture to a simulation-ready digital twin. Where traditional RGB-LiDAR setups require hardware that can run to tens of thousands of dollars, a 360 camera costing as little as $500 captures an entire street or large indoor space in a single 5-minute pass. It's a meaningful reduction in the cost and effort barrier to environment-specific policy training.
Overcoming the sim-to-real gap
Robotics developers face a persistent sim-to-real gap. Policies trained in synthetic environments often struggle when deployed in messy, irregular, real-world settings. Generalized training across thousands of locations makes robots better in the abstract, but it doesn’t prepare them for your customer’s warehouse, hallway, or specific environment.
The answer is a real-to-sim-to-real workflow where developers can ground in reality from the first capture to the final policy by exporting a Gaussian splat and mesh in a single USDZ file. Scan a real environment before a robot arrives, build a simulation-ready digital twin from it, train in that twin, and deploy a robot whose policy has already trained in a twin of its destination, ready to work from day one.
Why splats matter for robotics
As robot policies become increasingly vision-based, the camera feed becomes the policy's primary input, and the appearance of the training environment becomes as important as its physics. A splat captured from the real world carries the lighting, textures, and real-world clutter that synthetic environments approximate or omit. Training against it closes the visual sim-to-real gap, so a robot's perception system meets its destination already familiar with how that place looks.
The mesh still does the physical work - collision geometry, surface topology, the edges and elevation changes a robot navigates. What's different here is where the mesh comes from. It’s derived directly from the splat, in the same capture, rather than from a separate scan or an independent process.
Two things follow. The visual and physical layers come from a single capture, so what the simulation renders and what the robot collides with share one source, with no cross-registration step that introduces drift. And the mesh inherits the splat's fidelity and greater sense of depth, so that the depth computed from the splat yields smoother, truer surfaces than a standalone geometry scan would produce.
A real-world mesh lets a policy recognise the surface irregularities that decide whether a robot crosses a threshold confidently or fails at it; a mesh that inherits a high quality splat's accuracy lets it do so faithfully.
That value compounds after deployment. A persistent, accurate digital twin - visual and physical, aligned - supports ongoing policy refinement as robots learn and adapt, making it infrastructure for the long term rather than a one-time training asset.
Try this simple workflow and download sample USDZ scenes today
The workflow turns a previously manual, multi-step process into a single downloadable asset:
-
Capture a real-world environment using inexpensive and fast 360 cameras
-
Upload to Scaniverse web to generate a high-fidelity Gaussian splat from the scan. Scaniverse derives an aligned mesh directly from that splat, with mesh depth computed from the splat for smoother surfaces. The splat and mesh are packaged together as a USDZ file
-
Import the USDZ directly into NVIDIA Isaac Sim or Isaac Lab and begin training
Complete this form to download sample USDZ by scenes to load into your environment or get in touch with the Niantic Spatial team.