Understand - Spatial Intelligence for Real-World AI | Niantic Spatial, Inc.

3D semantic understanding adds meaning and context to a reconstructed environment, not just geometry. Niantic Spatial Understand encodes each 3D point with information about features such as geometry, materials, and context, creating maps that humans, machines, and AI agents can query, measure, and reason over.

Open-set semantics allow a system to identify and reason about concepts beyond a fixed list of predefined categories. In Understand, open-set generalization is designed to adapt to new concepts without retraining for every new category or relying only on a closed taxonomy. That makes the semantic layer better suited to open-ended questions about real environments, including objects, materials, relationships, and conditions.

Per-point semantic understanding attaches a semantic descriptor to each point in a 3D scene. The descriptor can capture geometric, material, and contextual features, allowing the system to search or compare specific parts of an environment rather than treating the entire scene as one label. This creates a spatially grounded foundation for segmentation, measurement, natural-language search, and downstream AI workflows.

Yes. It is designed to support open-ended, multimodal questions about a 3D environment, such as where vegetation encroaches on infrastructure, which terrain is traversable, or which structures provide shelter or shade. Cross-modal grounding connects visual, spatial, and linguistic information so the answer can be tied to a location or region in the mapped environment.

Traditional object detection or closed-set segmentation generally assigns predefined labels to visible regions. 3D semantic understanding adds spatial grounding and context: it can represent meaning at individual 3D points, connect relationships across objects and surfaces, and support open-ended queries. Understand is positioned as a layer for interpreting and operating in a measured 3D environment, not simply labeling isolated 2D images.

3D semantic understanding can support scene interpretation, situational awareness, semantic navigation, task recognition, inspection, and asset intelligence. Humans, robots, and AI agents in can use meaning, not geometry alone, to interpret surroundings, identify relevant areas, reason about conditions, and make more informed decisions.

Spatial semantics gives AI agents and robots a shared, queryable representation of the physical world. By grounding visual, spatial, and linguistic information in a 3D map, it helps machines interpret their surroundings and act on context, not just geometry. That foundation supports applications such as semantic navigation, task recognition, adaptive decision-making, inspection, and asset intelligence.