# How to Download a Mesh Using the API Source: https://www.nianticspatial.com/docs/nsdk/how-to/vps/mesh_download/ With the Mesh Download API, you can download and create a mesh of any Site at runtime, allowing you to dynamically create mesh overlays in AR scenes. This feature makes it easier to test AR experiences by allowing you to check that your mesh lines up with the real world without having to leave the test environment. For example, you can download a stored mesh after localizing to see how far offset your localization is and figure out how it needs to change. Mesh downloading also allows developers to place content in scenes and explore environmental interactions without needing to stop testing and set up each mesh they want to try. A Site mesh is delivered as a stream of **chunks** rather than as a single download. Identify the map by the **VPS map asset ID** of the map you localized to. The first request starts the download process, and mesh chunks are received via polling the API. Chunks arrive closest-first relative to a geographic position you supply, so the geometry nearest the user renders while the rest is still downloading. You can cap how much mesh data the request downloads to help with rendering large maps. > **Warning:** > > **Anchor payload mesh download APIs are deprecated** > > The APIs for downloading meshes using a Site's anchor payload are deprecated. > These APIs do not support maps captured by 360 cameras or location-based streaming. > Site IDs and asset IDs are the preferred mechanisms for identifying maps in NSDK. > Use the streaming, asset-based API described on this page instead. > > | Platform | Deprecated API | Migrate To | > | --- | --- | --- | > | Unity | `LocationMeshManager.GetLocationMeshForPayloadAsync` | `LocationMeshManager.GetLocationMeshChunksByVpsMapAssetIdAsync` | > | Swift | `NSDKMeshDownloader.requestLocationMesh(payload:)` | `NSDKMeshDownloader.requestLocationMeshByAsset(vpsMapAssetId:)` | > | Kotlin | `MeshDownloaderSession.download(payload =)` | `MeshDownloaderSession.downloadByAsset(vpsMapAssetId =)` | ### Platform: unity ## Prerequisites You will need: - a Unity project with NSDK installed and configured - an NSDK access token configured for the app; see [Authorization](https://www.nianticspatial.com/docs/nsdk/auth_getting_started/) - the **VPS map asset ID** of the Site, obtained through the [Sites API](https://www.nianticspatial.com/docs/nsdk/features/sites/) - a geographic position to order the chunks by, ideally the device's current location For project and scene setup, see [Set up the Niantic SDK for Unity](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-niantic-sdk-for-unity). ## Get the VPS map asset ID Fetch the Site's assets and use the ID of its production VPS asset. `AssetInfo.Id` is the VPS map asset ID to use with Mesh Downloader. ```cs using NianticSpatial.NSDK.AR.Sites; [SerializeField] private SitesClientManager sitesClientManager; ``` ```cs var assetsResult = await sitesClientManager.GetAssetsForSiteAsync(siteId); if (assetsResult.Status != SitesRequestStatus.Success) { return; } string vpsMapAssetId = null; foreach (var asset in assetsResult.Assets) { if (asset.VpsData.HasValue && asset.Deployment == AssetDeploymentType.Production) { vpsMapAssetId = asset.Id; break; } } ``` ## Stream a Site mesh [`LocationMeshManager`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.Subsystems.LocationMeshManager) is a component included with NSDK. Add **Location Mesh Manager** to a `GameObject`, then reference that component from the script that downloads the mesh: ```cs using NianticSpatial.NSDK.AR.Subsystems; [SerializeField] private LocationMeshManager locationMeshManager; ``` In the Inspector, drag the `GameObject` containing **Location Mesh Manager** into this field. Then iterate `GetLocationMeshChunksByVpsMapAssetIdAsync` with `await foreach`. Each iteration yields one chunk as a `GameObject` with its transform already applied. #### View the Unity mesh streaming code ```cs var meshRoot = new GameObject("DownloadedMesh"); meshRoot.transform.SetParent(siteAsset.transform, false); int chunkCount = 0; await foreach (var chunk in locationMeshManager.GetLocationMeshChunksByVpsMapAssetIdAsync( vpsMapAssetId, siteLatitude, siteLongitude, getTexture: true)) { // Chunks arrive inactive so they never flash at the scene origin. Set the hierarchy before activating it. chunk.transform.SetParent(meshRoot.transform, false); chunk.SetActive(true); chunkCount++; } // A finished stream that produced no chunks is the signal that no mesh was downloaded. if (chunkCount == 0) { Destroy(meshRoot); } ``` `latitude` and `longitude` only decide the order the chunks arrive in. They do not affect the position of the mesh itself. ## Position the mesh chunks Downloading a mesh does not localize the device or position the mesh in the AR scene. Chunk vertices are in the chunk's own local space, in meters, and each chunk's transform places the chunk relative to the VPS map's own origin. To know the correct real-world position for the mesh, we first need to localize to its map. Track the same VPS map asset and parent the chunks to its trackable, as in the code above: ```cs if (!arVps2Manager.TryTrackAsset(vpsMapAssetId, out ARVps2Asset siteAsset)) { return; } // Update visibility as tracking changes in your app. meshRoot.SetActive(siteAsset.trackingState == TrackingState.Tracking); ``` [`TryTrackAsset`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2Manager.TryTrackAsset) accepts any valid asset ID, but the trackable only receives a pose once the asset's Site has been submitted with [`TryLocalize`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2Manager.TryLocalize) and localized. See [Place virtual content with VPS2](https://www.nianticspatial.com/docs/nsdk/how-to/vps2/placing_virtual_content/) for that flow. #### Unity scene requirements and sample files A project that downloads and renders a mesh needs authorization; a `Location Mesh Manager`; a compatible mesh material; and a source for the Site's VPS asset. Placing the mesh at its physical Site also requires an `AR Session`, an `XR Origin` with an AR camera and `AR VPS2 Manager`, and camera and location permissions. The [NSDK Unity sample project](https://github.com/nianticspatial/nsdk-samples-csharp) shows how these pieces are connected: - `Assets/Samples/VPS2/Scenes/VPS2Localization.unity` contains the scene components. - `Assets/Samples/VPS2/Scripts/VPS2AssetLocalizeDemo.cs` contains the asset tracking and mesh streaming flow. - `Assets/Samples/VPS2/Scripts/SitesTargetListManager.cs` obtains the VPS asset ID and the Site's coordinates. ## Limit how much mesh data you download `GetLocationMeshChunksByVpsMapAssetIdAsync` also accepts options for download size, collision geometry, textures, and cancellation: ```cs await foreach (var chunk in locationMeshManager.GetLocationMeshChunksByVpsMapAssetIdAsync( vpsMapAssetId, siteLatitude, siteLongitude, getTexture: true, maxChunks: 8, maxSizeKb: 10240, addCollider: true, cancelOnDisable: true)) ``` - `maxChunks` downloads only the given number of nearest chunks. The default, `0`, downloads all chunks. - `maxSizeKb` stops the request once the downloaded mesh and texture data reaches this size. The default, `0`, sets no limit. - `getTexture` requests texture data. When it is `false`, chunks contain geometry with no texture image. - `addCollider` adds collision geometry to each chunk. - `cancelOnDisable` stops the stream if `LocationMeshManager` is disabled. The mesh must use a material compatible with the project's render pipeline. Set **Textured Mesh Material** and **Vertex Color Material** on the `Location Mesh Manager`: chunks that carry a texture use the textured material, and chunks without one use the vertex color material. Refer to the `Location Mesh Manager` in the sample scene for the setup used by the installed NSDK version. Textured meshes contain more download data; untextured meshes are smaller and require a material that renders vertex colors. ### Platform: swift ## Prerequisites You will need a Swift project with NSDK installed and the VPS map asset ID of the map you want a mesh for. For more information, see [Set up the NSDK in Swift](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-nsdk-in-swift) and [Niantic Spatial VPS](https://www.nianticspatial.com/docs/nsdk/features/lightship_vps/). ## Localize and get the VPS map asset ID First, localize to the Site. Then, fetch the Site's assets and get the ID of its production VPS asset. `Vps2LocalizedAsset` contains the asset ID of the VPS map: ```swift let vps2Session = nsdkSession.acquireVps2Session() try vps2Session.localize(siteId: siteId) // Alternatively, pass in a lat/long to automatically target nearby sites ``` ```swift vps2Session.$latestLocalization .receive(on: DispatchQueue.main) .sink { [weak self] localization in guard let self, let asset = localization?.localizedAsset else { return } self.startMeshDownload(vpsMapAssetId: asset.assetId) } .store(in: &cancellables) ``` `localizedAsset` is `nil` until a visual localization succeeds. If you need the asset ID before localizing, you can fetch it from the [Sites API](https://www.nianticspatial.com/docs/nsdk/features/sites/) instead: ```swift let sitesSession = nsdkSession.acquireSitesSession() let assets = try await sitesSession.requestAssetsForSite(siteId: siteId) let vpsMapAssetId = assets.assets.first { $0.assetType == .vpsInfo && $0.deployment == .production }?.id ``` ## Stream a Site mesh The `NSDKMeshDownloader` downloads mesh geometry and textures for Sites at runtime. Create an `NSDKMeshDownloader` instance from an existing `NSDKSession`: ```swift let nsdkSession = NSDKSession(accessToken: "YOUR_ACCESS_TOKEN") let meshDownloader = nsdkSession.acquireMeshDownloader() ``` Call `requestLocationMeshByAsset()` with the VPS map asset ID and the device's current geographic position. The download runs asynchronously, and mesh chunks are received sequentially by polling the API. For large maps, set `maxSizeKb` or `maxChunks` to limit the download size and help manage memory usage. Mesh downloading will automatically stop once it reaches either limit. ```swift let stream = meshDownloader.requestLocationMeshByAsset( vpsMapAssetId: vpsMapAssetId, latitude: coordinate?.latitude ?? 0.0, longitude: coordinate?.longitude ?? 0.0, getTexture: true // Optional parameters: // maxChunks -> number of nearest chunks to download, default: 0 (no limit) // maxSizeKb -> in kilobytes, default: 0 (no cap) // pollingInterval -> in seconds, default: 0.1 ) do { for try await chunk in stream { // Render this chunk (MeshDownloaderChunk) as soon as it arrives. addMeshChunk(chunk) } } catch MeshDownloaderResults.Error.sizeLimitReached { // A size cap stopped the download. Every chunk that fit under the cap has already // been delivered, so treat this as a successful early stop. } catch { print("Failed to download mesh: \(error)") } ``` Each `MeshDownloaderChunk` includes geometry data in [`MeshData`](https://www.nianticspatial.com/docs/api/swift/NSDK.class-MeshData), including `verticesPtr`, `indicesPtr`, `normalsPtr`, and `uvsPtr`, the texture image in `imageData`, and the chunk transform in `transform`. `latitude` and `longitude` only decide the order the chunks arrive in. They do not affect the position of the mesh itself. Cancelling the task that iterates the stream stops the download. ## Position the mesh chunks Chunk vertices are in the chunk's own local space, in meters, and each chunk's `transform` places the chunk relative to the VPS map's own origin. Get the pose of that origin in AR tracking space from `assetTrackingData(assetId:)` and place the chunks under it. Continue polling for tracking data: the pose refines as the device moves, and it's `nil` until the asset is fully tracked. ```swift guard let tracking = vps2Session.assetTrackingData(assetId: vpsMapAssetId) else { return } // Pose the parent entity once, then add each chunk with its own local transform relative to the VPS map origin. assetAnchor.transform = Transform(matrix: tracking.assetToTrackingTransform) ``` ```swift let modelEntity = ModelEntity(mesh: meshResource, materials: [material]) modelEntity.transform = Transform(matrix: chunk.transform) assetMeshRoot.addChild(modelEntity) ``` ### Rendering a downloaded mesh in SceneKit If you use SceneKit to render the mesh, [`MeshData.toSCNGeometry()`](https://www.nianticspatial.com/docs/api/swift/NSDK.MeshData.method-toSCNGeometry) converts the mesh coordinates and texture coordinates for you. You can create an `SCNGeometry` from each chunk and place it in an `SCNNode`. If you want the scanned texture, create an `SCNMaterial` from `imageData` and pass it to `toSCNGeometry(material:)`. ### Rendering a downloaded mesh in RealityKit If you build a RealityKit `MeshResource` or `ModelEntity` from `chunk.meshData`, handle the following NSDK-specific details so that each mesh chunk appears in the expected orientation and with the correct texture mapping. 1. Correct the coordinate convention by **negating Y and Z** components of the geometry data (rotating 180 degrees around the X axis). NSDK mesh coordinates use OpenCV conventions and RealityKit coordinates use OpenGL conventions, so this rotation corrects the mesh's orientation. For example, rotate the mesh entity by `simd_quatf(angle: .pi, axis: [1, 0, 0])`, in addition to applying each chunk's `transform`. 2. Apply the scanned texture. Download with `getTexture: true`, use `meshData.uvsPtr` as the `MeshDescriptor` texture coordinates, and decode `imageData` into a `TextureResource` for the material's base color. `imageData` is empty when the chunk has no texture, so fall back to an untextured material in that case. ### Platform: kotlin ## Prerequisites You will need a Kotlin project with NSDK installed and the VPS map asset ID of the map you want a mesh for. For more information, see [Set up the NSDK in Kotlin](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-nsdk-in-kotlin) and [Niantic Spatial VPS](https://www.nianticspatial.com/docs/nsdk/features/lightship_vps/). ## Localize and get the VPS map asset ID First, localize to the Site. Then, fetch the Site's assets and get the ID of its production VPS asset. `Vps2LocalizedAsset` contains the asset ID of the VPS map: ```kotlin val vps2Session = nsdkSession.vps2.acquire() vps2Session.localize(siteId = siteId) ``` ```kotlin vps2Session.localizationUpdates .onEach { localization -> val asset = localization.localizedAsset ?: return@onEach startMeshDownload(vpsMapAssetId = asset.assetId) } .launchIn(lifecycleScope) ``` `localizedAsset` is `null` until a visual localization succeeds. If you need the asset ID before localizing, fetch it from the [Sites API](https://www.nianticspatial.com/docs/nsdk/features/sites/) instead: ```kotlin val sitesSession = nsdkSession.sites.acquire() val assets = sitesSession.requestAssetsForSite(siteId) val vpsMapAssetId = assets.assets.firstOrNull { asset -> asset.assetType == AssetType.VPS_INFO && asset.deployment == AssetDeploymentType.PRODUCTION }?.id ``` ## Stream a Site mesh The `MeshDownloaderSession` downloads mesh geometry and textures for Sites at runtime. Create a session from an existing `NSDKSession` instance: ```kotlin val nsdkSession = NSDKSession( accessToken = "YOUR_ACCESS_TOKEN", useLidar = false ) val meshDownloaderSession = nsdkSession.meshDownload.acquire() ``` Call `downloadByAsset()` with the VPS map asset ID and the device's current geographic position. It returns a cold `Flow`, so the download starts when you collect it. The download runs asynchronously, and mesh chunks are received one by one: ```kotlin lifecycleScope.launch { var terminalError: MeshDownloaderError? = null meshDownloaderSession.downloadByAsset( vpsMapAssetId = vpsMapAssetId, latitude = latitude, longitude = longitude, getTexture = true, maxChunks = 0, // Number of nearest chunks to download, default: 0 (no limit) maxSizeKb = 0 // In kilobytes, default: 0 (no cap) ).collect { result -> when (result) { is NSDKResult.Success -> { // Render this chunk (MeshDownloaderData) as soon as it arrives. addMeshChunk(result.value) } is NSDKResult.Error -> terminalError = result.code } } when (terminalError) { null, MeshDownloaderError.NONE -> Log.i("MeshDownload", "Download complete") // A size cap stopped the download. Every chunk that fit under the cap has // already been delivered, so treat this as a successful early stop. MeshDownloaderError.SIZE_LIMIT_REACHED -> Log.i("MeshDownload", "Size limit reached") else -> Log.e("MeshDownload", "Download failed: $terminalError") } } ``` Each `MeshDownloaderData` exposes `meshData` (a `MeshData` with `vertices`, `indices`, `normals`, and `uvs`), `imageData` (the chunk's texture image), and `transform` (the chunk's pose). `latitude` and `longitude` only decide the order the chunks arrive in. They do not affect the position of the mesh itself. Cancelling the collecting coroutine stops the download. ## Position the mesh chunks Chunk vertices are in the chunk's own local space, in meters, and each chunk's `transform` places the chunk relative to the VPS map's own origin. Get the pose of that origin in AR tracking space from `getAssetTrackingData(assetId)` and place the chunks relative to it. Continue polling for tracking data: the pose refines as the device moves, and it's `null` until the asset is fully tracked. ```kotlin val tracking = vps2Session.getAssetTrackingData(vpsMapAssetId) ?: return val assetTransform = FloatArray(16).also { tracking.assetToLocalTrackingTransform.toMatrix(it, 0) } // modelMatrix = assetToLocalTrackingTransform * chunk.transform val modelMatrix = FloatArray(16) Matrix.multiplyMM(modelMatrix, 0, assetTransform, 0, chunk.transform, 0) ``` ### Rendering a downloaded mesh When you build geometry from `meshData`, make two adjustments so each chunk appears in the correct orientation and with the scanned texture: > **Tip:** > > **Convert the coordinate convention** > > VPS mesh coordinates use OpenCV convention (**Y-down, Z-forward**), while ARCore uses OpenGL convention (**Y-up, Z-back**). > Correct the coordinate convention by **negating Y and Z** components of the geometry data (rotating 180 degrees around the X axis) so that the mesh appears upright and stays aligned with the localized Site. > > If you apply the correction as a transform instead of modifying the vertex buffer itself, apply it to the mesh transform, after the asset transform and chunk transform. In other words, use `asset * chunk * correction`, not `correction * asset * chunk`. Apply the correction to the mesh itself rather than to the whole world transform. > **Tip:** > > **Apply the texture** > > To show the scanned texture, do the following: > > 1. Download the mesh with `getTexture = true`. > 2. Decode the chunk's `imageData` into a texture. Some meshes are untextured and return an empty `ByteArray`, so fall back to an untextured material in that case. > 3. Upload `meshData.uvs` and use those UVs when you sample the texture in your shader. > > If the texture appears upside down in a raw OpenGL renderer, flip the V coordinate in the shader with `1.0 - uv.y` because decoded image rows are top-down, while OpenGL texture coordinates are bottom-up. ## Textures and 360 captures Meshes generated from 360 camera captures are served as decimated, **untextured** meshes, so it's recommended to render them with a vertex color or wireframe material to avoid covering the AR camera view with opaque surfaces. Due to the decimation, their level of geometry detail will be lower than the original reconstruction in Scaniverse. ## Known Issues - Some meshes can be quite large. If download size or runtime performance is a concern, limit the number of chunks or set a maximum download size. - A maximum download size is checked after each chunk is delivered, so the chunk that crosses the limit is still delivered, and the total downloaded size can exceed the limit by up to one chunk. ## More Information ### Platform: unity See [`LocationMeshManager`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.Subsystems.LocationMeshManager) in the Unity API reference. ### Platform: swift See [`NSDKMeshDownloader`](https://www.nianticspatial.com/docs/api/swift/NSDK.class-NSDKMeshDownloader) in the Swift API reference. ### Platform: kotlin See [`MeshDownloaderSession`](https://www.nianticspatial.com/docs/api/kotlin/com.nianticspatial.nsdk.mesh.MeshDownloaderSession) in the Kotlin API reference.