# Creating Device Maps Source: https://www.nianticspatial.com/docs/nsdk/how-to/vps/creating_device_maps/ # How to Create a Device Map ### Platform: unity > **Caution:** > > **Attention!** > > This feature is experimental and may not work as expected. For more information about it, see the [Device Mapping Feature page](https://www.nianticspatial.com/docs/nsdk/features/device_mapping/). Before you can use a device map in your AR application, you will need to create and save it. In this tutorial, we will go over how to create and save a device map in your Unity project, as well as some tips for how to get the most out of device mapping. ## Prerequisites You will need a Unity project with NSDK installed and a basic AR scene. For more information, see [Set up the Niantic SDK for Unity](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-niantic-sdk-for-unity) and [Set up a basic AR scene](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-a-basic-ar-scene). ## Creating a Device Map To create and store a device map: 1. In the **Hierarchy**, select the **XROrigin**, then, in the **Inspector**, click **Add Component** and add an `ARDeviceMappingManager`. 2. In the **Hierarchy**, right-click the root of your AR scene, then select **Create Empty**. Name the new object `DeviceMappingDemo`. 3. Select `DeviceMappingDemo` from the **Hierarchy**, then, in the **Inspector**, click **Add Component**. Search for and add a **New Script** to it, then name it `Mapper.cs`. 4. Open `Mapper.cs` and replace its contents with the following snippet: ```cs using System; using System.Collections; using System.IO; using NianticSpatial.NSDK.AR.Mapping; using UnityEngine; using UnityEngine.UI; public class Mapper : MonoBehaviour { } ``` 4. Add UI elements for the mapper: 1. Right-click in the **Hierarchy**, then open the **Create** menu and select **Button** from the **UI** sub-menu. This will create a **Canvas** element and add the button to it. 5. Add this snippet to the `Mapper` class to map for ten seconds when the button is pressed: ```cs [SerializeField] private ARDeviceMappingManager _deviceMappingManager; [SerializeField] private Button _startMappingButton; private void Start() { _startMappingButton.onClick.AddListener(OnStartMappingClicked); } // Set this function to be called when button is clicked public void OnStartMappingClicked() { StartCoroutine(RunMapping()); } private IEnumerator RunMapping() { // disable the button after clicking so that only one map is made at a time _startMappingButton.gameObject.SetActive(false); _deviceMappingManager.SetDeviceMap(new ARDeviceMap()); _deviceMappingManager.StartMapping(); // change this value to map for more or less time yield return new WaitForSeconds(10.0f); _deviceMappingManager.StopMapping(); _startMappingButton.gameObject.SetActive(true); } ``` > **Note:** > > Ten seconds is enough time to map a small area, such as a desk or couch. If you are mapping a larger area, increase the mapping time to compensate. 6. After the UI code, add an event listener for `ARDeviceMappingManager.DeviceMapFinalized` so that we know when to save the map: ```cs private const string MapFileName = "SerializedDeviceMapData"; private void Start() { // Add this below the previous section's code in Start() _deviceMappingManager.DeviceMapFinalized += OnDeviceMapFinalized; } private void OnDeviceMapFinalized(ARDeviceMap map) { if (map.HasValidMap()) { // final map generated. save to the file system var serializedDeviceMap = map.Serialize(); var path = Path.Combine(Application.persistentDataPath, MapFileName); File.WriteAllBytes(path, serializedDeviceMap); Debug.Log($"Map saved"); } else { Debug.LogError("Map was empty"); } } ``` > **Note:** > > A final device map will be generated after `StopMapping()` is called, but it will not be available until the `DeviceMapFinalized` event is invoked. Make sure to **wait for the finalized event** after the button is pressed! ## Optional: Show the Mesh While Mapping By default, `ARDeviceMappingManager` does not provide any visual feedback while mapping an area. By adding meshing to your project, you can visualize your map's coverage and get a sense of where you still need to cover. The mesh cannot exactly show where the device map was generated or its localizability, but it can provide a general idea of the map's area. To learn about how to add meshing to your Niantic SDK project, see [Creating a Mesh](https://www.nianticspatial.com/docs/nsdk/how-to/ar/meshing/adding_meshing/). ## Device Mapping Tips - Because device mapping relies on a single scan, localization and tracking accuracy depend heavily on how well you scan the area. If you remember one tip, let it be this: **if you are having localization issues, don't be afraid to re-scan!** - Lighting matters a lot when scanning an area. If the lighting during localization doesn't match the scan, you will need to re-scan. (This includes both outdoor and indoor lighting!) - For best results, follow these guidelines when scanning: - Focus on distinctive objects in the area that are unlikely to move between scanning and localization, such as furniture, household fixtures, or statues. - Avoid looking primarily at flat surfaces without color variation, such as the ground, grass, walls, and floors. - Move around while scanning! The more viewpoints you see an object from, the better the map of it will be. ## Complete Device Mapping Script If you are having trouble with your mapping script, compare it to the finished product here! #### Click here to reveal the final Mapper.cs script ```cs using System; using System.Collections; using System.IO; using NianticSpatial.NSDK.AR.Mapping; using UnityEngine; using UnityEngine.UI; public class Mapper : MonoBehaviour { public const string MapFileName = "SerializedDeviceMapData"; [SerializeField] private ARDeviceMappingManager _deviceMappingManager; [SerializeField] private Button _startMappingButton; private void Start() { _deviceMappingManager.DeviceMapFinalized += OnDeviceMapFinalized; _startMappingButton.onClick.AddListener(OnStartMappingClicked); } private void OnDeviceMapFinalized(ARDeviceMap map) { if (map.HasValidMap()) { // final map generated. save to the file system var serializedDeviceMap = map.Serialize(); var path = Path.Combine(Application.persistentDataPath, MapFileName); File.WriteAllBytes(path, serializedDeviceMap); Debug.Log($"Map saved"); } else { Debug.LogError("Map was empty"); } } // Set this function to be called when button is clicked public void OnStartMappingClicked() { StartCoroutine(RunMapping()); } private IEnumerator RunMapping() { _startMappingButton.gameObject.SetActive(false); _deviceMappingManager.SetDeviceMap(new ARDeviceMap()); _deviceMappingManager.StartMapping(); yield return new WaitForSeconds(10.0f); _deviceMappingManager.StopMapping(); _startMappingButton.gameObject.SetActive(true); } } ``` ## Next Steps Once you have created and saved a device map, move on to [How to Place Real-World Content Using a Device Map](https://www.nianticspatial.com/docs/nsdk/how-to/vps/using_device_maps/). ### Platform: swift > **Caution:** > > **Attention!** > > This feature is experimental and may not work as expected. For more information about it, see the [Device Mapping Feature page](https://www.nianticspatial.com/docs/nsdk/features/device_mapping/). Before you can use a device map in your AR application, you will need to create and save it. In this tutorial, we will go over how to create and save a device map in your Swift project, as well as some tips for how to get the most out of device mapping. For the full source code, please download the [NSDK Samples](https://github.com/nianticspatial/nsdk-samples-csharp) project. ## 1. Creating a Device Map Map construction will run while a `NsdkDeviceMappingSession` is active, and mapping data is automaticlly added to internal NSDK storage. An `NsdkMapStorage` instance can be used to interact with the storage system from your Swift app. ```swift mapStorage = nsdkSession.createMapStorage() mappingSession = nsdkSession.createDeviceMappingSession() mappingSession.startMapping() ``` ## 2. Check on Mapping Progress The time it takes to construct a valid map will vary depending on the environment and also the local device's processing power. The `NsdkMapStorage` class provides information about the state of the map being constructed. ```swift // Function call periodically private func checkForMapUpdates() { guard let buffer = mapStorage.mapUpdate() else { // No update available at this time, which is normal during mapping return } print("Received map update with \(buffer.dataSize) bytes") } ``` Once a map is valid, you will be able to place an anchor at its origin. ```swift if let rootAnchor = mappingSession.createRootAnchor() { // Can save the map now, or continue to grow the map } ``` ## Optional: Show the Mesh While Mapping When mapping larger areas, it can be useful to visualize the growing map's coverage and get a sense of where you still need to scan. Once a root anchor has been obtained, `NsdkMapStorage.extractMapMetadata(anchorPayload:map:)` can be used to obtain the map's feature points. In addition to the map buffers returned incrementally by `NsdkMapStorage.mapUpdate()` seen in the previous section, a buffer containing the entire map is returned by `NsdkMapStorage.mapData()`. ```swift do { let mapMetadata = try mapStorage.extractMapMetadata(anchorPayload: rootAnchor, map: buffer) guard let metadata = mapMetadata else { print("Failed to extract map metadata. Check logs for error.") return } // Visualize points described in mapMetadata.points } catch { print("Failed to extract map metadata. Error: \(error)") return } ``` Feature points are extracted relative to the pose of the anchor passed in for the `anchorPayload` argument. So in order to render the points on screen, you must first track the pose of that anchor. See the guide [Localizing and Tracking anchors with VPS](https://www.nianticspatial.com/docs/nsdk/how-to/vps/tracking_anchors_swift/#tracking-existing-anchors) for how to do that. Then get the location of feature points relative to that anchor: ```swift let points = mapMetadata.points for i in stride(from: 0, to: points.count, by: 3) { let center = SIMD3(points[i], points[i + 1], points[i + 2]) } ``` The visualization code can be found in full in the sample project linked at the top of this page. ## Device Mapping Tips - Because device mapping relies on a single scan, localization and tracking accuracy depend heavily on how well you scan the area. If you remember one tip, let it be this: **if you are having localization issues, don't be afraid to re-scan.** - Lighting matters a lot when scanning an area. If the lighting during localization doesn't match the scan, you will need to re-scan. (This includes both outdoor and indoor lighting.) - For best results, follow these guidelines when scanning: - Focus on distinctive objects in the area that are unlikely to move between scanning and localization, such as furniture, household fixtures, or statues. - Avoid looking primarily at flat surfaces without color variation, such as the ground, grass, walls, and floors. - Move around while scanning! The more viewpoints you see an object from, the better the map of it will be. ## Sharing the Map Serialized map data can be saved for use on the local device or even share with other devices. To get the map data: ```swift guard let buffer = mapStorage.mapData() else { print("Failed to get map from storage") return } ``` To add the map data to a fresh NSDK session: ```swift let mapData = try Data(contentsOf: mapDataURL) let mapBuffer = NsdkBuffer(data: mapData) do { try mapStorage.addMap(map: mapBuffer) } catch { print("Failed to add map to storage: \(error)") } ``` Once a device map has been added to storage, the `NsdkVpsSession` APIs can be used to create and track anchors to the map. ### Platform: kotlin Before you can use a device map in your AR application, you will need to create and save it. In this tutorial, we will go over how to create and save a device map in your Kotlin project, as well as some tips for how to get the most out of device mapping. ## Prerequisites Ensure your Kotlin project is set up for Niantic SDK use. For more information, see [Set up the NSDK in Kotlin](https://www.nianticspatial.com/docs/nsdk/setup/#set-up-the-nsdk-in-kotlin). This example also assumes you are familiar with how to set up a VPS session (see [Getting Started with VPS](https://www.nianticspatial.com/docs/nsdk/how-to/vps/adding_vps/)). ## Creating a Device Map Before you can begin creating a device map, you will need to set up and configure the VPS session, as well as acquiring an instance of the `DeviceMappingSession` and `MappingStorageSession` from the NsdkSession. The VPS session must be configured to enable Device Map Localization in order to actually create a Device Map. ```kotlin // Acquire the NSDK sessions we will need val nsdkSession = NSDKSession(apiKey = "your-api-key") val vpsSession = nsdkSession.vps.acquire() val mappingSession = nsdkSession.mapping.acquire() val mapStoreSession = nsdkSession.mapStore.acquire() // Variables to store the data for the root map anchor, which you will need in order to place content within a Device Map var rootAnchorBase64 by mutableStateOf(null) var vpsRootAnchorId by mutableStateOf(null) // Example config, see documentation for all possible config options. // The important setting here is enableDeviceMapLocalization = true val config = VPSConfig( enableContinuousLocalization = true, enableDeviceMapLocalization = true ) try { vpsSession.configure(config) } catch (e: NsdkStatusException) { Log.e("DeviceMapping", "Configuration failed: ${e.message}") } // Lastly, start the mapping session mappingSession.start() ``` Now you can begin the recording by calling `mappingSession.startCreating()`, and end the recording by calling `mappingSession.stopCreating()`. > **Note:** > > Each time you want to record a new Device Map, remember to reset the VPS session. This will ensure the VPS anchors are localizing to the current map data. > > Additionally, it takes a few frames for the VPS session to begin successfully tracking the mapped space, so you will need to confirm the root anchor has been created before the map can be used. This coroutine will begin mapping, record for 10 seconds, then automatically stop: ```kotlin lifecycleScope.launch { // Begin the actual recording of the Device Map data mappingSession.startCreating() // Wait for the root anchor to be set up, and VPS tracking to succeed while(rootAnchorBase64 == null) { rootAnchorBase64 = mapStoreSession.createRootAnchor() if (rootAnchorBase64 == null) continue vpsSession.start() try { vpsRootAnchorId = vpsSession.trackAnchor(rootAnchorBase64!!) Log.i("DeviceMapping", "Started tracking anchor: ${vpsRootAnchorId}") } catch (e: NsdkStatusException) { Log.e("DeviceMapping", "Failed to track anchor ($e)") } } // Delay to allow the mapping session to record for 10 seconds delay(10000) // Stop the recording mappingSession.stopCreating() // Reset VPS and clear the root anchor data for the next recording vpsRootAnchorId?.let { vpsSession.removeAnchor(it) } vpsSession.stop() vpsRootAnchorId = null rootAnchorBase64 = null } ``` Once you are finished with your mapping session, be sure to clean up both the Mapping and VPS sessions by stopping and closing them: ```kotlin mappingSession.stop() mappingSession.close() vpsSession.stop() vpsSession.close() ``` ## Saving Device Maps Once you have finished recording, Device Map data in the `MappingStorageSession` can be saved to disk, which can be re-loaded at a later time or shared to other devices. To save this data, make sure you are not currently scanning, and use `mapStoreSession.getData()` to get the current Device Map data as a ByteArray, which can then be written to a file. The following function will save the map to a file with provided file name: ```kotlin // Saves the Device Map data to the application's default file directory. Note that fileName should include the extension if any is needed. fun saveMapToDisk(context: Context, fileName: String): Boolean { val mapData = mapStoreSession.getData() if (mapData == null) { Log.e("DeviceMapping", "Tried to save map but map has no data.") return false } try { val file = File(context.filesDir, fileName) FileOutputStream(file).use { output -> output.write(mapData) } Log.i("DeviceMapping", "Map (${mapData.size} bytes) saved successfully to: ${file.absolutePath}") return true } catch (e: Exception) { errorMessage = Log.e("DeviceMapping", "Failed to save map: ${e.message}") return false } } ``` ## Device Mapping Tips - Because device mapping relies on a single scan, localization and tracking accuracy depend heavily on how well you scan the area. If you remember one tip, let it be this: **if you are having localization issues, don't be afraid to re-scan!** - Lighting matters a lot when scanning an area. If the lighting during localization doesn't match the scan, you will need to re-scan. (This includes both outdoor and indoor lighting!) - For best results, follow these guidelines when scanning: - Focus on distinctive objects in the area that are unlikely to move between scanning and localization, such as furniture, household fixtures, or statues. - Avoid looking primarily at flat surfaces without color variation, such as the ground, grass, walls, and floors. - Move around while scanning! The more viewpoints you see an object from, the better the map of it will be. ## Next Steps Once you have created and saved a device map, move on to [How to Place Real-World Content Using a Device Map](https://www.nianticspatial.com/docs/nsdk/how-to/vps/using_device_maps/).