# Release Notes

Source: https://www.nianticspatial.com/docs/nsdk/release_notes/

# Release Notes 4.4.0

## Highlights

- [Stop localizing and convert AR poses to geolocations](#new-vps2-stop-localize-and-pose-to-geolocation-apis)
- [VPS2 site localization and asset tracking in Unity](#vps2-site-localization-and-asset-tracking-in-unity)
- [Starting VPS2 is optional](#starting-vps2-is-optional)
- [Site assets expose their areas](#site-assets-expose-their-areas)
- [Georeferenced mesh downloads](#georeferenced-mesh-downloads)

## Features/Fixes

### New VPS2 stop-localize and pose-to-geolocation APIs

Developers can now stop localizing against a requested site, or every requested site, using `stopLocalize(siteId:)` in [Swift](https://www.nianticspatial.com/docs/api/swift/NSDK.class-NSDKVps2Session), `stopLocalize(siteId)` in [Kotlin](https://www.nianticspatial.com/docs/api/kotlin/com.nianticspatial.nsdk.vps2.Vps2Session), or `TryStopLocalize(...)` in [Unity](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2Manager). Stopping removes the site's assets, their anchors, and their map data from the localization set.

Developers can also convert any AR pose to a geolocation using the latest VPS2 localization with `getGeolocation(pose:)` in [Swift](https://www.nianticspatial.com/docs/api/swift/NSDK.class-NSDKVps2Session), `getGeolocation(pose)` in [Kotlin](https://www.nianticspatial.com/docs/api/kotlin/com.nianticspatial.nsdk.vps2.Vps2Session), or `TryGetGeolocation(...)` in [Unity](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2Manager). This is the inverse of `getPose`; see [Getting VPS2 geoposition](https://www.nianticspatial.com/docs/nsdk/how-to/vps2/getting_vps2_geoposition/).

### VPS2 site localization and asset tracking in Unity

Unity developers can now localize to a site or to nearby sites with `TryLocalize(...)` on [`ARVps2Manager`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2Manager), and follow a VPS asset's origin in world space through the new [`ARVps2AssetManager`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2AssetManager) trackables. Anchors move to the new [`ARVps2AnchorManager`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2AnchorManager); see [Breaking Changes](#breaking-changes) and [Placing virtual content](https://www.nianticspatial.com/docs/nsdk/how-to/vps2/placing_virtual_content/).

### Starting VPS2 is optional

VPS2 now starts itself when an app calls a site or coordinate localize function, so an explicit `start()` is only needed to collect sensor data before a localization target is known. Asset tracking data is served for nearby assets before the device localizes against them, at reduced accuracy, and keeps being served after the device localizes elsewhere. See [Adding VPS2](https://www.nianticspatial.com/docs/nsdk/how-to/vps2/adding_vps2/).

### Site assets expose their areas

A VPS map too large for one mapping run is published as several areas. Sites now report every area of an asset through `AssetInfo.areas` and the new `AssetArea` type in Swift, Kotlin, and Unity, each with its own job status, anchor, nodes, and geographic bounds. VPS2 targets each area by its bounds, and the localized asset reports the area the device localized in.

### Georeferenced mesh downloads

Downloaded mesh chunks now carry a georeference, the geodetic origin and transform that place the mesh geographically, through `geoReference` on [`MeshDownloaderChunk`](https://www.nianticspatial.com/docs/api/swift/NSDK.struct-MeshDownloaderChunk) in Swift, [`MeshDownloaderData`](https://www.nianticspatial.com/docs/api/kotlin/com.nianticspatial.nsdk.MeshDownloaderData) in Kotlin, and the `MeshGeoReference` component in Unity. Each chunk also reports the VPS map asset ID and area ID it belongs to, and downloads by asset ID now cover every area of the map. See [Mesh download](https://www.nianticspatial.com/docs/nsdk/how-to/vps/mesh_download/).

## Changelog

### Added

- Added `stopLocalize(siteId)` to stop localizing against a requested site, or every site when null (`ARDK_VPS2_StopLocalize`, Unity `TryStopLocalize`).
- Added `getGeolocation(pose)` to convert an AR pose to a geolocation using the latest VPS2 localization (`ARDK_VPS2_GetGeolocation`, Unity `TryGetGeolocation`).
- Added `geolocation` to `Vps2AssetTrackingData`, reporting the asset origin's geolocation when the asset is georeferenced.
- Swift, Kotlin: Added `areaId` to `Vps2LocalizedAsset` and `Vps2AssetTrackingData`, naming the area of a split asset the device localized in.
- Added `AssetInfo.areas` and `AssetArea`, describing every area of a site asset with its job status, anchor, nodes, and bounds.
- Added the `Published` and `Draft` asset deployment types and the `Canceled` pipeline job status.
- Added `geoReference` to downloaded mesh chunks, carrying the geodetic origin and transform needed to place the mesh geographically.
- Added the VPS map asset ID and area ID to each downloaded mesh chunk (`vpsMapAssetId`, `areaId`; Unity `MeshChunkInfo`).
- Unity: Added `ARVps2Manager.TryLocalize` overloads for site and coordinate VPS2 localization.
- Unity: Added `ARVps2AssetManager`, `ARVps2Asset`, and `ARVps2Manager.TryGetAssetTrackingData` to follow VPS asset origins in world space.
- Kotlin: Added `captureDepth` to `createLiveDataSource` to feed ARCore Depth API images to NSDK as platform depth.
- Native: Added LiDAR depth and confidence to playback of datasets that record them.
- Added a Unity guide for VPS2 site localization and placing content on localized assets.

### Changed

- VPS2 site and coordinate localize calls now start VPS2 themselves; calling `start()` first is optional.
- Asset tracking data is now served for nearby assets before localization, at reduced accuracy, and after the device localizes elsewhere.
- VPS2 now filters map nodes by an asset's geographic bounds, using `anchorDistanceGateMeters` only for assets without bounds.
- Mesh download by VPS map asset ID now downloads every area of a map published in several areas.
- `AssetInfo.pipelineJobId` and `sourceScanIds` are now always empty, and `assetStatus` is always unspecified.
- Sessions that use only platform depth, such as meshing with LiDAR, no longer download or initialize the depth model.
- Native: Playback now loads frame images on demand instead of holding the whole dataset in memory.
- Kotlin: Maven artifacts are now served per release line, from the `nsdk-MAJOR.MINOR-release` repository URL.

### Deprecated

- Deprecated identifying maps by anchor payload, including `LocalizationTarget.defaultAnchorPayload`; localize by site ID or coordinate instead.
- Deprecated `AssetInfo` typed asset data, `assetStatus`, `pipelineJobId`, `pipelineJobStatus`, and `sourceScanIds`; read `AssetInfo.areas` instead.

### Fixed

- Fixed VPS2 discarding cloud localizations whose responses arrived after the device's tracking node changed.
- Fixed VPS2 dropping from precise to coarse tracking when a new tracking node had not yet reached the map graph.
- Fixed VPS2 device geolocation reporting a fabricated position and heading before the first camera pose arrived.
- Fixed a crash when reading depth or scene segmentation results while the feature was stopping.
- Fixed depth and scene segmentation acting on a stale model initialization after being stopped and restarted.
- Fixed a crash at shutdown when NSDK logged after its logger was torn down.
- Kotlin: Fixed incorrect colors in playback frames converted from bitmaps to YUV.
- Kotlin: Fixed playback content misaligning with the camera image when the viewport aspect ratio differed from the image.

## Breaking Changes

- All platforms: The VPS2 config fields `assetTargetingEnterMarginMeters` and `assetTargetingExitMarginMeters` are renamed to `areaTargetingEnterMarginMeters` and `areaTargetingExitMarginMeters`. Native and Python callers must use `area_targeting_enter_margin_meters` and `area_targeting_exit_margin_meters`.
- Unity: [`ARVps2Manager.TryGetLatestLocalization`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2Manager) now returns `true` whenever the subsystem exists, including while the tracking state is `Unavailable`. Callers that used the return value to decide whether the geolocation fields are valid must check `XRVps2Localization.TrackingState` instead.
- Unity: `ARVps2Manager` is no longer an `ARTrackableManager`; anchors are served by the new [`ARVps2AnchorManager`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.VPS2.ARVps2AnchorManager) and `XRVps2AnchorSubsystem`. Anchor calls through `ARVps2Manager` still compile, but casts to `ARTrackableManager` and subclass `Update()` overrides must use the `AnchorManager` property instead. Scenes must add `ARVps2AnchorManager` in the editor and re-wire the anchor prefab and `trackablesChanged` listeners on it, and callers of the anchor methods on `XRVps2Subsystem` must move to `XRVps2AnchorSubsystem`.

## Known Issues

- Importing the NSDK UPM in a project started from Universal 3D template might cause a few build errors reported in the console. Restarting the editor should fix this.
- On iPad Pro running iOS 17 in the Unity External Sample, rotating the device from landscape to portrait may cause a soft lock and screen flickering.