# Guide users during localization Source: https://www.nianticspatial.com/docs/nsdk/how-to/vps2/guide_users_during_localization/ Coach users toward a successful localization by reacting to VPS2's localization request records. ## Prerequisites This guide assumes that you have already completed: - [Get started with VPS2](https://www.nianticspatial.com/docs/nsdk/how-to/vps2/adding_vps2/) ## Localization request records VPS2 exposes a stream of **localization request records** describing the cloud requests sent for universal localization and VPS map localization. Each record carries a request `type`, a `status`, an `error`, timing information, and the `frameId` of the camera frame it came from. Use this stream to tell your users what to do differently in the moment to increase the odds of successful localizations. It is not the way to read localization results; the VPS2 tracking state and asset tracking data report those. To investigate localization behaviour, use the [VPS Debugger](https://www.nianticspatial.com/docs/nsdk/how-to/vps2/analyze_tracking_issues/) instead. A record is written every time a request changes status, not once per request. ### Platform: swiftSubscribe to [`localizationRequestRecords`](https://www.nianticspatial.com/docs/api/swift/NSDK.class-NSDKVps2Session#property-localizationrequestrecords) ### Platform: kotlinCollect [`localizationRequestRecords`](https://www.nianticspatial.com/docs/api/kotlin/com.nianticspatial.nsdk.vps2.Vps2Session#property-localizationrequestrecords) ### Platform: unityRead the latest records with [`XRVps2Subsystem.GetLatestLocalizationRequestRecords()`](https://www.nianticspatial.com/docs/api/unity/NianticSpatial.NSDK.AR.XRSubsystems.XRVps2Subsystem.GetLatestLocalizationRequestRecords) to receive them. Each ### Platform: swift, kotlinemission ### Platform: unitycall reports **only what changed since the previous one**, not a running history. The `status` field describes the fate of the *request*; the `error` field describes the fate of the *localization*. Because a sent request produces both a `pending` record and a final one, a single request appears twice in the stream. Count `completed` and `failed` records, not `pending` ones, when tallying attempts. | Status | What it means | Records it produces | Localization outcome? | |---|---|---|---| | pending | The request was sent and is waiting for a response. | First of a pair. The final record shares its request identifier. If the session stops or the request is cancelled, this is the last record that identifier produces. | No -- the outcome is not known yet | | completed | The server answered. The `error` field says whether the localization itself succeeded: a successful localization is `completed` with no error, and a failed one is `completed` with an error set. | Final record of a pair. | Yes | | failed | The request never got a usable answer: network error, HTTP error, or a response that could not be read. Nothing was learned about the location. | Final record of a pair. | No -- it reports a transport failure, not a localization outcome | | frameRejected | Not sent, because the camera frame was unsuitable -- for example, pointing at the ground. Written during on-device frame checks. | Stands alone. Nothing was sent, so there is no `pending` record to match it. | No -- no request was made | | frameFlagged | The frame had a minor quality problem such as blurriness but was still sent. | Advisory, and stands alone: it has its own request identifier that no other record shares. The frame is still sent as a separate request with its own `pending` and final records; match on `frameId` to find them, and read the final record for the outcome. | No -- it describes the frame, not the outcome | ## Example: prompt users through blurry frames This is one example of coaching users from the localization request records: watch the stream and react to a record's `status` and `error`. It handles blurry frames, but the same pattern extends to other `frameFlagged` errors and to `frameRejected` reasons -- for example, prompting the user to aim at mapped surroundings when a frame is rejected for pointing at the ground. VPS2 can flag camera frames that are too blurry to localize well, so your app can coach the user while the device localizes to the Site. Frames blurred by fast device motion localize poorly, and the check lets you prompt the user to hold the device steady. The check is advisory: a flagged frame is still sent for localization, never rejected, so the same frame usually also produces its own `pending` and `completed` request records. Blur detection is off by default and runs only while VPS map localization requests are being sent, when the device is near a target with a processed VPS map. It does not run for universal localization. Enable it with the `Blur Detection Enabled` and `Min Sharpness Score` configuration fields. A frame is flagged when its sharpness score falls below the threshold; higher scores mean sharper images. The default threshold of `0` flags nothing; `1000` is a useful starting point and is the value the VPS2 sample apps use. Both fields are applied live, so you can change them while VPS2 is running. Flagged frames appear in the [localization request records](#localization-request-records) with status `frameFlagged` and error `ImageTooBlurry`. Show guidance such as "Image too blurry--hold the device steady" while flagged records keep arriving, then hide it about two seconds after they stop. ### Platform: unity Enable the check with the **Blur Detection Enabled** and **Min Sharpness Score** fields in the `ARVps2Manager` inspector, or set `BlurDetectionEnabled` and `MinSharpnessScore` from code. Then subscribe to `LocalizationRequestRecordAdded`: ```csharp _arVps2Manager.LocalizationRequestRecordAdded += OnLocalizationRequestRecord; private void OnLocalizationRequestRecord(XRVps2LocalizationRequestRecord record) { if (record.Status == Vps2LocalizationRequestStatus.FrameFlagged && record.ErrorCode == Vps2LocalizationError.ImageTooBlurry) { _blurWarningText.enabled = true; _lastBlurWarningTime = Time.unscaledTime; } } private void Update() { // Hide the hint a couple of seconds after flagged records stop arriving if (_blurWarningText.enabled && Time.unscaledTime - _lastBlurWarningTime > 2f) { _blurWarningText.enabled = false; } } ``` For the complete UI behavior, see the `VPS2Localization` sample scene and `VPS2AssetLocalizeDemo.cs`. ### Platform: swift Enable the check when configuring the session: ```swift let config = NSDKVps2Session.Configuration( blurDetectionEnabled: true, minSharpnessScore: 1000 ) try vps2Session.configure(with: config) ``` Then watch the `localizationRequestRecords` publisher for flagged records. A debounced subject can keep the hint visible while records arrive and hide it two seconds after they stop: ```swift private let blurFlagged = PassthroughSubject() vps2Session.localizationRequestRecords .receive(on: DispatchQueue.main) .sink { [weak self] records in for record in records where record.status == .frameFlagged && record.error == .imageTooBlurry { self?.blurFlagged.send() } } .store(in: &cancellables) blurFlagged .sink { [weak self] _ in self?.showBlurHint = true } .store(in: &cancellables) blurFlagged .debounce(for: .seconds(2), scheduler: DispatchQueue.main) .sink { [weak self] _ in self?.showBlurHint = false } .store(in: &cancellables) ``` For the complete UI behavior, see `VPS2View` and `VPS2ViewModel` in the NsdkSamples app. ### Platform: kotlin Enable the check when configuring the session: ```kotlin val config = Vps2Config( enableBlurDetection = true, minSharpnessScore = 1000f, ) vps2Session.configure(config) ``` Then collect flagged records from the `localizationRequestRecords` flow: ```kotlin vps2Session.localizationRequestRecords .filter { it.status == Vps2LocalizationRequestStatus.FRAME_FLAGGED && it.error == Vps2LocalizationError.IMAGE_TOO_BLURRY } .onEach { showBlurHint() } // e.g. show a toast or overlay text .launchIn(lifecycleScope) ``` For the complete UI behavior, see `VPS2Manager` in the NsdkSamples app, which shows the warning as a toast.