Skip to main content

Map-free Visual Relocalization | Niantic Spatial, Inc.
HomeDatasetLeaderboardSubmit
Account

Map-free Visual Relocalization

Metric pose relative to a single image. ECCV 2022.

Abstract

Relocalizing in a scene from a single reference image

Can we relocalize in a scene represented by a single reference image? Standard visual relocalization requires hundreds of images and scale calibration to build a scene-specific 3D map. In contrast, we propose Map-free Relocalization, i.e., using only one photo of a scene to enable instant, metric scaled relocalization.

Existing datasets are not suitable to benchmark map-free relocalization, due to their focus on large scenes or their limited variability. Thus, we have constructed a new dataset of 655 small places of interest, such as sculptures, murals and fountains, collected worldwide. Each place comes with a reference image to serve as a relocalization anchor, and dozens of query images with known, metric camera poses.

The dataset features changing conditions, stark viewpoint changes, high variability across places, and queries with low to no visual overlap with the reference image. We identify two viable families of existing methods to provide baseline results: relative pose regression, and feature matching combined with single-image depth prediction. While these methods show reasonable performance on some favorable scenes, map-free relocalization proves to be a challenge that requires new, innovative solutions.

Dataset

655 places, collected worldwide

We make available a dataset of 655 places, collected by non-expert users worldwide. The full dataset is publicly available on the dataset page.

Animated image showing example scenes from the dataset.
A sample of scenes from the dataset — sculptures, murals, fountains and other small places of interest.
Download dataset

Benchmark

A new metric for map-free relocalization

We propose new metrics based on the reprojection error of virtual objects. To successfully compete in our benchmark, methods need to be precise but also provide a robust confidence estimate of their predictions.

You can submit to our public leaderboard and compare against the state of the art.

View LeaderboardSubmit

Citation

BibTeX

Please cite our paper if you use our dataset or code.

@inproceedings{arnold2022mapfree,
  title={Map-free Visual Relocalization: Metric Pose Relative to a Single Image},
  author={Arnold, Eduardo and Wynn, Jamie and Vicente, Sara and Garcia-Hernando, Guillermo and Monszpart, {\'{A}}ron and Prisacariu, Victor Adrian and Turmukhambetov, Daniyar and Brachmann, Eric},
  booktitle={ECCV},
  year={2022},
}

To report any issues with the dataset, the leaderboard, or for any other questions relating to this research project, please contact mapfree-reloc@nianticspatial.com.