Evaluation Leaderboard
Click a column to sort. Bars indicate relative performance per metric.
Public benchmark
All submissions
Methods are ranked by AUC (VCRE < 45px) by default. Lower-is-better metrics sort ascending first; higher-is-better metrics sort descending first.
Methods evaluated with a single reference image and a single query frame.
| Method | AUC (VCRE < 45px)↕ | AUC (VCRE < 90px)↕ | Precision (VCRE < 45px)↕ | Precision (VCRE < 90px)↕ | Median Reproj. Error (px)↕ | AUC (Err < 25cm, 5°)↕ | Precision (Err < 25cm, 5°)↕ | Median Trans. Error (m)↕ | Median Rot. Error (°)↕ | Estimates for % of frames↕ |
|---|---|---|---|---|---|---|---|---|---|---|
|
0.840
|
0.943
|
64.4%
|
81.1%
|
47.43
|
0.768
|
55.9%
|
0.389
|
4.564
|
99.6%
|
|
|
0.834
|
0.939
|
63.8%
|
79.5%
|
49.70
|
0.754
|
53.8%
|
0.411
|
4.596
|
99.9%
|
|
|
David Nordström, David Nordström, Xinyue Zhang, Thibaut Loiseau, Vincent Lepetit, Fredrik Kahl · Chalmers University of TechnologyCode
|
0.817
|
0.945
|
60.2%
|
79.2%
|
55.37
|
0.755
|
50.2%
|
0.440
|
5.041
|
100.0%
|
|
||||||||||
|
0.817
|
0.933
|
63.0%
|
79.3%
|
48.79
|
0.740
|
54.7%
|
0.368
|
2.202
|
99.4%
|
|
|
||||||||||
|
0.796
|
0.925
|
57.2%
|
75.7%
|
56.70
|
0.754
|
48.8%
|
0.500
|
4.358
|
99.2%
|
|
|
||||||||||
|
Shigeki Kobayashi · GO Drive Inc.
|
0.736
|
0.870
|
48.2%
|
63.2%
|
80.84
|
0.662
|
39.8%
|
0.834
|
11.89
|
100.0%
|
|
||||||||||
|
Lili Zhao, Zhili Liu, Lei Yang, Meng Guo · China Mobile Research Institute, N/A, China Mobile Research Institute, China Mobile Research InstitutePaper
|
0.681
|
0.796
|
39.9%
|
51.4%
|
125.6
|
0.593
|
31.9%
|
1.754
|
31.22
|
100.0%
|
|
||||||||||
|
Mingyu Xiao ; Runze Chen ; Haiyong Luo ; Fang Zhao;Juan Wang;XuePeng Ma · Beijing University of Posts and Telecommunications;Research Center for Ubiquitous Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences;Shouguang Cheng Zhi Feng Xing Technology Co., Ltd.Paper
|
0.656
|
0.849
|
45.3%
|
65.0%
|
79.96
|
0.469
|
28.6%
|
0.828
|
11.74
|
99.0%
|
|
||||||||||
|
David Nordström · Chalmers University of Technology
|
0.641
|
0.780
|
36.0%
|
49.4%
|
126.0
|
0.551
|
27.5%
|
1.693
|
26.71
|
98.5%
|
|
||||||||||
|
RoMa: Edstedt et al. - MicKey: Barroso-Laguna et al. · RoMa: Linköping University. - MicKey: Niantic.
|
0.604
|
0.734
|
37.8%
|
52.4%
|
111.9
|
0.546
|
31.4%
|
1.177
|
15.60
|
99.9%
|
|
||||||||||
|
0.572
|
0.748
|
31.2%
|
49.3%
|
129.5
|
0.325
|
13.3%
|
1.657
|
27.26
|
100.0%
|
|
|
0.558
|
0.741
|
30.1%
|
49.2%
|
126.9
|
0.283
|
12.0%
|
1.591
|
26.00
|
100.0%
|
|
|
SuperGlue: Sarlie et al. - MicKey: Barroso-Laguna et al. · SuperGlue: ETH Zurich, Magic Leap, Inc. - MicKey: Niantic
|
0.556
|
0.711
|
29.8%
|
43.0%
|
139.9
|
0.490
|
23.5%
|
1.696
|
26.13
|
98.5%
|
|
||||||||||
|
LoFTR: Sun et al. - MicKey: Barroso-Laguna et al. · LoFTR: Zhejiang University, SenseTime Research - MicKey: Niantic
|
0.550
|
0.715
|
27.2%
|
40.7%
|
155.0
|
0.467
|
20.3%
|
1.922
|
33.60
|
100.0%
|
|
||||||||||
|
Hitesh Jain, Sagar Verma · Granular AIPaper
|
0.548
|
0.720
|
28.0%
|
45.5%
|
142.0
|
0.273
|
10.8%
|
1.842
|
30.78
|
100.0%
|
|
||||||||||
|
0.481
|
0.680
|
25.3%
|
44.2%
|
137.1
|
0.392
|
17.7%
|
1.482
|
17.26
|
100.0%
|
|
|
||||||||||
|
0.459
|
0.698
|
28.7%
|
50.4%
|
115.8
|
0.394
|
21.4%
|
0.989
|
7.131
|
99.6%
|
|
|
0.456
|
0.668
|
25.8%
|
44.2%
|
135.5
|
0.351
|
17.1%
|
1.498
|
17.28
|
100.0%
|
|
|
||||||||||
|
Johan Edstedt · Linköping University
|
0.437
|
0.670
|
26.4%
|
45.7%
|
128.9
|
0.408
|
22.9%
|
1.235
|
11.19
|
100.0%
|
|
||||||||||
|
0.420
|
0.635
|
19.4%
|
34.4%
|
165.1
|
0.295
|
11.1%
|
2.238
|
37.82
|
100.0%
|
|
|
||||||||||
|
0.420
|
0.631
|
19.5%
|
34.0%
|
167.5
|
0.278
|
10.5%
|
2.326
|
39.52
|
100.0%
|
|
|
||||||||||
|
Hongkai Chen, Zixin Luo, Lei Zhou, Yurun Tian, Mingmin Zhen, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan · Apple Inc, HKUSTPaper
|
0.414
|
0.641
|
20.8%
|
37.0%
|
161.8
|
0.361
|
16.3%
|
1.901
|
29.25
|
100.0%
|
|
||||||||||
|
|
0.397
|
0.618
|
18.2%
|
33.5%
|
166.8
|
0.269
|
9.8%
|
2.312
|
39.42
|
100.0%
|
|
||||||||||
|
|
0.387
|
0.615
|
18.8%
|
34.7%
|
167.6
|
0.346
|
15.5%
|
1.987
|
30.58
|
100.0%
|
|
||||||||||
|
|
0.385
|
0.602
|
20.6%
|
36.2%
|
160.3
|
0.346
|
16.8%
|
1.881
|
25.44
|
100.0%
|
|
||||||||||
|
|
0.378
|
0.599
|
19.6%
|
36.0%
|
157.0
|
0.253
|
10.7%
|
2.108
|
32.58
|
100.0%
|
|
||||||||||
|
Shigeki Kobayashi · GO Drive Inc.
|
0.373
|
0.541
|
37.3%
|
54.1%
|
106.2
|
0.220
|
22.0%
|
1.360
|
18.20
|
100.0%
|
|
||||||||||
|
Shigeki Kobayashi · GO Drive Inc.
|
0.369
|
0.533
|
36.9%
|
53.3%
|
108.1
|
0.213
|
21.3%
|
1.400
|
19.30
|
100.0%
|
|
||||||||||
|
Shigeki Kobayashi · GO Drive Inc.
|
0.360
|
0.526
|
36.0%
|
52.6%
|
113.1
|
0.187
|
18.7%
|
1.470
|
20.70
|
100.0%
|
|
||||||||||
|
Map-free leaderboard baseline · Niantic - Map-free leaderboard baseline
|
0.355
|
0.527
|
19.5%
|
33.2%
|
138.8
|
0.314
|
15.9%
|
1.445
|
18.52
|
78.0%
|
|
||||||||||
|
Michał J. Tyszkiewicz, Pascal Fua, Eduard Trulls · École Polytechnique Fédérale de Lausanne (EPFL), Google Research, Zurich
|
0.346
|
0.536
|
15.1%
|
26.9%
|
208.2
|
0.264
|
10.2%
|
2.593
|
51.96
|
100.0%
|
|
||||||||||
|
Shigeki Kobayashi · GO Drive Inc.
|
0.327
|
0.502
|
32.7%
|
50.2%
|
118.9
|
0.153
|
15.3%
|
1.630
|
23.00
|
100.0%
|
|
||||||||||
|
0.325
|
0.531
|
16.9%
|
31.2%
|
167.4
|
0.265
|
12.5%
|
2.021
|
30.30
|
88.0%
|
|
|
|
0.313
|
0.505
|
13.7%
|
25.0%
|
222.9
|
0.253
|
10.3%
|
2.930
|
61.43
|
100.0%
|
|
||||||||||
|
|
0.299
|
0.468
|
14.3%
|
25.1%
|
192.0
|
0.191
|
7.8%
|
3.354
|
63.74
|
100.0%
|
|
||||||||||
|
0.192
|
0.312
|
9.8%
|
18.1%
|
176.4
|
0.157
|
7.3%
|
2.208
|
33.81
|
52.2%
|
|
|
|
0.181
|
0.355
|
18.1%
|
35.5%
|
166.4
|
0.105
|
10.5%
|
1.831
|
23.27
|
100.0%
|
|
||||||||||
|
|
0.181
|
0.387
|
18.1%
|
38.7%
|
148.7
|
0.061
|
6.1%
|
1.698
|
22.94
|
100.0%
|
|
||||||||||
|
|
0.156
|
0.402
|
15.6%
|
40.2%
|
147.1
|
0.061
|
6.1%
|
1.682
|
22.52
|
100.0%
|
|
||||||||||
Methods evaluated using sequences of multiple query frames (9-frame window).
| Method | AUC (VCRE < 45px)↕ | AUC (VCRE < 90px)↕ | Precision (VCRE < 45px)↕ | Precision (VCRE < 90px)↕ | Median Reproj. Error (px)↕ | AUC (Err < 25cm, 5°)↕ | Precision (Err < 25cm, 5°)↕ | Median Trans. Error (m)↕ | Median Rot. Error (°)↕ | Estimates for % of frames↕ |
|---|---|---|---|---|---|---|---|---|---|---|
|
0.572
|
0.752
|
31.3%
|
49.6%
|
128.9
|
0.326
|
13.6%
|
1.656
|
26.92
|
100.0%
|
|
|
||||||||||
|
Mingyu Xiao ; Runze Chen ; Haiyong Luo ; Fang Zhao · Beijing University of Posts and TelecommunicationsPaper
|
0.387
|
0.721
|
25.2%
|
53.1%
|
105.8
|
0.190
|
10.8%
|
0.951
|
15.10
|
99.1%
|
|
||||||||||
Click a column header to sort. Bars indicate relative performance per metric. Entries update automatically from public benchmark submissions.