Geo-localization based on CNN feature matching

Abstract

A geo-localization method is proposed for military and civilian applications, which is used when no global navigation satellite system (GNSS) information is available. The open graphics library (OpenGL) is used to build a three-dimensional geographic model of the test area using digital elevation model (DEM) data, and the skyline can thus be extracted with the model to form a database. Then, MultiSkip DeepLab (MS-DeepLab), a fully convolutional semantic segmentation network with multiple skip structures, is proposed to extract the skyline from the query image. Finally, a matching model based on convolutional neural network (CNN) feature is adopted to calculate the similarity between the skyline features of the query image and the DEM database to realize automatic geo-localization. The experiments are conducted at a 202.6 km² test site in north-eastern Changsha, China. 50 test points are selected to verify the effectiveness of the proposed method, and an excellent result with an average positioning error of 49.29 m is obtained.

References

[1] TalluriR, AggarwalJ. Position estimation for an autonomous mobile robot in an outdoor environment[J]. IEEE transactions on robotics and automation, 1992, 8(5):573-584

[2] SteinF, MedioniG. Map-based localization using the panoramic horizon[J]. IEEE transactions on robotics and automation, 1996, 11(6):892-896

[3] NAVAL P C. Camera pose estimation by alignment from a single mountain image[EB/OL]. (2010-07) [2021-10-11]. http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=87E95F9B1E103A9679DA9009-C4CF1-F74?doi=10.1.1.102.9949&rep=rep1&type=pdf.

[4] NAVAL P C, MUKUNOKI M, MINOH M, et al. Estimating camera position and orientation from geographical map and mountain image[EB/OL]. (1997-04) [2021-10-11]. http://citeseerx.ist.psu.edu/viewdoc/Download?doi=10.1.1.14.3619&rep=rep1&type=pdf.

[5] WOO J, SON K, LI T, et al. Vision-based UAV navigation in mountain area[C]//Proceedings of the IAPR Conference on Machine Vision Applications (IAPR MVA), May 16–18, 2007, Tokyo, Japan. 2007: 236–239.

[6] BaatzG, SaurerO, KoserK, et al.. Large scale visual geo-localization of images in mountainous terrain, 2012, Berlin, Heidelberg, Springer-Verlag: 517-530

[7] TzengE, ZhaiA, ClementsM, et al.. User-driven geolocation of untagged desert imagery using digital elevation models, 2013, New York, IEEE: 237-244

[8] PorziL, BuloS R, ValigiP, et al.. Learning contours for automatic annotations of mountains pictures on smartphone, 2014, New York, ACM: 131-136

[9] BaboudL, CadikM, EisemannE, et al.. Automatic photo-to-terrain alignment for the annotation of mountain picture, 2011, New York, IEEE: 41-48

[10] HammoudR I, KuzdebaS A, BerardB, et al.. Overhead-based image and video geo-localization framework, 2013, New York, IEEE: 320-327

[11] ChenY, QianG, GundaK, et al.. Camera geolocation from mountain images, 2015, New York, IEEE: 1587-1596

[12] SaurerO, BaatzG, KoserK, et al.. Image based geo-localization in the Alps[J]. International journal of computer vision, 2016, 116(3):213-225

[13] GrelssonB, RobinsonA, FelsbergM, et al.. GNSS-level accurate camera localization with HorizonNet[J]. Journal of field robotics, 2020, 37(6):951-971

[14] ChiodiniS, PertileM, DebeiS, et al.. Mars rovers localization by matching local horizon to surface digital elevation models, 2017, New York, IEEE

[15] FukudaS, NakataniS, NishiyamaM, et al.. Geo-localization using ridgeline features extracted from 360-degree images of sand dunes, 2020, New York, IEEE: 621-627

[16] DANIEL S T, LI M, MARGARET H. Face recognition: from traditional to deep learning methods[EB/OL]. (2019-11-15) [2021-10-11]. https://arxiv.org/pdf/1811.00116.pdf.

[17] RonnebergerO, FischerP, BroxT. U-net: convolutional networks for biomedical image segmentation, 2015, Berlin, Heidelberg, Springer-Verlag: 234-241

[18] ChenL C, ZhuY, PapandreouG, et al.. Encoder-decoder with atrous separable convolution for semantic image segmentation, 2018, Berlin, Heidelberg, Springer-Verlag: 833-851

[19] LongJ, ShelhamerE, DarrellT. Fully convolutional networks for semantic segmentation, 2015, New York, IEEE: 3431-3440

[20] ChenL C, PapandeouG, KokkinosI, et al.. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40: 834-848

[21] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. (2014-07-15) [2021-10-11]. https://arxiv.org/pdf/1409.1556v6.pdf.

[22] BrejchaJ, CadikM. State-of-the-art in visual geo-localization[J]. Pattern analysis and applications, 2017, 20: 613-637