Fast image reconstruction network in image stitching

Abstract

Compared with the traditional feature-based image stitching algorithm, the free-view image stitching algorithm based on deep learning has the advantages of fast stitching speed and good effect. However, these algorithms still cannot achieve real-time splicing speed. For the image reconstruction stage, we redesign a new fast image reconstruction network. This network is designed based on ShuffleNet, and the new network structure and loss function will reduce the time required for image reconstruction. In addition, this network can also reduce the performance loss after the network is lightweight. It is proved by experiments that the fast image reconstruction network can realize real-time high-resolution free-view image reconstruction.

References

[1] AndersonR, GallupD, BarronJ T, et al.. Jump: virtual reality video[J]. ACM transactions on graphics (TOG), 2016, 35(6):1-13

[2] LiJ, ZhaoY, YeW, et al.. Attentive deep stitching and quality assessment for 360° omnidirectional images[J]. IEEE journal of selected topics in signal processing, 2019, 14(1):209-221

[3] WangL, YuW, LiB. Multi-scenes image stitching based on autonomous driving[C], 2020, New York, IEEE: 694-6981

[4] LiD, HeQ, LiuC, et al.. Medical image stitching using parallel sift detection and transformation fitting by particle swarm optimization[J]. Journal of medical imaging and health informatics, 2017, 7(6):1139-1148

[5] GaddamV R, RieglerM, EgR, et al.. Tiling in interactive panoramic video: approaches and evaluation[J]. IEEE transactions on multimedia, 2016, 18(9):1819-1831

[6] HeK, ChenX, XieS, et al.. Masked autoencoders are scalable vision learners[C], 2022, New York, IEEE: 16000-16009

[7] FengF, GanJ A, NongJ, et al.. Data transmission with up to 100 orbital angular momentum modes via commercial multi-mode fiber and parallel neural networks[J]. Optics express, 2022, 30(13):23149-23162

[8] FengF, GanJ, ChenP F, et al.. AI-assisted spectrometer based on multi-mode optical fiber speckle patterns[J]. Optics communications, 2022, 522: 128675

[9] ZhangF, LiuF. Casual stereoscopic panorama stitching[C], 2015, New York, IEEE: 2002-2010

[10] ZhangX, ZhouX, LinM, et al.. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C], 2018, New York, IEEE: 6848-6856

[11] HeK, ZhangX, RenS, et al.. Deep residual learning for image recognition[C], 2016, New York, IEEE: 770-778

[12] HoangV D, TranD P, NhuN G, et al.. Deep feature extraction for panoramic image stitching[C], 2020, Berlin, Heidelberg, Springer International Publishing: 141-151

[13] ShenC, JiX, MiaoC. Real-time image stitching with convolutional neural networks[C], 2019, New York, IEEE: 192-197

[14] GaoJ, KimS J, BrownM S. Constructing image panoramas using dual-homography warping[C], 2011, New York, IEEE: 49-56

[15] ZaragozaJ, ChinT J, BrownM S, et al.. As-projective-as-possible image stitching with moving DLT[C], 2013, New York, IEEE: 2339-2346

[16] LiJ, WangZ, LaiS, et al.. Parallax-tolerant image stitching based on robust elastic warping[J]. IEEE transactions on multimedia, 2017, 20(7):1672-1687

[17] NieL, LinC, LiaoK, et al.. A view-free image stitching network based on global homography[J]. Journal of visual communication and image representation, 2020, 73: 102950

[18] NieL, LinC, LiaoK, et al.. Unsupervised deep image stitching: reconstructing stitched features to images[J]. IEEE transactions on image processing, 2021, 30: 6184-6197

[19] JustinJ, AlexandreA, LiF F. Perceptual losses for real-time style transfer and super-resolution[C], 2016, Heidelberg, Springer International Publishing: 694-711

[20] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2014-09-04) [2022-12-25]. https://arxiv.org/abs/1409.1556.

[21] ZhangJ, WangC, LiuS, et al.. Content-aware unsupervised deep homography estimation[C], 2020, Heidelberg, Springer International Publishing: 653-669

[22] NIE L, LIN C, LIAO K, et al. Learning edge-preserved image stitching from large-baseline deep homography[EB/OL]. (2020-12-11) [2022-12-25]. https://arxiv.org/abs/2012.06194.