Video image mosaic via multi-module cooperation

Abstract

We propose a video image mosaic method based on multi-module cooperation. This method stitches the video into a panorama with a large field of view, divided into three modules: the key frame selection module, the image mosaic module, and the optimization module. The key frame selection module obtains key frames by comprehensively evaluating the overlap rate and image quality. The image mosaic module stitches the key frames into a panoramic image to generate an initial mosaic result. The optimization module makes the mosaic result more natural and eliminates ghosts by using object detection advantages. Our method is tested on videos taken in real scenes, and the results have a more comprehensive and natural description.

References

[1] XiangT Z, XiaG S, ZhangL P. Image stitching by line-guided local warping with global similarity constraint[J]. Pattern recognition, 2018, 83: 481-497

[2] HerrmannC, WangC, BowenR S, et al.. Object-centered image stitching[C]//European Conference on Computer Vision, September 8–14, 2018, Munich, Germany. Berlin: Springer, 2018, 11207: 846-861

[3] LiaoT L, LiN. Single-perspective warps in natural image stitching[J]. IEEE transactions on image processing, 2020, 29: 724-735

[4] ZaragozaJ, ChinT J, TranQ H, et al.. As-projective-as-possible image stitching with moving dlt[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 36: 1285-1298

[5] ChangC H, SatoY, ChuangY Y. Shape-preserving half-projective warps for image stitching[C], 2014, New York, IEEE: 3254-3261

[6] LinC C, PankantiS U, RamamurthyK N, et al.. Adaptive as-natural-as-possible image stitching[C], 2015, New York, IEEE: 1155-1163

[7] LiN, XuY F, WangC. Quasi-homography warps in image stitching[J]. IEEE transactions on multimedia, 2018, 20(6):1365-1375

[8] GaoJ, KimS J, BrownM S. Constructing image panoramas using dual-homography warping[C], 2011, New York, IEEE: 49-56

[9] LiJ, WangZ M, LaiS M, et al.. Parallax-tolerant image stitching based on robust elastic warping[J]. IEEE transactions on multimedia, 2018, 20(7):1672-1687

[10] LeeK Y, SimJ Y. Warping residual based image stitching for large parallax[C], 2020, New York, IEEE: 8195-8203

[11] LinK, JiangN J, CheongL F, et al.. Seagull: seam-guided local alignment for parallax-tolerant image stitching[C], 2016, Berlin, Springer: 370-385

[12] YuanY T, FangF M, ZhangG X. Superpixel-based seamless image stitching for UAV images[J]. IEEE transactions on geoscience and remote sensing, 2021, 59(2):1565-1576

[13] XueW L, XieW L, ZhangY, et al.. Stable linear structures and seam measurements for parallax image stitching[J]. IEEE transactions on circuits and systems for video technology, 2021, PP(99):1-1

[14] YangT, LiJ, YuS, et al.. Diverse scene stitching from a large-scale aerial video dataset[J]. Remote sensing, 2015, 7(6):6932-6949

[15] BangS, KimH, KimH. UAV-based automatic generation of high-resolution panorama at a construction site with a focus on preprocessing for image stitching[J]. Automation in construction, 2017, 84: 70-80

[16] Crété-RoffetF, DolmiereT, LadretP, et al.. The blur effect: perception and estimation with a new no-reference perceptual blur metric[J]. Proceedings of SPIE-the international society for optical engineering, 2007, 12: 64920I-64920I-11

[17] ZhangZ, XueW, HuangW, et al.. Effective video frame acquisition for image stitching[J]. IEEE access, 2020, 8: 217086-217097

[18] LiuW, AnguelovD, ErhanD, et al.. SSD: single shot multibox detector[C], 2016, Berlin, Springer: 21-37

[19] PérezP, GangnetM, BlakeA. Poisson image editing[J]. ACM transactions on graphics, 2003, 22(3):313-318