View planning for visual detection coverage tasks of large airplane upper surface using UAVs

Abstract

In order to enhance the efficiency of visual inspection and effectively carry out 3D visual coverage tasks, this paper focuses on the 3D view planning problem concerning the visual coverage of an airplane’s surface using unmanned aerial vehicles (UAV). Our objective is to attain a sufficiently high coverage rate with the least number of viewpoints. The contributions of this work are enumerated as follows. Firstly, the 3D model of the target aircraft is spatially extended in accordance with the depth range of the camera mounted on the drone, thereby confining the sampling range of 3D viewpoints. Next, a candidate set of viewpoints is generated through random sampling and the probabilistic potential field technique. Subsequently, we propose a novel hyper-heuristic algorithm. In this algorithm, a genetic algorithm serves as a high-level heuristic strategy, in tandem with multiple low-level heuristic operators devised for combinatorial optimization. This not only augments the capacity to seek the global optimal solution but also expedites the convergence rate, aiming to ascertain the optimal subset of viewpoints. Moreover, we devise a new fitness function for appraising candidate solution vectors in the set covering problem (SCP), strengthening the evolutionary guidance for genetic algorithms. Eventually, experimental findings on the simulated and real airplanes corroborate the efficacy of the proposed method, i.e., it markedly diminishes the requisite number of viewpoints and augments inspection efficiency.

Keywords

Airplane surface inspection / View planning / Visual coverage / Combinatorial optimization / Unmanned aerial vehicle

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

Appendix A. Supplementary data

Supplementary material related to this article can be found online at https://doi.org/10.1016/j.birob.2025.100228.

References

[1] D. Jiajia, G. Xiaoxi, W. Jun, Coverage path planning method of unmanned aerial vehicle for aircraft surface detection task, J. Mech. Eng. 59 (16)(2023) 243-253.

[2] D.A. Rodriguez, C.L. Tafur, P.F.M. Daza, J.A.V. Vidales, J.C.D. Rincón, Inspec-tion of aircrafts and airports using UAS: a review, Results Eng. 22 (2024) 102330.

[3] R. Almadhoun, T. Taha, L. Seneviratne, J. Dias, G. Cai, Aircraft inspection using unmanned aerial vehicles, in: International Micro Air Vehicles, Conferences and Competitions Proceedings, 2016, pp. 1-7.

[4] W. Jing, J. Polden, W. Lin, K. Shimada, Sampling-based view planning for 3D visual coverage task with Unmanned Aerial Vehicle, in: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2016, pp. 1808-1815.

[5] S. Chen, Y. Li, N.M. Kwok, Active vision in robotic systems: A survey of recent developments, Int. J. Robot. Res. 30 (11) (2011) 1343-1377.

[6] M. Maboudi, M. Homaei, S. Song, S. Malihi, M. Saadatseresht, M. Gerke, A review on viewpoints and path planning for UAV-based 3-D reconstruction, IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 16 (2023) 5026-5048.

[7] W.R. Scott, Model-based view planning, Mach. Vis. Appl. 20 (1) (2009) 47-69.

[8] Y. Choi, Y. Choi, S. Briceno, D.N. Mavris, Three-dimensional UAS trajectory optimization for remote sensing in an irregular terrain environment, in: 2018 International Conference on Unmanned Aircraft Systems, ICUAS, IEEE, 2018, pp. 1101-1108.

[9] Y. Zheng, G. Liu, Y. Ding, G. Tian, A new clustering-based view planning method for building inspection with drone, IEEE Robot. Autom. Lett. 9 (11)(2024) 9781-9788.

[10] W. Hua, X. Xiaoshun, B. Xiaoqing, Multi-viewpoint planning for unmanned aerial vehicles based on continuous differentiable sampling, J. Comput. Integr. Manuf. Syst. 30 (3) (2024) 1161.

[11] V. Staderini, T. Glück, P. Schneider, R. Mecca, A. Kugi, Surface sampling for optimal viewpoint generation, in: 2023 IEEE 13th International Conference on Pattern Recognition Systems, ICPRS, IEEE, 2023, pp. 1-7.

[12] Y. Wang, T. Peng, W. Wang, M. Luo, High-efficient view planning for surface inspection based on parallel deep reinforcement learning, Adv. Eng. Inf. 55 (2023) 101849.

[13] F. Kong, F. Du, D. Zhao, Station-viewpoint joint coverage path planning towards mobile visual inspection, Robot. Comput. -Integr. Manuf. 91 (2025) 102821.

[14] H. Razip, N. Zakaria, Genetic algorithm with approximation algorithm based initial population for the set covering problem, in: Proceed-ings of International Conference on Communication and Computational Technologies, ICCCT 2021, Springer, 2021, pp. 59-78.

[15] T. Dokeroglu, T. Kucukyilmaz, E.-G. Talbi, Hyper-heuristics: A survey and taxonomy, Comput. Ind. Eng. 187 (2024) 109815.

[16] F. Zhao, Y. Liu, N. Zhu, T. Xu, et al., A selection hyper-heuristic algorithm with Q-learning mechanism, Appl. Soft Comput. 147 (2023) 110815.

[17] H.-B. Song, J. Lin, A genetic programming hyper-heuristic for the distributed assembly permutation flow-shop scheduling problem with sequence dependent setup times, Swarm Evol. Comput. 60 (2021) 100807.

[18] C.S. Tan, R. Mohd-Mokhtar, M.R. Arshad, A comprehensive review of coverage path planning in robotics using classical and heuristic algorithms, IEEE Access 9 (2021) 119310-119342.

[19] P. Gospodnetic, D. Mosbach, M. Rauhut, H. Hagen, Viewpoint placement for inspection planning, Mach. Vis. Appl. 33 (1) (2022) 2.

[20] W. Jing, K. Shimada, Model-based view planning for building inspection and surveillance using voxel dilation, medial objects, and random-key genetic algorithm, J. Comput. Des. Eng. 5 (3) (2018) 337-347.

[21] R.M. Haralick, S.R. Sternberg, X. Zhuang, Image analysis using mathematical morphology, IEEE Trans. Pattern Anal. Mach. Intell. (4) (1987) 532-550.

[22] T. Möller, B. Trumbore, Fast minimum storage ray-triangle intersection, J. Graph. Tools 2 (1) (1997) 21-28.

[23] A.S. Ferreira, A. Pozo, R.A. Gonçalves, et al., An ant colony based hyper-heuristic approach for the set covering problem, Adv. Distrib. Comput. Artif. Intell. J. 4 (1) (2015) 1-21.

[24] E.K. Burke, M.R. Hyde, G. Kendall, G. Ochoa, E. Özcan, J.R. Wood-ward, A classification of hyper-heuristic approaches: revisited, Handb. Metaheuristics (2019) 453-477.

[25] T.M. Shami, A.A. El-Saleh, M. Alswaitti, Q. Al-Tashi, M.A. Summakieh, S. Mirjalili, Particle swarm optimization: A comprehensive survey, IEEE Access 10 (2022) 10031-10061.

[26] S. Katoch, S.S. Chauhan, V. Kumar, A review on genetic algorithm: past, present, and future , Multimedia Tools Appl. 80 (2021) 8091-8126.

[27] J.E. Beasley, P.C. Chu, A genetic algorithm for the set covering problem, European J. Oper. Res. 94 (2) (1996) 392-404.

[28] J. Jianping., C. Kun., C. Wang., Set covering problem based on improved genetic algorithm, Math. Pract. Theory 24 (5) (2012) 120-126.

[29] G. Koulinas, L. Kotsikas, K. Anagnostopoulos, A particle swarm optimiza-tion based hyper-heuristic algorithm for the classic resource constrained project scheduling problem, Inform. Sci. 277 (2014) 680-693.

Funding

the Trade Union Employee Innovation Foundation of China(2022270024)