Detection and removal of excess materials in aircraft wings using continuum robot end-effectors

Abstract

Excess materials are left inside aircraft wings due to manual operation errors, and the removal of excess materials is very crucial. To increase removal efficiency, a continuum robot (CR) with a removal end-effector and a stereo camera is used to remove excess objects. The size and weight characteristics of excess materials in aircraft wings are analyzed. A novel negative pressure end-effector and a two-finger gripper are designed based on the CR. The negative pressure end-effector aims to remove nuts, small rivets, and small volumes of aluminum shavings. A two-finger gripper is designed to remove large volumes of aluminum shavings. A stereo camera is used to achieve automatic detection and localization of excess materials. Due to poor lighting conditions in the aircraft wing compartment, supplementary lighting devices are used to improve environmental lighting. Then, You Only Look Once (YOLO) v5 is used to classify and detect excess objects, and two training data sets of excess objects in two wings are constructed. Due to the limited texture features inside the aircraft wings, this paper adopts an image-matching method based on the results of YOLO v5 detection. This matching method avoids the performance instability problem based on Oriented Fast and Rotated BRIEF feature point matching. Experimental verification reveals that the detection accuracy of each type of excess exceeds 90%, and the visual localization error is less than 2 mm for four types of excess objects. Results show the two end-effectors can work well for the task of removing excess material from the aircraft wings using a CR.

Graphical abstract

Keywords

end-effectors / continuum robot / visual detection and localization / removal of excess materials / gripper

References

[1] Long T F, Li Y, Chen J. Productivity prediction in aircraft final assembly lines: comparisons and insights in different productivity ranges. Journal of Manufacturing Systems, 2022, 62: 377–389

[2] Zhao Q J, Kong Y H, Sheng S J, Zhu J J. Redundant object detection method for civil aircraft assembly based on machine vision and smart glasses. Measurement Science & Technology, 2022, 33(10): 105011

[3] Yasuda Y D V, Cappabianco F A M, Martins L E G, Gripp J A B. Aircraft visual inspection: a systematic literature review. Computers in Industry, 2022, 141: 103695

[4] Tzitzilonis V, Malandrakis K, Zanotti Fragonara L, Gonzalez Domingo J A, Avdelidis N P, Tsourdos A, Forster K. Inspection of aircraft wing panels using unmanned aerial vehicles. Sensors, 2019, 19(8): 1824

[5] Axinte D, Dong X, Palmer D, Rushworth A, Guzman S C, Olarra A, Arizaga I, Gomez-Acedo E, Txoperena K, Pfeiffer K, Messmer F, Gruhler M, Kell J. MiRoR—miniaturized robotic systems for holistic in-situ repair and maintenance works in restrained and hazardous environments. IEEE/ASME Transactions on Mechatronics, 2018, 23(2): 978–981

[6] Guo J M, Zhang J W, Wu D, Gai Y H, Chen K. An algorithm based on bidirectional searching and geometric constrained sampling for automatic manipulation planning in aircraft cable assembly. Journal of Manufacturing Systems, 2020, 57: 158–168

[7] Mei B, Zhu W D. Accurate positioning of a drilling and riveting cell for aircraft assembly. Robotics and Computer-Integrated Manufacturing, 2021, 69: 102112

[8] Shafi I, Mazhar M F, Fatima A, Alvarez R M, Miró Y, Espinosa J C M, Ashraf I. Deep learning-based real time defect detection for optimization of aircraft manufacturing and control performance. Drones, 2023, 7(1): 31

[9] WangM F, Palmer D, DongX, AlatorreD, AxinteD, NortonA. Design and development of a slender dual-structure continuum robot for in-situ aeroengine repair. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid: IEEE, 2018, 5648–5653

[10] Li G X, Yu J J, Tang Y C, Pan J, Cao S G, Pei X. Design and modeling of continuum robot based on virtual-center of motion mechanism. Frontiers of Mechanical Engineering, 2023, 18(2): 23

[11] Jha M, Chauhan N R. A review on snake-like continuum robots for medical surgeries. IOP Conference Series: Materials Science and Engineering, 2019, 691(1): 012093

[12] Russo M, Sadati S M H, Dong X, Mohammad A, Walker I D, Bergeles C, Xu K, Axinte D A. Continuum robots: an overview. Advanced Intelligent Systems, 2023, 5(5): 2370020

[13] Wang M F, Dong X, Ba W M, Mohammad A, Axinte D, Norton A. Design, modelling and validation of a novel extra slender continuum robot for in-situ inspection and repair in aeroengine. Robotics and Computer-Integrated Manufacturing, 2021, 67: 102054

[14] NiuG C, Wang J K, XuK L. Model analysis for a continuum aircraft fuel tank inspection robot based on the Rzeppa universal joint. Advances in Mechanical Engineering, 2018, 10(5): 1687814018778229

[15] Russo M, Raimondi L, Dong X, Axinte D, Kell J. Task-oriented optimal dimensional synthesis of robotic manipulators with limited mobility. Robotics and Computer-Integrated Manufacturing, 2021, 69: 102096

[16] Dong X, Wang M F, Mohammad A, Ba W M, Russo M, Norton A, Kell J, Axinte D. Continuum robots collaborate for safe manipulation of high-temperature flame to enable repairs in challenging environments. IEEE/ASME Transactions on Mechatronics, 2022, 27(5): 4217–4220

[17] Troncoso D A, Robles-Linares J A, Russo M, Elbanna M A, Wild S, Dong X, Mohammad A, Kell J, Norton A D, Axinte D. A continuum robot for remote applications: from industrial to medical surgery with slender continuum robots. IEEE Robotics & Automation Magazine, 2023, 30(3): 94–105

[18] Norouzi-Ghazbi S, Mehrkish A. H. Fallah M, Janabi-Sharifi F. Constrained visual predictive control of tendon-driven continuum robots. Robotics and Autonomous Systems, 2021, 145: 103856

[19] Norouzi-GhazbiS, MehrkishA, Janabi-Sharifi F. Jacobian formulation for two classes of cooperative continuum robots. In: Proceedings of the Canadian Society for Mechanical Engineering International Congress 2020. Charlottetown: Progress in Canadian Mechanical Engineering, 2020, 3–8

[20] Dong X, Raffles M, Cobos-Guzman S, Axinte D, Kell J. A novel continuum robot using twin-pivot compliant joints: design, modeling, and validation. Journal of Mechanisms and Robotics, 2016, 8(2): 021010

[21] Oliver-Butler K, Till J, Rucker C. Continuum robot stiffness under external loads and prescribed tendon displacements. IEEE Transactions on Robotics, 2019, 35(2): 403–419

[22] Hernandez J, Sunny M S H, Sanjuan J, Rulik I, Zarif M I I, Ahamed S I, Ahmed H U, Rahman M H. Current designs of robotic arm grippers: a comprehensive systematic review. Robotics, 2023, 12(1): 5

[23] Zhang B H, Xie Y X, Zhou J, Wang K, Zhang Z. State-of-the-art robotic grippers, grasping and control strategies, as well as their applications in agricultural robots: a review. Computers and Electronics in Agriculture, 2020, 177: 105694

[24] SamadikhoshkhoZ, ZareiniaK, Janabi-Sharifi F. A brief review on robotic grippers classifications. In: Proceedings of the IEEE Canadian Conference of Electrical and Computer Engineering. Edmonton: IEEE, 2019, 1–4

[25] Firth C, Dunn K, Haeusler M H, Sun Y. Anthropomorphic soft robotic end-effector for use with collaborative robots in the construction industry. Automation in Construction, 2022, 138: 104218

[26] CorreaF. Integrating industry 4.0 associated technologies into automated and traditional construction. In: Proceedings of the 37th International Symposium on Automation and Robotics in Construction. Kitakyushu: The International Association for Automation And Robotics in Construction, 2020, 285–292

[27] Ramon Soria P, Arrue B C, Ollero A. Detection, location and grasping objects using a stereo sensor on UAV in outdoor environments. Sensors, 2017, 17(1): 103

[28] AlhammadM, Avdelidis N P, DeaneS, Ibarra-CastanedoC, Pant S, NooralishahiP, AhmadiM, GenestM, ZolotasA, Zanotti-Fragonara L, Valdes J, Maldague X P V. Diagnosis of composite materials in aircraft applications: towards a UAV-based active thermography inspection approach. In: Zalameda J N, Mendioroz A, eds. Thermosense: Thermal Infrared Applications XLIII. Florida: SPIE, 2021, 1174306

[29] Zou Z X, Chen K Y, Shi Z W, Guo Y H, Ye J P. Object detection in 20 years: a survey. Proceedings of the IEEE, 2023, 111(3): 257–276

[30] RedmonJ, Divvala S, GirshickR, FarhadiA. You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016, 779–788

[31] NazirA, Wani M A. You only look once - object detection models: a review. In: Proceedings of the 10th International Conference on Computing for Sustainable Global Development. New Delhi: IEEE, 2023, 1088–1095

[32] ChangJ R, Chen Y S. Pyramid stereo matching network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018, 5410–5418

[33] Mathew M P, Mahesh T Y. Leaf-based disease detection in bell pepper plant using YOLO v5. Signal, Image and Video Processing, 2022, 16(3): 841–847

[34] Fusiello A, Trucco E, Verri A. A compact algorithm for rectification of stereo pairs. Machine Vision and Applications, 2000, 12(1): 16–22

[35] GranaC, Borghesani D, ManfrediM, CucchiaraR. A fast approach for integrating ORB descriptors in the bag of words model. In: Proceedings of the Multimedia Content and Mobile Devices. Burlingame: SPIE, 2013, 866709

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (Grant No. U1813221). The authors are grateful to Xuewen Liu for assisting in the design of the end-effector, SIASUN Robot & Automation Co., Ltd., China for providing the continuum robot, Shanghai Shangfei Airplane Manufacturing Co., Ltd., China for supplying the aircraft wings, and AVIC Chengdu Aircraft Industrial Co., Ltd., China and Shanghai Jiao Tong University, China for their extending assistance.

Conflict of Interest

Jingjun Yu is a member of the Editorial Board of Frontiers of Mechanical Engineering, who was excluded from the peer review process and all editorial decisions related to the acceptance and publication of this article. Peer review was handled independently by the other editors to minimize bias.

RIGHTS & PERMISSIONS

2024 Higher Education Press