DeckVis: visual analysis of carrier-based aircraft deck support operation scenarios

Abstract

Conducting multi-scale visual analyses of carrier-based aircraft deck support operation scenarios is of significant strategic importance for enhancing aircraft carrier combat effectiveness. However, existing methods primarily focus on task recognition and lack intuitive, interactive analysis of the spatiotemporal coupling and evolutionary processes among multiple tasks. Moreover, these recognition approaches rely on high-quality annotated datasets, limiting their applicability in real-world scenarios. To address these challenges, we propose a visualization-based analytical method tailored for deck support operation scenarios. Specifically, to accurately and efficiently recognize tasks, we introduce a feature-enhanced clustering-based task recognition method that improves trajectory representations by incorporating both spatiotemporal and discriminative features. Building on this foundation, we develop DeckVis, a visualization system that presents the spatiotemporal features of tasks through coordinated multi-view displays, thereby enabling interactive, multi-scale scenario analysis. A comparative analysis, carefully designed case studies, and in-depth user evaluations demonstrate the effectiveness of our recognition method and visual system.

Graphical abstract

Keywords

support operation scenario / trajectory-based / task recognition / feature enhanced clustering / visual analysis

References

[1] Jewell A. Sortie generation capacity of embarked airwings. Alexandria: Center for Naval Analyses, 1998

[2] Jewell A, Wigge M A, Gagnon C M K, Lynn L A, Kirk K M, Berg K K, Roberts T A, Hale A J, Jones W L, Matheny A M, Hall J O, Measell B H. USS Nimitz and carrier airwing nine surge demonstration. Alexandria: Center for Naval Analyses, 1998

[3] Wang X, Liu J, Su X, Peng H, Zhao X, Lu C . A review on carrier aircraft dispatch path planning and control on deck. Chinese Journal of Aeronautics, 2020, 33( 12): 3039–3057

[4] Liu Y, Han W, Su X, Cui R . Optimization of fixed aviation support resource station configuration for aircraft carrier based on aircraft dispatch mission scheduling. Chinese Journal of Aeronautics, 2023, 36( 2): 127–138

[5] Li Y F, Gao L, Hao H J, Jin Y Y, Wang K, Xu M L . Human-machine collaborative decision-making for carrier aircraft support operations. Scientia Sinica Informationis, 2023, 53( 12): 2493–2510

[6] Li Y F, Wu Q S, Xu M L, Lv P, Jiang X H, Zhu R J, Zhou B . Real-time scheduling for carrier-borne aircraft support operations: a reinforcement learning approach. Scientia Sinica Informationis, 2021, 51( 2): 247–262

[7] Xu M, Wang Z, Meng Y, Luo W, Wang H, He S, Li C, Guo Y, Li Y, Lv P . Virtual-physical digital twin testbed for heterogeneous crowd operations. Science China Information Sciences, 2025, 68( 5): 154101

[8] Wan X, Li W. Research on the recognition and decision-making of carrier-based aircraft interception and landing based on visual information. In: Proceedings of the 22nd International Conference on Man-Machine-Environment System Engineering. 2022, 464−470

[9] Zhu X, Tian S, Fan J, Wang Z, Liu T . Research on target detection of carrier-based aircraft based on deep convolutional neural network. IOP Conference Series: Materials Science and Engineering, 2020, 853: 012057

[10] Zhao C, Shao Y, Yao D. A neural network-based method for tracking and locating the carrier-based aircraft. In: Proceedings of the 11th IEEE Joint International Information Technology and Artificial Intelligence Conference. 2023, 1222−1225

[11] Huo Q, Shi Y, Liu C, Tarokh V, Ferrari S. Online action change detection for automatic vision-based ground control of aircraft. In: Proceedings of AIAA SCITECH 2022 Forum. 2022

[12] Elboushaki A, Hannane R, Afdel K, Koutti L . MultiD-CNN: a multi-dimensional feature learning approach based on deep convolutional networks for gesture recognition in RGB-D image sequences. Expert Systems with Applications, 2020, 139: 112829

[13] Kurban O C, Calik N, Yildirim T . Human and action recognition using adaptive energy images. Pattern Recognition, 2022, 127: 108621

[14] Sun S, Dong Z, Zhao J . Conditional random fields for multiview sequential data modeling. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33( 3): 1242–1253

[15] Guo X, Li J, Wang H, Liu J, Li Y F, Xu M . Recognition of carrier-based aircraft flight deck operations based on dynamic graph. Chinese Journal of Aeronautics, 2025, 38( 3): 103256

[16] Yang D, Wu Y, Sun F, Chen J, Zhai D, Fu C . Freeway accident detection and classification based on the multi-vehicle trajectory data and deep learning model. Transportation Research Part C: Emerging Technologies, 2021, 130: 103303

[17] Zhu L, Guo F, Krishnan R, Polak J W. A deep learning approach for traffic incident detection in urban networks. In: Proceedings of the 21st International Conference on Intelligent Transportation Systems. 2018, 1011−1016

[18] Yu W, Wang G . Graph based embedding learning of trajectory data for transportation mode recognition by fusing sequence and dependency relations. International Journal of Geographical Information Science, 2023, 37( 12): 2514–2537

[19] Wu Y, Cao N, Gotz D, Tan Y-P, Keim D A . A survey on visual analytics of social media data. IEEE Transactions on Multimedia, 2016, 18( 11): 2135–2148

[20] Guo Y, Guo S, Jin Z, Kaul S, Gotz D, Cao N . Survey on visual analysis of event sequence data. IEEE Transactions on Visualization and Computer Graphics, 2022, 28( 12): 5091–5112

[21] Guo S, Xu K, Zhao R, Gotz D, Zha H, Cao N . EventThread: visual summarization and stage analysis of event sequence data. IEEE Transactions on Visualization and Computer Graphics, 2018, 24( 1): 56–65

[22] Sun D, Huang R, Chen Y, Wang Y, Zeng J, Yuan M, Pong T-C, Qu H . PlanningVis: a visual analytics approach to production planning in smart factories. IEEE Transactions on Visualization and Computer Graphics, 2020, 26( 1): 579–589

[23] Law P-M, Liu Z, Malik S, Basole R C . MAQUI: interweaving queries and pattern mining for recursive event sequence exploration. IEEE Transactions on Visualization and Computer Graphics, 2019, 25( 1): 396–406

[24] Lu J, Pan B, Chen J, Feng Y, Hu J, Peng Y, Chen W . AgentLens: visual analysis for agent behaviors in LLM-based autonomous systems. IEEE Transactions on Visualization and Computer Graphics, 2025, 31( 8): 4182–4197

[25] Liang M, Liu R W, Li S, Xiao Z, Liu X, Lu F . An unsupervised learning method with convolutional auto-encoder for vessel trajectory similarity computation. Ocean Engineering, 2021, 225: 108803

[26] Fang Z, Du Y, Chen L, Hu Y, Gao Y, Chen G. E²DTC: an end to end deep trajectory clustering framework via self-training. In: Proceedings of the 37th IEEE International Conference on Data Engineering. 2021, 696−707

[27] Li X, Zhao K, Cong G, Jensen C S, Wei W. Deep representation learning for trajectory similarity computation. In: Proceedings of the 34th IEEE International Conference on Data Engineering. 2018, 617−628

RIGHTS & PERMISSIONS

Higher Education Press