Extreme Attitude Prediction of Amphibious Vehicles Based on Improved Transformer Model and Extreme Loss Function

Abstract

Amphibious vehicles are more prone to attitude instability compared to ships, making it crucial to develop effective methods for monitoring instability risks. However, large inclination events, which can lead to instability, occur frequently in both experimental and operational data. This infrequency causes events to be overlooked by existing prediction models, which lack the precision to accurately predict inclination attitudes in amphibious vehicles. To address this gap in predicting attitudes near extreme inclination points, this study introduces a novel loss function, termed generalized extreme value loss. Subsequently, a deep learning model for improved waterborne attitude prediction, termed iInformer, was developed using a Transformer-based approach. During the embedding phase, a text prototype is created based on the vehicle’s operation log data is constructed to help the model better understand the vehicle’s operating environment. Data segmentation techniques are used to highlight local data variation features. Furthermore, to mitigate issues related to poor convergence and slow training speeds caused by the extreme value loss function, a teacher forcing mechanism is integrated into the model, enhancing its convergence capabilities. Experimental results validate the effectiveness of the proposed method, demonstrating its ability to handle data imbalance challenges. Specifically, the model achieves over a 60% improvement in root mean square error under extreme value conditions, with significant improvements observed across additional metrics.

Keywords

Amphibious vehicle / Attitude prediction / Extreme value loss function / Enhanced transformer architecture / External information embedding

References

[1] Arjovsky M, Bottou L (2017) Towards principled methods for training generative adversarial networks. Machine Learning, ArXiv abs/1701.04862

[2] BengioS, VinyalsO, JaitlyN, ShazeerN. Scheduled sampling for sequence prediction with recurrent neural networks. Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NIPS), 2015

[3] ChawlaNV, BowyerKW, HallLO, KegelmeyerWP. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 2002, 16: 321-357

[4] ChenQ, ZhengS, LiM, ZhuoH. Research on prediction error of ship rolling motion. Ship Engineering, 2021, 43(2): 42-47

[5] DingDZ, ZhangM, PanXD, YangM, HeXNAssoc Comp M. Modeling extreme events in time series prediction. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 20191114-1122

[6] FanJQ, FanYY, BarutE. Adaptive robust variable selection. Annals of Statistics, 2014, 42(1): 324-351

[7] HaanL, FerreiraAExtreme value theory: An introduction, 2006, New York, USA, Springer

[8] HarrouF, DairiA, DorbaneA, SunY. Enhancing wind power prediction with self-attentive variational autoencoders: A comparative study. Results in Engineering, 2024, 23: 13

[9] HarrouF, ZeroualA, KadriF, SunY. Enhancing road traffic flow prediction with improved deep learning using wavelet transforms. Results in Engineering, 2024, 23: 14

[10] HittaweMM, HarrouF, TogouMA, SunY, KnioO. Time-series weather prediction in the Red sea using ensemble transformers. Applied Soft Computing, 2024, 164: 16

[11] HornikK, StinchcombeM, WhiteH. Multilayer feedforward networks are universal approximators. Neural Networks, 1989, 2(5): 359-366

[12] HouX, XiaS. Short-term prediction of ship roll motion in waves based on convolutional neural network. Journal of Marine Science and Engineering, 2024, 12(1): 102

[13] LiuXY, WuJ, ZhouZH. Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008, 39(2): 539-550

[14] MaX, ChangL. Investigation on nonlinear rolling dynamics of amphibiousvehicle under wind and wave load. Journal of Measurement Science and Instrumentation, 2015, 6(3): 275-281

[15] Nie Y, Nguyen NH, Sinthong P, Kalagnanam J (2022) A time series is worth 64 words: Long-term forecasting with transformers. Machine Learning, arXiv preprint arXiv:2211.14730

[16] RosenblattM. Remarks on some nonparametric estimates of a density function. Ann. Math. Stat., 1956, 27(3): 832-837

[17] RossTY, DollárG. Focal loss for dense object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 20172980-2988

[18] ShafiqM, GuZQ. Deep residual learning for image recognition: A survey. Applied Sciences-Basel, 2022, 12(18): 43

[19] SilvermanBW. Density estimation for statistics and data analysis. J. Roy. Stat. Soc. Ser. C, 1986, 37: 120-121

[20] SunY, WongAK, KamelMS. Classification of imbalanced data: A review. International Journal of Pattern Recognition and Artificial Intelligence, 2009, 23(4): 687-719

[21] TashkovaK, SilcJ, AtanasovaN, DzeroskiS. Parameter estimation in a nonlinear dynamic model of an aquatic ecosystem with meta-heuristic optimization. Ecological Modelling, 2012, 226: 36-61

[22] VaswaniA, ShazeerN, ParmarN, UszkoreitJ, JonesL, GomezAN, KaiserL, PolosukhinI. Attention is all you need. Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), 2017

[23] WangG, HanB, SunW. Short-term prediction of ship motion based on LSTM. Ship Science and Technology, 2017, 39(13): 69-72(in Chinese)

[24] XuG, ZhouJ, YaoX, GuoQ. Toss motion of amphibious vehicle in sea-way. Acta Armamentarii, 2005, 26(4): 433-437(in Chinese)

[25] XueYF, LiuYJ, JiC, XueG. Hydrodynamic parameter identification for ship manoeuvring mathematical models using a Bayesian approach. Ocean Engineering, 2020, 195: 106612

[26] YinJC, PerakisAN, WangN. A real-time ship roll motion prediction using wavelet transform and variable RBF network. Ocean Engineering, 2018, 160: 10-19

[27] ZhangM, DingDZ, PanXD, YangM. Enhancing time series predictors with generalized extreme value loss. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(2): 1473-1487

[28] Zhou HY, Zhang SH, Peng JQ, Zhang S, Li JX, Xiong H, Zhang WC (2021) Informer: Beyond efficient transformer for long sequence time-series forecasting. Machine Learning, arXiv: 2012.07436

[29] ZhouT, MaZQ, WenQS, WangX, SunL, JinR. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. Proceedings of the 39th International Conference on Machine Learning (ICML), 2022

RIGHTS & PERMISSIONS

Harbin Engineering University and Springer-Verlag GmbH Germany, part of Springer Nature