Rts: learning robustly from time series data with noisy label

Abstract

Significant progress has been made in machine learning with large amounts of clean labels and static data. However, in many real-world applications, the data often changes with time and it is difficult to obtain massive clean annotations, that is, noisy labels and time series are faced simultaneously. For example, in product-buyer evaluation, each sample records the daily time behavior of users, but the long transaction period brings difficulties to analysis, and salespeople often erroneously annotate the user’s purchase behavior. Such a novel setting, to our best knowledge, has not been thoroughly studied yet, and there is still a lack of effective machine learning methods. In this paper, we present a systematic approach RTS both theoretically and empirically, consisting of two components, Noise-Tolerant Time Series Representation and Purified Oversampling Learning. Specifically, we propose reducing label noise’s destructive impact to obtain robust feature representations and potential clean samples. Then, a novel learning method based on the purified data and time series oversampling is adopted to train an unbiased model. Theoretical analysis proves that our proposal can improve the quality of the noisy data set. Empirical experiments on diverse tasks, such as the house-buyer evaluation task from real-world applications and various benchmark tasks, clearly demonstrate that our new algorithm robustly outperforms many competitive methods.

Graphical abstract

Keywords

weakly-supervised learning / time-series classification / class-imbalanced learning

Zhi Zhou received a BSc degree from Jilin University, China in 2020. He is currently working toward a PhD degree with the National Key Laboratory for Novel Software Technology, Nanjing University, China. His research interests include weakly-supervised learning, representation learning, and out-of-distribution generalization

Yi-Xuan Jin received a BSc degree from Northwestern Polytechnical University, China in 2021. He is currently working toward a MS degree with the National Key Laboratory for Novel Software Technology, Nanjing University, China. His research interests include noisy label learning, model reuse and learnware

Yu-Feng Li received the BSc and PhD degrees in computer science from Nanjing University, China in 2006 and 2013, respectively. He joined the National Key Laboratory for Novel Software Technology at Nanjing University, China in 2013 and is currently a professor. He is a member of the LAMDA group. He is interested in weakly supervised learning, statistical learning, and optimization. He has received an outstanding doctoral dissertation award from China Computer Federation (CCF) and Jiangsu Province. He published more than 70 papers in top-tier journals and conferences such as JMLR, TPAMI, ICML, NIPS. He served as an editorial board member of MLJ, co-chair of ACML22/21 journal track, and area chair of top-tier conferences such as ICML23/22, AISTATS23, NeurIPS23/22, and IJCAI21

References

[1] Zhou Z H. Machine Learning. Singapore: Springer, 2021

[2] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770−778

[3] Adomavicius G, Tuzhilin A . Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 2005, 17( 6): 734–749

[4] Kononenko I . Machine learning for medical diagnosis: history, state of the art and perspective. Artificial Intelligence in Medicine, 2001, 23( 1): 89–109

[5] Cao H, Li X L, Woon Y K, Ng S K. SPO: structure preserving oversampling for imbalanced time series classification. In: Proceedings of the 11th IEEE International Conference on Data Mining. 2011, 1008−1013

[6] Guo L Z, Kuang F, Liu Z X, Li Y F, Ma N, Qie X H. IWE-Net: instance weight network for locating negative comments and its application to improve traffic user experience. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 4052−4059

[7] Frenay B, Verleysen M . Classification in the presence of label noise: a survey. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25( 5): 845–869

[8] Atkinson G, Metsis V. A survey of methods for detection and correction of noisy labels in time series data. In: Proceedings of the 17th International Conference on Artificial Intelligence Applications and Innovations. 2021, 479−493

[9] Wei T, Wang H, Tu W, Li Y F . Robust model selection for positive and unlabeled learning with constraints. Science China Information Science, 2022, 65( 11): 212101

[10] Pelletier C, Valero S, Inglada J, Champion N, Sicre C M, Dedieu G . Effect of training class label noise on classification performances for land cover mapping with satellite image time series. Remote Sensing, 2017, 9( 2): 173

[11] Castellani A, Schmitt S, Hammer B. Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noise. In: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2021, 469−484

[12] Atkinson G, Metsis V. Identifying label noise in time-series datasets. In: Proceedings of 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of 2020 ACM International Symposium on Wearable Computers. 2020, 238−243

[13] Cao H, Li X L, Woon D Y K, Ng S K . Integrated oversampling for imbalanced time series classification. IEEE Transactions on Knowledge and Data Engineering, 2013, 25( 12): 2809–2822

[14] Kim B, Choi J H, Choo J. Augmenting imbalanced time-series data via adversarial perturbation in latent space. In: Proceedings of the 13th Asian Conference on Machine Learning. 2021, 1633−1644

[15] Huang H, Xu C, Yoo S, Yan W, Wang T, Xue F. Imbalanced time series classification for flight data analyzing with nonlinear granger causality learning. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 2533−2540

[16] Geng Y, Luo X . Cost-sensitive convolutional neural networks for imbalanced time series classification. Intelligent Data Analysis, 2019, 23( 2): 357–370

[17] Ward M, Malmsten K, Salamy H, Min C H. Data balanced bagging ensemble of convolutional- LSTM neural networks for time series data classification with an imbalanced dataset. In: Proceedings of 2021 IEEE International Symposium on Circuits and Systems. 2021, 1−5

[18] Wei T, Shi J X, Tu W W, Li Y F. Robust long-tailed learning under label noise. 2021, arXiv preprint arXiv: 2108.11569

[19] Wei T, Shi J X, Li Y F, Zhang M L. Prototypical classifier for robust class-imbalanced learning. In: Proceedings of the 26th Pacific-Asia Conference on Knowledge Discovery and Data Mining. 2022, 44−57

[20] Gui X J, Wang W, Tian Z H. Towards understanding deep learning from noisy labels with small-loss criterion. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence. 2021, 2469−2475

[21] Lukasik M, Bhojanapalli S, Menon A K, Kumar S. Does label smoothing mitigate label noise?. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 598

[22] Laine S, Aila T. Temporal ensembling for semi-supervised learning. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[23] Han B, Yao Q, Liu T, Niu G, Tsang I W, Kwok J T, Sugiyama M. A survey of label-noise representation learning: past, present and future. 2021, arXiv preprint arXiv: 2011.04406

[24] Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I W, Sugiyama M. Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 8536−8546

[25] Li Y F, Liang D M . Safe semi-supervised learning: a brief introduction. Frontiers of Computer Science, 2019, 13( 4): 669–676

[26] Jia L H, Guo L Z, Zhou Z, Li Y F. Lamda-ssl: a comprehensive semi-supervised learning toolkit. Science China Information Science, 2023

[27] Dau H A, Bagnall A J, Kamgar K, Yeh C M, Zhu Y, Gharghabi S, Ratanamahatana C A, Keogh E J. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 2019, 6(6): 1293–1305

[28] Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[29] Tavenard R, Faouzi J, Vandewiele G, Divo F, Androz G, Holtz C, Payne M, Yurchak R, Rußwurm M, Kolar K, Woods E . Tslearn, a machine learning toolkit for time series data. The Journal of Machine Learning Research, 2020, 21( 1): 118

[30] Han B, Niu G, Yu X, Yao Q, Xu M, Tsang I W, Sugiyama M. SIGUA: forgetting may make learning with noisy labels more robust. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 4006−4016

[31] Chawla N V, Bowyer K W, Hall L O, Kegelmeyer W P . SMOTE: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 2002, 16( 1): 321–357

[32] Cao K, Wei C, Gaidon A, Arechiga N, Ma T. Learning imbalanced datasets with label-distribution-aware margin loss. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 140

[33] Brodersen K H, Ong C S, Stephan K E, Buhmann J M. The balanced accuracy and its posterior distribution. In: Proceedings of the 20th International Conference on Pattern Recognition. 2010, 3121−3124

Acknowledgements

This research was supported by the National Key R&D Program of China (2022YFC3340901) and the National Natural Science Foundation of China (Grant No. 62176118).

Competing interests

The authors declare that they have no competing interests or financial conflicts to disclose.

RIGHTS & PERMISSIONS

2024 Higher Education Press