Data fusing and joint training for learning with noisy labels

Abstract

It is well known that deep learning depends on a large amount of clean data. Because of high annotation cost, various methods have been devoted to annotating the data automatically. However, a larger number of the noisy labels are generated in the datasets, which is a challenging problem. In this paper, we propose a new method for selecting training data accurately. Specifically, our approach fits a mixture model to the per-sample loss of the raw label and the predicted label, and the mixture model is utilized to dynamically divide the training set into a correctly labeled set, a correctly predicted set, and a wrong set. Then, a network is trained with these sets in the supervised learning manner. Due to the confirmation bias problem, we train the two networks alternately, and each network establishes the data division to teach the other network. When optimizing network parameters, the labels of the samples fuse respectively by the probabilities from the mixture model. Experiments on CIFAR-10, CIFAR-100 and Clothing1M demonstrate that this method is the same or superior to the state-of-the-art methods.

Graphical abstract

Keywords

deep learning / noisy labels / data fusing

References

[1] Yan Y , Rosales R , Fung G , Subramanian R , Dy J . Learning from multiple annotators with varying expertise. Machine Learning, 2014, 95( 3): 291– 327

[2] Yu X, Liu T, Gong M, Tao D. Learning with biased complementary labels. In: Proceedings of the 15th European Conference on Computer Vision. 2018, 69– 85

[3] Blum A , Kalai A , Wasserman H . Noise-tolerant learning, the parity problem, and the statistical query model. Journal of the ACM, 2003, 50( 4): 506– 519

[4] Tanno R, Saeedi A, Sankaranarayanan S, Alexander D C, Silberman N. Learning from noisy labels by regularized estimation of annotator confusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019, 1123 6– 11245

[5] Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. Understanding deep learning requires rethinking generalization. In: Proceedings of the 5th International Conference on Learning Representations (ICLR). 2017

[6] Goldberger J, Ben-Reuven E. Training deep neural-networks using a noise adaptation layer. In: Proceedings of the ICLR. 2017

[7] Liu T , Tao D . Classification with noisy labels by importance reweighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38( 3): 447– 461

[8] Patrini G, Rozza A, Menon A K, Nock R, Qu L. Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017, 2233−2241

[9] Reed S E, Lee H, Anguelov D, Szegedy C, Erhan D, Rabinovich A. Training deep neural networks on noisy labels with bootstrapping. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR). 2015

[10] Tanaka D, Ikami D, Yamasaki T, Aizawa K. Joint optimization framework for learning with noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2018, 5552−5560

[11] Arazo E, Ortego D, Albert P, O’Connor N E, McGuinness K. Unsupervised label noise modeling and loss correction. In: Proceedings of the 36th International Conference on Machine Learning (ICML). 2019, 312– 321

[12] Zhang H, Cissé M, Dauphin Y N, Lopez-Paz D. mixup: beyond empirical risk minimization. In: Proceedings of the 6th International Conference on Learning Representations (ICLR). 2018

[13] Ren M, Zeng W, Yang B, Urtasun R. Learning to reweight examples for robust deep learning. In: Proceedings of the 35th International Conference on Machine Learning (ICML). 2018, 4331−4340

[14] Jiang L, Zhou Z, Leung T, Li L, Fei-Fei L. MentorNet: learning data-driven curriculum for very deep neural networks on corrupted labels. In: Proceedings of the 35th International Conference on Machine Learning (ICML). 2018, 2309−2318

[15] Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I W, Sugiyama M. Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems (NeurIPS). 2018, 8536−8546

[16] Yu X, Han B, Yao J, Niu G, Tsang I W, Sugiyama M. How does disagreement help generalization against label corruption? In: Proceedings of the 36th International Conference on Machine Learning (ICML). 2019, 7164−7173

[17] Wei H, Feng L, Chen X, An B. Combating noisy labels by agreement: a joint training method with co-regularization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020, 1372 3– 13732

[18] Li J, Socher R, Hoi S C H. DivideMix: learning with noisy labels as semi-supervised learning. In: Proceedings of the 8th International Conference on Learning Representations. 2020

[19] Li Y, Yang J, Song Y, Cao L, Luo J, Li L. Learning from noisy labels with distillation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2017, 1928−1936

[20] Xiao T, Xia T, Yang Y, Huang C, Wang X. Learning from massive noisy labeled data for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015, 2691−2699

[21] Vahdat A. Toward robustness against label noise in training deep discriminative neural networks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS). 2017, 5601−5610

[22] Veit A, Alldrin N, Chechik G, Krasin I, Gupta A, Belongie S J. Learning from noisy large-scale datasets with minimal supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017, 6575−6583

[23] Lee K H, He X, Zhang L, Yang L. CleanNet: transfer learning for scalable image classifier training with label noise. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2018, 5447−5456

[24] Yi K, Wu J. Probabilistic end-to-end noise correction for learning with noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019, 7010−7018

[25] Ma X, Wang Y, Houle M E, Zhou S, Erfani S M, Xia S, Wijewickrema S N R, Bailey J. Dimensionality-driven learning with noisy labels. In: Proceedings of the 35th International Conference on Machine Learning (ICML). 2018, 3361−3370

[26] Hendrycks D, Mazeika M, Wilson D, Gimpel K. Using trusted data to train deep networks on labels corrupted by severe noise. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems (NeurIPS). 2018, 1047 7– 10486

[27] Thulasidasan S, Bhattacharya T, Bilmes J A, Chennupati G, Mohd-Yusof J. Combating label noise in deep learning using abstention. In: Proceedings of the 36th International Conference on Machine Learning (ICML). 2019, 6234−6243

[28] Shen Y, Sanghavi S. Learning with bad training data via iterative trimmed loss minimization. In: Proceedings of the 36th International Conference on Machine Learning (ICML). 2019, 5739−5748

[29] Ghosh A, Kumar H, Sastry P S. Robust loss functions under label noise for deep neural networks. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI). 2017, 1919−1925

[30] Wang Y, Ma X, Chen Z, Luo Y, Yi J, Bailey J. Symmetric cross entropy for robust learning with noisy labels. In: Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019, 322– 330

[31] Ding Y, Wang L, Fan D, Gong B. A semi-supervised two-stage approach to learning from noisy labels. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV). 2018, 1215−1224

[32] Kong K , Lee J , Kwak Y , Kang M , Kim S G , Song W J . Recycling: semi-supervised learning with noisy labels in deep neural networks. IEEE Access, 2019, 7 : 66998– 67005

[33] Berthelot D, Carlini N, Goodfellow I J, Oliver A, Papernot N, Raffel C. MixMatch: a holistic approach to semi-supervised learning. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS). 2019, 454

[34] Arpit D, Jastrzebski S, Ballas N, Krueger D, Bengio E, Kanwal M S, Maharaj T, Fischer A, Courville A C, Bengio Y, Lacoste-Julien S. A closer look at memorization in deep networks. In: Proceedings of the 34th International Conference on Machine Learning (ICML). 2017, 233−242

[35] Chen P, Liao B, Chen G, Zhang S. Understanding and utilizing deep neural networks trained with noisy labels. In: Proceedings of the 36th International Conference on Machine Learning (ICML). 2019, 1062−1070

[36] Permuter H , Francos J , Jermyn I . A study of Gaussian mixture models of color and texture features for image classification and segmentation. Pattern Recognition, 2006, 39( 4): 695– 706

[37] Tarvainen A, Valpola H. Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS). 2017, 1195−1204

[38] Krizhevsky A. Learning multiple layers of features from tiny images. University of Toronto, Dissertation, 2009

[39] Li J, Wong Y, Zhao Q, Kankanhalli M S. Learning to learn from noisy labeled data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019, 5046−5054

[40] Wang Y, Liu W, Ma X, Bailey J, Zha H, Song L, Xia S T. Iterative learning with open-set noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2018, 8688−8696

[41] He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. In: Proceedings of the 14th European Conference on Computer Vision (ECCV). 2016, 630−645

Acknowledgements

This work was supported by SRC-Open Project of Research Center of Security Video and Image Processing Engineering Technology of Guizhou ([2020]001]), Beijing Advanced Innovation Center for Intelligent Robots and Systems (2018IRS20) and National Natural Science Foundation of China (Grant No. 61973334).

RIGHTS & PERMISSIONS

2022 Higher Education Press