Debiased label enhancement for class-imbalanced semi-supervised learning

Abstract

In class-imbalanced semi-supervised learning, the goal is to leverage abundant unlabeled data when labeled examples are scarce in a class-imbalanced setting. Classifiers of pseudo-label-based algorithms tend to become biased and suffer from degraded representation quality due to the utilization of skewed pseudo-labels for training. Previous pseudo-label-based algorithms employ the classifier itself to generate pseudo-labels for unlabeled data, leading to suboptimal performance on imbalanced tasks. The classifier is optimized to achieve uniform accuracy across all classes, mitigating the bias toward majority classes, while pseudo-labeling strives to accurately annotate the training unlabeled data in a class-imbalanced distribution. This misalignment causes confirmation bias, reinforcing bias in the pseudo-labeling process. To address this issue, we propose a novel semi-supervised framework that disentangles pseudo-label generation from the classification task via designing a dedicated pseudo-label generator to align the class distributions between labeled and unlabeled data. Specifically, we alternately train the pseudo-label generator and the predictive model, where the pseudo-label generator is trained on a debiased label enhancement objective, and the predictive model then leverages these pseudo-labels along with class-level debiasing. Experiments on the imbalanced benchmark datasets validate the effectiveness of the proposed framework.

Graphical abstract

Keywords

Class-imbalanced semi-supervised learning / label enhancement

References

[1] French G, Mackiewicz M, Fisher M. Self-ensembling for visual domain adaptation. In: Proceedings of the 6th International Conference on Learning Representations. 2018

[2] Sajjadi M, Javanmardi M, Tasdizen T. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 1171−1179

[3] Laine S, Aila T. Temporal ensembling for semi-supervised learning. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[4] Wei C, Sohn K, Mellina C, Yuille A, Yang F. CReST: a class-rebalancing self-training framework for imbalanced semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, 10852−10861

[5] Lee H, Shin S, Kim H. ABC: auxiliary balanced classifier for class-imbalanced semi-supervised learning. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021, 542

[6] Guo L Z, Li Y F. Class-imbalanced semi-supervised learning with adaptive thresholding. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 8082−8094

[7] Fan Y, Dai D, Kukleva A, Schiele B. CoSSL: co-learning of representation and classifier for imbalanced semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 14554−14564

[8] Lai Z, Wang C, Gunawan H, Cheung S C S, Chuah C N. Smoothed adaptive weighting for imbalanced semi-supervised learning: improve reliability against unknown distribution data. In: Proceedings of the 39th International Conference on Machine Learning. 2022, 11828−11843

[9] Du C, Han Y, Huang G. SimPro: a simple probabilistic framework towards realistic long-tailed semi-supervised learning. In: Proceedings of the 41st International Conference on Machine Learning. 2024, 464

[10] Sohn K, Berthelot D, Li C L, Zhang Z, Carlini N, Cubuk E D, Kurakin A, Zhang H, Raffel C. FixMatch: Simplifying semi-supervised learning with consistency and confidence. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 51

[11] Berthelot D, Carlini N, Cubuk E D, Kurakin A, Sohn K, Zhang H, Raffel C. ReMixMatch: semi-supervised learning with distribution matching and augmentation anchoring. In: Proceedings of International Conference on Learning Representations. 2020

[12] Miyato T, Maeda S I, Koyama M, Ishii S . Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41( 8): 1979–1993

[13] Wang Y, Chen H, Heng Q, Hou W, Fan Y, Wu Z, Wang J, Savvides M, Shinozaki T, Raj B, Schiele B, Xie X. FreeMatch: self-adaptive thresholding for semi-supervised learning. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[14] Berthelot D, Carlini N, Goodfellow I, Oliver A, Papernot N, Raffel C. MixMatch: a holistic approach to semi-supervised learning. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 454

[15] Zhang B, Wang Y, Hou W, Wu H, Wang J, Okumura M, Shinozaki T. FlexMatch: boosting semi-supervised learning with curriculum pseudo labeling. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. 2021, 1407

[16] Chen H, Tao R, Fan Y, Wang Y, Wang J, Schiele B, Xie X, Raj B, Savvides M. SoftMatch: addressing the quantity-quality tradeoff in semi-supervised learning. In: Proceedings of the 11th International Conference on Learning Representations. 2023

[17] Yu H, Zhang N, Deng S, Yuan Z, Jia Y, Chen H. The devil is the classifier: investigating long tail relation classification with decoupling analysis. 2020, arXiv preprint arXiv: 2009.07022

[18] Xu N, Liu Y P, Geng X . Label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 2021, 33( 4): 1632–1643

[19] Xu N, Shu J, Zheng R, Geng X, Meng D, Zhang M L . Variational label enhancement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45( 5): 6537–6551

[20] Liu B, Xu N, Lv J, Geng X. Revisiting pseudo-label for single-positive multi-label learning. In: Proceedings of the 40th International Conference on Machine Learning. 2023, 924

[21] Liu B, Xu N, Fang X, Geng X. Correlation-induced label prior for semi-supervised multi-label learning. In: Proceedings of the 41st International Conference on Machine Learning. 2024, 1305

[22] Li M, Wu R, Liu H, Yu J, Yang X, Han B, Liu T. InstanT: semi-supervised learning with instance-dependent thresholds. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. 2023, 130

[23] Patrini G, Rozza A, Krishna Menon A, Nock R, Qu L. Making deep neural networks robust to label noise: a loss correction approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 2233−2241

[24] Xia X, Liu T, Wang N, Han B, Gong C, Niu G, Sugiyama M. Are anchor points really indispensable in label-noise learning?. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 614

[25] Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images. Toronto: University of Toronto, 2009

[26] Coates A, Ng A, Lee H. An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. 2011, 215−223

[27] Huh M, Agrawal P, Efros A A. What makes ImageNet good for transfer learning?. 2016, arXiv preprint arXiv: 1608.08614

[28] Lee H, Kim H. CDMAD: class-distribution-mismatch-aware debiasing for class-imbalanced semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, 23891−23900

[29] Huang C, Li Y, Loy C C, Tang X. Learning deep representation for imbalanced classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 5375−5384

[30] Wang Y, Chen H, Fan Y, Sun W, Tao R, , et al. USB: a unified semi-supervised learning benchmark for classification. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 285

[31] Oliver A, Odena A, Raffel C, Cubuk E D, Goodfellow I J. Realistic evaluation of deep semi-supervised learning algorithms. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 3239−3250

[32] Oh Y, Kim D J, Kweon I S. DASO: distribution-aware semantics-oriented pseudo-label for imbalanced semi-supervised learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, 9776−9786

[33] Maurer A, Pontil M. Empirical bernstein bounds and sample-variance penalization. In: Proceedings of the 22nd Annual Conference on Learning Theory. 2009

[34] Bennett G . Probability inequalities for the sum of independent random variables. Journal of the American Statistical Association, 1962, 57( 297): 33–45

[35] Ma C, Elezi I, Deng J, Dong W, Xu C. Three heads are better than one: complementary experts for long-tailed semi-supervised learning. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence. 2024, 14229−14237

[36] Wei T, Gan K. Towards realistic long-tailed semi-supervised learning: consistency is all you need. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, 3469−3478

[37] Kim J, Hur Y, Park S, Yang E, Hwang S J, Shin J. Distribution aligning refinery of pseudo-label for imbalanced semi-supervised learning. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 1221

[38] van der Maaten L, Hinton G . Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9( 86): 2579–2605

Rights & permissions

Higher Education Press