Revisiting multi-dimensional classification from a dimension-wise perspective

Abstract

Real-world objects exhibit intricate semantic properties that can be characterized from a multitude of perspectives, which necessitates the development of a model capable of discerning multiple patterns within data, while concurrently predicting several Labeling Dimensions (LDs) — a task known as Multi-dimensional Classification (MDC). While the class imbalance issue has been extensively investigated within the multi-class paradigm, its study in the MDC context has been limited due to the imbalance shift phenomenon. A sample’s classification as a minor or major class instance becomes ambiguous when it belongs to a minor class in one LD and a major class in another. Previous MDC methodologies predominantly emphasized instance-wise criteria, neglecting prediction capabilities from a dimension aspect, i.e., the average classification performance across LDs. We assert the significance of dimension-wise metrics in real-world MDC applications and introduce two such metrics. Furthermore, we observe imbalanced class distributions within each LD and propose a novel Imbalance-Aware fusion Model (IMAM) for addressing the MDC problem. Specifically, we first decompose the task into multiple multi-class classification problems, creating imbalance-aware deep models for each LD separately. This straightforward method performs well across LDs without sacrificing performance in instance-wise criteria. Subsequently, we employ LD-wise models as multiple teachers and transfer their knowledge across all LDs to a unified student model. Experimental results on several real-world datasets demonstrate that our IMAM approach excels in both instance-wise evaluations and the proposed dimension-wise metrics.

Graphical abstract

Keywords

multi-dimensional classification / dimension perspective / class imbalance learning

Yi Shi received the master degree in computer science from Nanjing University, China in 2022. At the same year, he became a PhD student with the School of Artificial Intelligence, Nanjing University, China. His research interests are mainly in machine learning and data mining, including distance metric learning, multi-modal/multi-task learning, and semantic mining

Hanjia Ye received the PhD degree in computer science from Nanjing University, China in 2019. At the same year, he became a faculty member with the School of Artificial Intelligence, Nanjing University, China. His research interests lie primarily in machine learning, including distance metric learning, multi-modal/multi-task learning, meta-learning, and semantic mining. He serves as PC in leading conferences such as ICLR, CVPR, ICCV, ICML, and NeurIPS

Dongliang Man received the BA degree in medicine clinical laboratory diagnostics from China Medical University, China in 2010. At the same year, he became a faculty member with the Department of Laboratory Medicine at the First Hospital of China Medical University, China. He is currently a lecturer and PhD student with China Medical University, China. He has authored or coauthored more than 10 papers in national and international journals. His research interests include laboratory medicine and laboratory medicine intelligence

Xiaoxu Han received the MD degree in clinical medicine from China Medical University, China in 2006. At the same year, she became a faculty member with the First Hospital of China Medical University, China. She is currently a professor with China Medical University. She has authored or coauthored more than 100 papers in national and international journals. Her research interests include molecular diagnostic techniques and laboratory medicine intelligence. She is an invited reviewer for J Biol Chem, BMC Genomics, Virologica Sinica, and other journals. She is currently the vice director of the National Clinical Research Center for Laboratory Medicine, vice director of the Department of Laboratory Medicine at the First Hospital of China Medical University, and director of the Laboratory of Molecular Biology at the Key Laboratory of AIDS Immunology of the National Health Care Commission

Dechuan Zhan received the PhD degree in computer science from Nanjing University, China in 2010. At the same year, he became a faculty member in the Department of Computer Science and Technology at Nanjing University, China. He is currently a professor with the School of Artificial Intelligence at Nanjing University from since 2019. His research interests are mainly in machine learning and data mining. Up until now, He has published over 90 papers in national and international journals or conferences such as TPAMI, TKDD, TIFS, TSMSB, IJCAI, AAAI, ICML, NeurIPS, serves as the editorial board member of IDA and IJAPR, and as SPC/PCs in leading conferences such as IJCAI, AAAI, ICML, NeurIPS. He is the deputy director of LAMDA group, NJU

Yuan Jiang is now a full professor in School of Artificial Intelligence, Nanjing University, China. She received the PhD degree in computer science from Nanjing University, China in 2004. Her research interests are mainly in machine learning, data mining and artificial intelligence applications. She has published more than 50 papers in leading international/national journals and conferences. She was selected in the Program for New Century Excellent Talents in University, Ministry of Education in 2009

References

[1] Zhang C, Yankov D, Wu C T, Shapiro S, Hong J, Wu W. What is that building?: an end-to-end system for building recognition from streetside images. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2425−2433

[2] Sahoo D, Wang H, Shu K, Wu X, Le H, Achananuparp P, Lim E P, Hoi S C H. FoodAI: food image recognition via deep learning for smart food logging. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 2260−2268

[3] Borisyuk F, Gordo A, Sivakumar V. Rosetta: large scale system for text detection and recognition in images. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 71−79

[4] Yang X, Zeng Z, Teo S G, Wang L, Chandrasekhar V, Hoi S. Deep learning for practical image recognition: case study on kaggle competitions. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 923−931

[5] Wang Z, Long C, Cong G, Ju C. Effective and efficient sports play retrieval with deep representation learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 499−509

[6] Huang J T, Sharma A, Sun S, Xia L, Zhang D, Pronin P, Padmanabhan J, Ottaviano G, Yang L. Embedding-based retrieval in facebook search. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2553−2561

[7] Jia X, Zhao H, Lin Z, Kale A, Kumar V. Personalized image retrieval with sparse graph representation learning. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2735−2743

[8] Tan R, Vasileva M I, Saenko K, Plummer B A. Learning similarity conditions without explicit supervision. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 10373−10382

[9] Lin Y L, Tran S D, Davis L S. Fashion outfit complementary item retrieval. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 3311−3319

[10] Kim D, Saito K, Mishra S, Sclaroff S, Saenko K, Plummer B A. Self-supervised visual attribute learning for fashion compatibility. In: Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. 2021, 1057−1066

[11] Wang Z, Wang Y, Feng B, Mudigere D, Muthiah B, Ding Y. El-rec: efficient large-scale recommendation model training via tensor-train embedding table. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2022, 1−14

[12] Kononenko I . Machine learning for medical diagnosis: history, state of the art and perspective. Artificial Intelligence in Medicine, 2001, 23( 1): 89–109

[13] Amato F, López A, María Peña-Méndez E, Vaňhara P, Hampl A, Havel J . Artificial neural networks in medical diagnosis. Journal of Applied Biomedicine, 2013, 11( 2): 47–58

[14] Turnbull D, Barrington L, Torres D, Lanckriet G . Semantic annotation and retrieval of music and sound effects. IEEE Transactions on Audio, Speech, and Language Processing, 2008, 16( 2): 467–476

[15] Serafino F, Pio G, Ceci M, Malerba D. Hierarchical multidimensional classification of web documents with MultiWebClass. In: Proceedings of the 18th International Conference on Digital Society. 2015, 236−250

[16] Shatkay H, Pan F, Rzhetsky A, Wilbur W J . Multi-dimensional classification of biomedical text: toward automated, practical provision of high-utility text to diverse users. Bioinformatics, 2008, 24( 18): 2086–2093

[17] Barutcuoglu Z, Schapire R E, Troyanskaya O G . Hierarchical multi-label prediction of gene function. Bioinformatics, 2006, 22( 7): 830–836

[18] Feng B, Wang Y, Ding Y. Saga: sparse adversarial attack on EEG-based brain computer interface. In: Proceedings of 2021 IEEE International Conference on Acoustics, Speech and Signal Processing. 2021, 975−979

[19] Jia B B, Zhang M L. Multi-dimensional classification via sparse label encoding. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 4917−4926

[20] Wang H, Chen C, Liu W, Chen K, Hu T, Chen G. Incorporating label embedding and feature augmentation for multi-dimensional classification. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 6178−6185

[21] Ma Z, Chen S . Multi-dimensional classification via a metric approach. Neurocomputing, 2018, 275: 1121–1131

[22] Read J, Pfahringer B, Holmes G, Frank E . Classifier chains for multi-label classification. Machine Learning, 2011, 85( 3): 333–359

[23] Zhang M L, Zhou Z H . A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 2014, 26( 8): 1819–1837

[24] Jia B B, Zhang M L . Multi-dimensional classification via kNN feature augmentation. Pattern Recognition, 2020, 106: 107423

[25] Wu T F, Lin C J, Weng R C . Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research, 2004, 5: 975–1005

[26] Zhang M L, Zhou Z H. A k-nearest neighbor based algorithm for multi-label classification. In: Proceedings of 2005 IEEE International Conference on Granular Computing. 2005, 718−721

[27] Tang L, Rajan S, Narayanan V K. Large scale multi-label classification via metalabeler. In: Proceedings of the 18th International Conference on World Wide Web. 2009, 211−220

[28] Wu J H, Wu X, Chen Q G, Hu Y, Zhang M L. Feature-induced manifold disambiguation for multi-view partial multi-label learning. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 557−565

[29] Jia B B, Zhang M L . Multi-dimensional classification via selective feature augmentation. Machine Intelligence Research, 2022, 19( 1): 38–51

[30] Jia B B, Zhang M L . Multi-dimensional classification via stacked dependency exploitation. Science China Information Sciences, 2020, 63( 12): 222102

[31] Zhang Y, Zhao P, Cao J, Ma W, Huang J, Wu Q, Tan M. Online adaptive asymmetric active learning for budgeted imbalanced data. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 2768−2777

[32] Wang J, Zhang M L. Towards mitigating the class-imbalance problem for partial label learning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 2427−2436

[33] Marchant N G, Rubinstein B I P. Needle in a haystack: label-efficient evaluation under extreme class imbalance. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1180−1190

[34] Feng B, Wang Y, Li G, Xie Y, Ding Y. Palleon: a runtime system for efficient video processing toward dynamic class skew. In: Proceedings of 2021 USENIX Annual Technical Conference. 2021, 427−441

[35] Buda M, Maki A, Mazurowski M A . A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 2018, 106: 249–259

[36] Liu Z, Miao Z, Zhan X, Wang J, Gong B, Yu S X. Large-scale long-tailed recognition in an open world. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 2537−2546

[37] Shen L, Lin Z, Huang Q. Relay backpropagation for effective learning of deep convolutional neural networks. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 467−482

[38] Wang S, Liu W, Wu J, Cao L, Meng Q, Kennedy P J. Training deep neural networks on imbalanced data sets. In: Proceedings of 2016 International Joint Conference on Neural Networks. 2016, 4368−4374

[39] Cui Y, Jia M, Lin T Y, Song Y, Belongie S. Class-balanced loss based on effective number of samples. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 9268−9277

[40] Cao K, Wei C, Gaidon A, Aréchiga N, Ma T. Learning imbalanced datasets with label-distribution-aware margin loss. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 140

[41] Ye H J, Chen H Y, Zhan D C, Chao W L. Identifying and compensating for feature deviation in imbalanced deep learning. 2020, arXiv preprint arXiv: 2001.01385

[42] Ren J, Yu C, Sheng S, Ma X, Zhao H, Yi S, Li H. Balanced meta-softmax for long-tailed visual recognition. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 351

[43] Zhou Z H . Learnware: on the future of machine learning. Frontiers of Computer Science, 2016, 10( 4): 589–590

[44] Kuzborskij I, Orabona F . Fast rates by transferring from auxiliary hypotheses. Machine Learning, 2017, 106( 2): 171–195

[45] Du S S, Koushik J, Singh A, Póczos B. Hypothesis transfer learning via transformation functions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 574−584

[46] Li X, Grandvalet Y, Davoine F. Explicit inductive bias for transfer learning with convolutional networks. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 2830−2839

[47] Srinivas S, Fleuret F. Knowledge transfer with Jacobian matching. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 4730−4738

[48] Ye H J, Zhan D C, Jiang Y, Zhou Z H. Rectify heterogeneous models with semantic mapping. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1904−1913

[49] Ye H J, Zhan D C, Jiang Y, Zhou Z H . Heterogeneous few-shot model rectification with semantic mapping. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43( 11): 3878–3891

[50] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. 2015, arXiv preprint arXiv: 1503.02531

[51] Phuong M, Lampert C. Towards understanding knowledge distillation. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 5142−5151

[52] Gotmare A, Keskar N S, Xiong C, Socher R. A closer look at deep learning heuristics: learning rate restarts, warmup and distillation. In: Proceedings of the 37th International Conference on Learning Representations. 2019

[53] Heo B, Kim J, Yun S, Park H, Kwak N, Choi J Y. A comprehensive overhaul of feature distillation. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 1921−1930

[54] Cho J H, Hariharan B. On the efficacy of knowledge distillation. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 4794−4802

[55] Sau B B, Balasubramanian V N. Deep model compression: distilling knowledge from noisy teachers. 2016, arXiv preprint arXiv: 1610.09650

[56] Wang Q, Zhan L, Thompson P, Zhou J. Multimodal learning with incomplete modalities by knowledge distillation. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 1828−1838

[57] Liang S, Gong M, Pei J, Shou L, Zuo W, Zuo X, Jiang D. Reinforced iterative knowledge distillation for cross-lingual named entity recognition. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 3231−3239

[58] Zhang W, Jiang Y, Li Y, Sheng Z, Shen Y, Miao X, Wang L, Yang Z, Cui B. ROD: reception-aware online distillation for sparse graphs. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 2232−2242

[59] Xu C, Li Q, Ge J, Gao J, Yang X, Pei C, Sun F, Wu J, Sun H, Ou W. Privileged features distillation at Taobao recommendations. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2590−2598

[60] Yu A, Grauman K. Fine-grained visual comparisons with local learning. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014, 192−199

[61] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770−778

[62] Liu C, Zhao P, Huang S J, Jiang Y, Zhou Z H. Dual set multi-label learning. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 3635−3642

[63] Ge Y, Abu-El-Haija S, Xin G, Itti L. Zero-shot synthesis with group-supervised learning. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[64] van der Maaten L, Hinton G . Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9( 86): 2579–2605

Acknowledgments

This research was supported by the National Key R&D Program of China (2020AAA0109401, 2020AAA0109405), (62376118, 62006112, 62250069, 62206245), the Young Elite Scientists Sponsorship Program of Jiangsu Association for Science and the Technology 2021-020, Collaborative Innovation Center of Novel Software Technology and Industrialization.

Competing interests

The authors declare that they have no competing interests or financial conflits to disclose.

RIGHTS & PERMISSIONS

2025 Higher Education Press