Revisiting multi-dimensional classification from a dimension-wise perspective

Abstract

Real-world objects exhibit intricate semantic properties that can be characterized from a multitude of perspectives, which necessitates the development of a model capable of discerning multiple patterns within data, while concurrently predicting several Labeling Dimensions (LDs) — a task known as Multi-dimensional Classification (MDC). While the class imbalance issue has been extensively investigated within the multi-class paradigm, its study in the MDC context has been limited due to the imbalance shift phenomenon. A sample’s classification as a minor or major class instance becomes ambiguous when it belongs to a minor class in one LD and a major class in another. Previous MDC methodologies predominantly emphasized instance-wise criteria, neglecting prediction capabilities from a dimension aspect, i.e., the average classification performance across LDs. We assert the significance of dimension-wise metrics in real-world MDC applications and introduce two such metrics. Furthermore, we observe imbalanced class distributions within each LD and propose a novel Imbalance-Aware fusion Model (IMAM) for addressing the MDC problem. Specifically, we first decompose the task into multiple multi-class classification problems, creating imbalance-aware deep models for each LD separately. This straightforward method performs well across LDs without sacrificing performance in instance-wise criteria. Subsequently, we employ LD-wise models as multiple teachers and transfer their knowledge across all LDs to a unified student model. Experimental results on several real-world datasets demonstrate that our IMAM approach excels in both instance-wise evaluations and the proposed dimension-wise metrics.

Graphical abstract

Keywords

multi-dimensional classification / dimension perspective / class imbalance learning

References

[1] Zhang C, Yankov D, Wu C T, Shapiro S, Hong J, Wu W. What is that building?: an end-to-end system for building recognition from streetside images. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2425−2433

[2] Sahoo D, Wang H, Shu K, Wu X, Le H, Achananuparp P, Lim E P, Hoi S C H. FoodAI: food image recognition via deep learning for smart food logging. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 2260−2268

[3] Borisyuk F, Gordo A, Sivakumar V. Rosetta: large scale system for text detection and recognition in images. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 71−79

[4] Yang X, Zeng Z, Teo S G, Wang L, Chandrasekhar V, Hoi S. Deep learning for practical image recognition: case study on kaggle competitions. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 923−931

[5] Wang Z, Long C, Cong G, Ju C. Effective and efficient sports play retrieval with deep representation learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2019, 499−509

[6] Huang J T, Sharma A, Sun S, Xia L, Zhang D, Pronin P, Padmanabhan J, Ottaviano G, Yang L. Embedding-based retrieval in facebook search. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2553−2561

[7] Jia X, Zhao H, Lin Z, Kale A, Kumar V. Personalized image retrieval with sparse graph representation learning. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2735−2743

[8] Tan R, Vasileva M I, Saenko K, Plummer B A. Learning similarity conditions without explicit supervision. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 10373−10382

[9] Lin Y L, Tran S D, Davis L S. Fashion outfit complementary item retrieval. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 3311−3319

[10] Kim D, Saito K, Mishra S, Sclaroff S, Saenko K, Plummer B A. Self-supervised visual attribute learning for fashion compatibility. In: Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. 2021, 1057−1066

[11] Wang Z, Wang Y, Feng B, Mudigere D, Muthiah B, Ding Y. El-rec: efficient large-scale recommendation model training via tensor-train embedding table. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 2022, 1−14

[12] Kononenko I . Machine learning for medical diagnosis: history, state of the art and perspective. Artificial Intelligence in Medicine, 2001, 23( 1): 89–109

[13] Amato F, López A, María Peña-Méndez E, Vaňhara P, Hampl A, Havel J . Artificial neural networks in medical diagnosis. Journal of Applied Biomedicine, 2013, 11( 2): 47–58

[14] Turnbull D, Barrington L, Torres D, Lanckriet G . Semantic annotation and retrieval of music and sound effects. IEEE Transactions on Audio, Speech, and Language Processing, 2008, 16( 2): 467–476

[15] Serafino F, Pio G, Ceci M, Malerba D. Hierarchical multidimensional classification of web documents with MultiWebClass. In: Proceedings of the 18th International Conference on Digital Society. 2015, 236−250

[16] Shatkay H, Pan F, Rzhetsky A, Wilbur W J . Multi-dimensional classification of biomedical text: toward automated, practical provision of high-utility text to diverse users. Bioinformatics, 2008, 24( 18): 2086–2093

[17] Barutcuoglu Z, Schapire R E, Troyanskaya O G . Hierarchical multi-label prediction of gene function. Bioinformatics, 2006, 22( 7): 830–836

[18] Feng B, Wang Y, Ding Y. Saga: sparse adversarial attack on EEG-based brain computer interface. In: Proceedings of 2021 IEEE International Conference on Acoustics, Speech and Signal Processing. 2021, 975−979

[19] Jia B B, Zhang M L. Multi-dimensional classification via sparse label encoding. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 4917−4926

[20] Wang H, Chen C, Liu W, Chen K, Hu T, Chen G. Incorporating label embedding and feature augmentation for multi-dimensional classification. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 6178−6185

[21] Ma Z, Chen S . Multi-dimensional classification via a metric approach. Neurocomputing, 2018, 275: 1121–1131

[22] Read J, Pfahringer B, Holmes G, Frank E . Classifier chains for multi-label classification. Machine Learning, 2011, 85( 3): 333–359

[23] Zhang M L, Zhou Z H . A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 2014, 26( 8): 1819–1837

[24] Jia B B, Zhang M L . Multi-dimensional classification via kNN feature augmentation. Pattern Recognition, 2020, 106: 107423

[25] Wu T F, Lin C J, Weng R C . Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research, 2004, 5: 975–1005

[26] Zhang M L, Zhou Z H. A k-nearest neighbor based algorithm for multi-label classification. In: Proceedings of 2005 IEEE International Conference on Granular Computing. 2005, 718−721

[27] Tang L, Rajan S, Narayanan V K. Large scale multi-label classification via metalabeler. In: Proceedings of the 18th International Conference on World Wide Web. 2009, 211−220

[28] Wu J H, Wu X, Chen Q G, Hu Y, Zhang M L. Feature-induced manifold disambiguation for multi-view partial multi-label learning. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 557−565

[29] Jia B B, Zhang M L . Multi-dimensional classification via selective feature augmentation. Machine Intelligence Research, 2022, 19( 1): 38–51

[30] Jia B B, Zhang M L . Multi-dimensional classification via stacked dependency exploitation. Science China Information Sciences, 2020, 63( 12): 222102

[31] Zhang Y, Zhao P, Cao J, Ma W, Huang J, Wu Q, Tan M. Online adaptive asymmetric active learning for budgeted imbalanced data. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 2768−2777

[32] Wang J, Zhang M L. Towards mitigating the class-imbalance problem for partial label learning. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 2427−2436

[33] Marchant N G, Rubinstein B I P. Needle in a haystack: label-efficient evaluation under extreme class imbalance. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1180−1190

[34] Feng B, Wang Y, Li G, Xie Y, Ding Y. Palleon: a runtime system for efficient video processing toward dynamic class skew. In: Proceedings of 2021 USENIX Annual Technical Conference. 2021, 427−441

[35] Buda M, Maki A, Mazurowski M A . A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 2018, 106: 249–259

[36] Liu Z, Miao Z, Zhan X, Wang J, Gong B, Yu S X. Large-scale long-tailed recognition in an open world. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 2537−2546

[37] Shen L, Lin Z, Huang Q. Relay backpropagation for effective learning of deep convolutional neural networks. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 467−482

[38] Wang S, Liu W, Wu J, Cao L, Meng Q, Kennedy P J. Training deep neural networks on imbalanced data sets. In: Proceedings of 2016 International Joint Conference on Neural Networks. 2016, 4368−4374

[39] Cui Y, Jia M, Lin T Y, Song Y, Belongie S. Class-balanced loss based on effective number of samples. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 9268−9277

[40] Cao K, Wei C, Gaidon A, Aréchiga N, Ma T. Learning imbalanced datasets with label-distribution-aware margin loss. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 140

[41] Ye H J, Chen H Y, Zhan D C, Chao W L. Identifying and compensating for feature deviation in imbalanced deep learning. 2020, arXiv preprint arXiv: 2001.01385

[42] Ren J, Yu C, Sheng S, Ma X, Zhao H, Yi S, Li H. Balanced meta-softmax for long-tailed visual recognition. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 351

[43] Zhou Z H . Learnware: on the future of machine learning. Frontiers of Computer Science, 2016, 10( 4): 589–590

[44] Kuzborskij I, Orabona F . Fast rates by transferring from auxiliary hypotheses. Machine Learning, 2017, 106( 2): 171–195

[45] Du S S, Koushik J, Singh A, Póczos B. Hypothesis transfer learning via transformation functions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 574−584

[46] Li X, Grandvalet Y, Davoine F. Explicit inductive bias for transfer learning with convolutional networks. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 2830−2839

[47] Srinivas S, Fleuret F. Knowledge transfer with Jacobian matching. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 4730−4738

[48] Ye H J, Zhan D C, Jiang Y, Zhou Z H. Rectify heterogeneous models with semantic mapping. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1904−1913

[49] Ye H J, Zhan D C, Jiang Y, Zhou Z H . Heterogeneous few-shot model rectification with semantic mapping. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43( 11): 3878–3891

[50] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network. 2015, arXiv preprint arXiv: 1503.02531

[51] Phuong M, Lampert C. Towards understanding knowledge distillation. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 5142−5151

[52] Gotmare A, Keskar N S, Xiong C, Socher R. A closer look at deep learning heuristics: learning rate restarts, warmup and distillation. In: Proceedings of the 37th International Conference on Learning Representations. 2019

[53] Heo B, Kim J, Yun S, Park H, Kwak N, Choi J Y. A comprehensive overhaul of feature distillation. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 1921−1930

[54] Cho J H, Hariharan B. On the efficacy of knowledge distillation. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 4794−4802

[55] Sau B B, Balasubramanian V N. Deep model compression: distilling knowledge from noisy teachers. 2016, arXiv preprint arXiv: 1610.09650

[56] Wang Q, Zhan L, Thompson P, Zhou J. Multimodal learning with incomplete modalities by knowledge distillation. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 1828−1838

[57] Liang S, Gong M, Pei J, Shou L, Zuo W, Zuo X, Jiang D. Reinforced iterative knowledge distillation for cross-lingual named entity recognition. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 3231−3239

[58] Zhang W, Jiang Y, Li Y, Sheng Z, Shen Y, Miao X, Wang L, Yang Z, Cui B. ROD: reception-aware online distillation for sparse graphs. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 2232−2242

[59] Xu C, Li Q, Ge J, Gao J, Yang X, Pei C, Sun F, Wu J, Sun H, Ou W. Privileged features distillation at Taobao recommendations. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 2590−2598

[60] Yu A, Grauman K. Fine-grained visual comparisons with local learning. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014, 192−199

[61] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770−778

[62] Liu C, Zhao P, Huang S J, Jiang Y, Zhou Z H. Dual set multi-label learning. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 3635−3642

[63] Ge Y, Abu-El-Haija S, Xin G, Itti L. Zero-shot synthesis with group-supervised learning. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[64] van der Maaten L, Hinton G . Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9( 86): 2579–2605

RIGHTS & PERMISSIONS

Higher Education Press