Integrating Hierarchical Dependencies and Correlation for Multi-Label Text Classification

Jiaxin ZHU , Guohua LIU

Journal of Donghua University(English Edition) ›› 2026, Vol. 43 ›› Issue (5) : 121 -130.

PDF (2349KB)
Journal of Donghua University(English Edition) ›› 2026, Vol. 43 ›› Issue (5) :121 -130. DOI: 10.19884/j.1672-5220.202503006
Information Technology and Artificial Intelligence
research-article
Integrating Hierarchical Dependencies and Correlation for Multi-Label Text Classification
Author information +
History +
PDF (2349KB)

Abstract

This study addresses the challenges in scientific paper classification caused by the exponential growth of scientific papers, which intensifies demands for efficient manuscript categorization and cross-disciplinary knowledge retrieval. To overcome limitations of conventional methods in handling multi-label scenarios and domain-specific nuances, a novel multi-label text classification model is proposed, integrating semantic-aware label correlation modeling with advanced text representation learning. This model constructs a discipline-aware label graph through a graph attention network, incorporating hierarchical information and co-occurrence patterns from domain knowledge, while employing DeBERTaV3 to capture deep contextual semantics through its enhanced Transformer architecture. Text embeddings and label representations undergo systematic fusion, with hybrid sampling strategies and text augmentation techniques mitigating data imbalance issues. Experimental results demonstrate that the proposed model achieves higher performance in cross-disciplinary text classification compared to existing state-of-the-art methods.

Keywords

multi-label text classification / graph attention network / DeBERTaV3 / label representation learning

Cite this article

Download citation ▾
Jiaxin ZHU, Guohua LIU. Integrating Hierarchical Dependencies and Correlation for Multi-Label Text Classification. Journal of Donghua University(English Edition), 2026, 43 (5) : 121-130 DOI:10.19884/j.1672-5220.202503006

登录浏览全文

4963

注册一个新账户 忘记密码

References

[1]

Dhar A, Mukherjee H, Dash N S, et al. Text categorization: past and present[J]. Artificial Intelligence Review, 2021, 54(4): 3007-3054.

[2]

Cover T, Hart P. Nearest neighbor pattern classification[J]. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.

[3]

Mitchell T M. Machine learning[M]. New York: McGraw-Hill, 1997.

[4]

Joachims T. Making large-scale support vector machine learning practical[M]// Advances in Kernel Methods. Cambridge: The MIT Press, 1998.

[5]

Boutell M R, Luo J B, Shen X P, et al. Learning multi-label scene classification[J]. Pattern Recognition, 2004, 37(9): 1757-1771.

[6]

Tsoumakas G, Katakis I. Multi-label classification: an overview[J]. International Journal of Data Warehousing and Mining, 2007, 3(3): 1-13.

[7]

Read J, Pfahringer B, Holmes G, et al. Classifier chains for multi-label classification[J]. Machine Learning, 2011, 85(3): 333-359.

[8]

Clare A, King R D. Knowledge discovery in multi-label phenotype data[C]// Principles of Data Mining and Knowledge Discovery. Berlin, Heidelberg: Springer, 2001: 42-53.

[9]

Elisseeff A, Weston J. A kernel method for multi-labelled classification[M]// Advances in Neural Information Processing Systems 14. Cambridge: The MIT Press, 2002: 681-688.

[10]

Zhang M L, Zhou Z H. ML-KNN: a lazy learning approach to multi-label learning[J]. Pattern Recognition, 2007, 40(7): 2038-2048.

[11]

Chawla N V, Bowyer K W, Hall L O, et al. SMOTE: synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16(1): 321-357.

[12]

Huang Y, Giledereli B, Köksal A, et al. Balancing methods for multi-label text classification with long-tailed class distribution[PP/OL]. V2. arXiv (2021-10-15)[2025-05-02]. https://arxiv.org/pdf/2109.04712v2.

[13]

Mikolov T, Chen K, Corrado G S, et al. Efficient estimation of word representations in vector space[PP/OL]. V3. arXiv (2013-09-07)[2025-05-02]. https://arxiv.org/pdf/1301.3781.

[14]

Pennington J, Socher R, Manning C. GloVe: global vectors for word representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Kerrville: Association for Computational Linguistics, 2014: 1532-1543.

[15]

Ilić S, Marrese-Taylor E, Balazs J, et al. Deep contextualized word representations for detecting sarcasm and irony[PP/OL]. V1. arXiv (2018-09-26)[2025-05-02]. https://arxiv.org/pdf/1809.09795.

[16]

Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[PP/OL]. V2. arXiv (2019-05-24)[2025-05-02]. https://arxiv.org/pdf/1810.04805.

[17]

Elman J L. Finding structure in time[J]. Cognitive Science, 1990, 14(2): 179-211.

[18]

LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[C]// Proceedings of the IEEE. Piscataway: IEEE, 2002: 2278-2324.

[19]

Scarselli F, Gori M, Tsoi A C, et al. The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.

[20]

Kim Y. Convolutional neural networks for sentence classification[PP/OL]. V2. arXiv (2014-09-03)[2025-05-02]. https://arxiv.org/pdf/1408.5882.

[21]

Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[PP/OL]. V4. arXiv (2017-02-22)[2025-05-02]. https://arxiv.org/pdf/1609.02907.

[22]

Veličković P, Cucurull G, Casanova A, et al. Graph attention networks[PP/OL]. V3. arXiv (2018-02-04)[2025-05-02]. https://arxiv.org/pdf/1710.10903.

[23]

Xiao L, Huang X, Chen B L, et al. Label-specific document representation for multi-label text classification[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Kerrville: Association for Computational Linguistics, 2019: 466-475.

[24]

Vu H, Nguyen M, Nguyen V, et al. Label correlation based graph convolutional network for multi-label text classification[C]// 2022 International Joint Conference on Neural Networks (IJCNN). Piscataway: IEEE, 2022: 1-8.

[25]

He P C, Liu X D, Gao J F, et al. DeBERTa: decoding-enhanced BERT with disentangled attention[PP/OL]. V6. arXiv (2021-10-06)[2025-05-02]. https://arxiv.org/pdf/2006.03654.

[26]

Clark K, Luong M T, Le Q V, et al. ELECTRA: pre-training text encoders as discriminators rather than generators[PP/OL]. arXiv (2020-03-23)[2025-05-02]. https://arxiv.org/pdf/2003.10555.

[27]

Hadsell R, Rao D, Rusu A A, et al. Embracing change: continual learning in deep neural networks[J]. Trends in Cognitive Sciences, 2020, 24(12): 1028-1040.

[28]

He P C, Gao J F, Chen W Z. DeBERTaV3: improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing[PP/OL]. V4. arXiv (2023-03-24)[2025-05-02]. https://arxiv.org/pdf/2111.09543.

[29]

Kowsari K, Brown D E, Heidarysafa M, et al. HDLTex: hierarchical deep learning for text classification[C]// 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). Piscataway: IEEE, 2018: 364-371.

[30]

Yang P C, Sun X, Li W, et al. SGM: sequence generation model for multi-label classification[C]// Proceedings of the 27th International Conference on Computational Linguistics. Kerrville: Association for Computational Linguistics, 2018: 3915-3926.

[31]

You R, Zhang Z, Wang Z, et al. AttentionXML: label tree-based attention-aware deep model for high performance extreme multi-label text classification[J]. Advances in Neural Information Processing Systems, 2019: 5812-5822.

[32]

Xiao L, Zhang X L, Jing L P, et al. Does head label help for long-tailed multi-label text classification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(16): 14103-14111.

[33]

Ma Q W, Yuan C Y, Zhou W, et al. Label-specific dual graph neural network for multi-label text classification[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Kerrville: Association for Computational Linguistics, 2021: 3855-3864.

PDF (2349KB)

0

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉