Hierarchical long and short-term user preference modeling for sequential recommendation

Abstract

Sequential recommendation is an important research task in the field of recommendation systems, where precise modeling of the dynamic evolution of user interests from historical interactions is essential for enhancing performance. To address the limitations of existing methods in capturing the diversity of long-term interests, the dynamics of short-term user interest, and the hierarchical relationship between them, this paper proposes an end-to-end hierarchical long and short-term sequential recommendation model. First, the proposed model leverages a dynamic routing mechanism to adaptively aggregate users’ long-term historical interactions, generating a multi-vector representation of long-term user preference. Simultaneously, a self-attention mechanism is employed to aggregate short-term interaction sequences, effectively capturing short-term user interest. In addition, a hierarchical matching mechanism is designed to align long and short-term user interest, mining the long-term user preference most relevant to the current short-term user interest through similarity-based extraction, and fusing them using time encoding to produce the final user preference representation. Finally, a prediction framework based on attention mechanisms integrates both long-term user preference and short-term interaction information to achieve efficient sequential recommendation. The experimental results indicate that the proposed method achieves significantly better performance than existing state-of-the-art sequential recommendation models across multiple evaluation metrics, validating its effectiveness and superiority.

Graphical abstract

Keywords

sequential recommendation / user preference representation / hierarchical modeling / recommender system

References

[1] Fang H, Guo G, Zhang D, Shu Y. Deep learning-based sequential recommender systems: concepts, algorithms, and evaluations. In: Proceedings of the 19th International Conference on Web Engineering. 2019, 574−577

[2] Wang S, Hu L, Wang Y, Cao L, Sheng Q Z, Orgun M. Sequential recommender systems: challenges, progress and prospects. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 6332−6338

[3] Fan Z, Liu Z, Wang S, Zheng L, Yu P S. Modeling sequences as distributions with uncertainty for sequential recommendation. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 3019−3023

[4] Fang H, Zhang D, Shu Y, Guo G . Deep learning for sequential recommendation: algorithms, influential factors, and evaluations. ACM Transactions on Information Systems (TOIS), 2021, 39( 1): 1–42

[5] Wang S, Cao L, Wang Y, Sheng Q Z, Orgun M A, Lian D . A survey on session-based recommender systems. ACM Computing Surveys (CSUR), 2022, 54( 7): 154

[6] Wu S, Sun F, Zhang W, Xie X, Cui B . Graph neural networks in recommender systems: a survey. ACM Computing Surveys, 2023, 55( 5): 97

[7] Fan M, Huang Y, Qalati S A, Shah S M M, Ostic D, Pu Z . Effects of information overload, communication overload, and inequality on digital distrust: a cyber-violence behavior mechanism. Frontiers in Psychology, 2021, 12: 643981

[8] Batmaz Z, Yurekli A, Bilge A, Kaleli C . A review on deep learning for recommender systems: challenges and remedies. Artificial Intelligence Review, 2019, 52( 1): 1–37

[9] Bontempelli T, Chapus B, Rigaud F, Morlon M, Lorant M, Salha-Galvan G. Flow moods: recommending music by moods on deezer. In: Proceedings of the 16th ACM Conference on Recommender Systems. 2022, 452−455

[10] Chen C W, Lamere P, Schedl M, Zamani H. Recsys challenge 2018: automatic music playlist continuation. In: Proceedings of the 12th ACM Conference on Recommender Systems. 2018, 527−528

[11] Quadrana M, Cremonesi P, Jannach D . Sequence-aware recommender systems. ACM Computing Surveys (CSUR), 2019, 51( 4): 66

[12] Sun F, Liu J, Wu J, Pei C, Lin X, Ou W, Jiang P. BERT4Rec: sequential recommendation with bidirectional encoder representations from transformer. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2019, 1441−1450

[13] Guo Q, Zhuang F, Qin C, Zhu H, Xie X, Xiong H, He Q . A survey on knowledge graph-based recommender systems. IEEE Transactions on Knowledge and Data Engineering, 2022, 34( 8): 3549–3568

[14] Hidasi B, Tikk D . General factorization framework for context-aware recommendations. Data Mining and Knowledge Discovery, 2016, 30( 2): 342–371

[15] He R, Kang W C, McAuley J. Translation-based recommendation: a scalable method for modeling sequential behavior. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 5264−5268

[16] Kombrink S, Mikolov T, Karafiát M, Burget L. Recurrent neural network based language modeling in meeting recognition. In: Proceedings of the 12th Annual Conference of the International Speech Communication Association. 2011, 2877−2880

[17] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[18] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 6000−6010

[19] Hidasi B, Karatzoglou A, Baltrunas L, Tikk D. Session-based recommendations with recurrent neural networks. In: Proceedings of the 4th International Conference on Learning Representations. 2016

[20] Zhang T, Zheng W, Cui Z, Zong Y, Li Y . Spatial−temporal recurrent neural network for emotion recognition. IEEE Transactions on Cybernetics, 2019, 49( 3): 839–847

[21] Zhao P, Luo A, Liu Y, Xu J, Li Z, Zhuang F, Sheng V S, Zhou X . Where to go next: a spatio-temporal gated network for next poi recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022, 34( 5): 2512–2524

[22] Wang E, Jiang Y, Xu Y, Wang L, Yang Y. Spatial-temporal interval aware sequential POI recommendation. In: Proceedings of the IEEE 38th International Conference on Data Engineering. 2022, 2086−2098

[23] Jiang Y, Yang Y, Xu Y, Wang E . Spatial-temporal interval aware individual future trajectory prediction. IEEE Transactions on Knowledge and Data Engineering, 2024, 36( 10): 5374–5387

[24] Beck M, Pöppel K, Spanring M, Auer A, Prudnikova O, Kopp M, Klambauer G, Brandstetter J, Hochreiter S. xLSTM: extended long short-term memory. In: Proceedings of the 38th International Conference on Neural Information Processing Systems., 2024

[25] Jiang Y, Xu Y, Yang Y, Yang F, Wang P, Li C, Zhuang F, Xiong H . T_RIMLP: a foundational MLP-like architecture for sequential recommendation. ACM Transactions on Information Systems, 2024, 42( 6): 157

[26] Yuan G, Yuan F, Li Y, Kong B, Li S, Chen L, Yang M, Yu C, Hu B, Li Z, Xu Y, Qi X. Tenrec: a large-scale multipurpose benchmark dataset for recommender systems. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 834

[27] Xu D, Ruan C, Korpeoglu E, Kumar S, Achan K. A temporal kernel approach for deep learning with continuous-time information. In: Proceedings of the 9th International Conference on Learning Representations. 2021

[28] Yue Z, Wang Y, He Z, Zeng H, McAuley J, Wang D. Linear recurrent units for sequential recommendation. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining. 2024, 930−938

[29] Wu S, Tang Y, Zhu Y, Wang L, Xie X, Tan T. Session-based recommendation with graph neural networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 346−353

[30] Fan Z, Liu Z, Zhang J, Xiong Y, Zheng L, Yu P S. Continuous-time sequential recommendation with temporal graph collaborative transformer. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 433−442

[31] Hsu C, Li C T. RetaGNN: relational temporal attentive graph neural networks for holistic sequential recommendation. In: Proceedings of the Web Conference 2021. 2021, 2968−2979

[32] Zhang M, Wu S, Yu X, Liu Q, Wang L . Dynamic graph neural networks for sequential recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022, 35( 5): 4741–4753

[33] Liu Y, Xia L, Huang C. SelfGNN: self-supervised graph neural networks for sequential recommendation. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2024, 1609−1618

[34] Kang W C, McAuley J. Self-attentive sequential recommendation. In: Proceedings of 2018 IEEE International Conference on Data Mining. 2018, 197−206

[35] Li J, Wang Y, McAuley J. Time interval aware self-attention for sequential recommendation. In: Proceedings of the 13th International Conference on Web Search and Data Mining. 2020, 322−330

[36] Tanjim M M, Su C, Benjamin E, Hu D, Hong L, McAuley J. Attentive sequential models of latent intent for next item recommendation. In: Proceedings of the Web Conference 2020. 2020, 2528−2534

[37] Fan Z, Liu Z, Wang Y, Wang A, Nazari Z, Zheng L, Peng H, Yu P S. Sequential recommendation via stochastic self-attention. In: Proceedings of the ACM Web Conference 2022. 2022, 2036−2047

[38] Liu Q, Zeng Y, Mokhosi R, Zhang H. STAMP: short-term attention/memory priority model for session-based recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018, 1831−1839

[39] Cho S M, Park E, Yoo S. MEANTIME: mixture of attention mechanisms with multi-temporal embeddings for sequential recommendation. In: Proceedings of the 14th ACM Conference on Recommender Systems. 2020, 515−520

[40] Tran V A, Salha-Galvan G, Sguerra B, Hennequin R. Attention mixtures for time-aware sequential recommendation. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023, 1821−1826

[41] Dang Y, Yang E, Guo G, Jiang L, Wang X, Xu X, Sun Q, Liu H. Uniform sequence better: time interval aware data augmentation for sequential recommendation. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence. 2023, 4225−4232

[42] Du X, Yuan H, Zhao P, Qu J, Zhuang F, Liu G, Liu Y, Sheng V S. Frequency enhanced hybrid attention network for sequential recommendation. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023, 78−88

[43] Zhu S, Ye J, Jiang W, Xue S, Zhang Q, Wu Y, Li J. CoCA: fusing position embedding with collinear constrained attention in transformers for long context window extending. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 2024, 4247−4262

[44] Tian Z, Zhao W X, Zhang C, Zhao X, Ma Z, Wen J R. EulerFormer: sequential user behavior modeling with complex vector attention. In: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2024, 1619−1628

[45] He R, McAuley J. Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th International Conference on World Wide Web. 2016, 507−517

[46] McAuley J, Targett C, Shi Q, Van Den Hengel A. Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2015, 43−52

[47] Harper F M, Konstan J A . The MovieLens datasets: history and context. ACM Transactions on Interactive Intelligent Systems (TiiS), 2016, 5( 4): 19

[48] McAuley J, Leskovec J. Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM Conference on Recommender Systems. 2013, 165−172

[49] Ni J, Li J, McAuley J. Justifying recommendations using distantly-labeled reviews and fined-grained aspects. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019, 188−197

[50] Tang J, Wang K. Personalized top-N sequential recommendation via convolutional sequence embedding. In: Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 2018, 565−573

[51] Xie Y, Gao J, Zhou P, Ye Q, Hua Y, Kim J B, Wu F, Kim S. Rethinking multi-interest learning for candidate matching in recommender systems. In: Proceedings of the 17th ACM Conference on Recommender Systems. 2023, 283−293

RIGHTS & PERMISSIONS

Higher Education Press