Representation learning: serial-autoencoder for personalized recommendation

Abstract

Nowadays, the personalized recommendation has become a research hotspot for addressing information overload. Despite this, generating effective recommendations from sparse data remains a challenge. Recently, auxiliary information has been widely used to address data sparsity, but most models using auxiliary information are linear and have limited expressiveness. Due to the advantages of feature extraction and no-label requirements, autoencoder-based methods have become quite popular. However, most existing autoencoder-based methods discard the reconstruction of auxiliary information, which poses huge challenges for better representation learning and model scalability. To address these problems, we propose Serial-Autoencoder for Personalized Recommendation (SAPR), which aims to reduce the loss of critical information and enhance the learning of feature representations. Specifically, we first combine the original rating matrix and item attribute features and feed them into the first autoencoder for generating a higher-level representation of the input. Second, we use a second autoencoder to enhance the reconstruction of the data representation of the prediciton rating matrix. The output rating information is used for recommendation prediction. Extensive experiments on the MovieTweetings and MovieLens datasets have verified the effectiveness of SAPR compared to state-of-the-art models.

Graphical abstract

Keywords

personalized recommendation / autoencoder / representation learning / collaborative filtering

Yi Zhu is currently an assistant professor in the School of Information Engineering, Yangzhou University, China. He received the BS degree from Anhui University, the MS degree from the University of Science and Technology of China, and the PhD from Hefei University of Technology, China. His research interests include data mining and recommendation systems

Yishuai Geng is currently a graduate student in the School of Information Engineering, Yangzhou University, China. He received the BS degree from Wuxi Taihu University, China. His research interests include recommendation system and data mining

Yun Li is currently a professor in the School of Information Engineering, Yangzhou University, China. He received the MS degree in computer science and technology from Hefei University of Technology, China in 1991, and the PhD in control theory and control engineering from Shanghai University, China in 2005. He has published more than 100 scientific papers. His research interests include data mining and cloud computing

Jipeng Qiang is currently an associate professor in the School of Information Engineering of Yangzhou University, China. He received his PhD in computer science and technology from Hefei University of Technology, China in 2016. He was a PhD visiting student in the Artificial Intelligence Lab at the University of Massachusetts Boston, USA from 2014 to 2016. He has published more than 40 papers, including AAAI, TKDE, TKDD, and TASLP. His research interests mainly include natural language processing and data mining

Xindong Wu is a professor in the School of Computer Science and Information Engineering of Hefei University of Technology, China, and the president of Mininglamp Academy of Sciences, Minininglamp, China, and a fellow of IEEE and AAAS. He received his BS and MS degrees in computer science from Hefei University of Technology, China, and his PhD degree in artificial intelligence from the University of Edinburgh, Britain. His research interests include data mining, big data analytics, and knowledge-based systems

References

[1] Geng Y, Zhu Y, Li Y, Sun X, Li B . Multi-feature extension via semi-autoencoder for personalized recommendation. Applied Sciences, 2022, 12( 23): 12408

[2] Liu Y, Liang C, Chiclana F, Wu J . A knowledge coverage-based trust propagation for recommendation mechanism in social network group decision making. Applied Soft Computing, 2021, 101: 107005

[3] Rahayu N W, Ferdiana R, Kusumawardani S S . A systematic review of ontology use in E-Learning recommender system. Computers and Education: Artificial Intelligence, 2022, 3: 100047

[4] Rajendran D P D, Sundarraj R P . Using topic models with browsing history in hybrid collaborative filtering recommender system: experiments with user ratings. International Journal of Information Management Data Insights, 2021, 1( 2): 100027

[5] Ghasemi N, Momtazi S . Neural text similarity of user reviews for improving collaborative filtering recommender systems. Electronic Commerce Research and Applications, 2021, 45: 101019

[6] Wang F, Zhu H, Srivastava G, Li S, Khosravi M R, Qi L . Robust collaborative filtering recommendation with user-item-trust records. IEEE Transactions on Computational Social Systems, 2022, 9( 4): 986–996

[7] Zhu Y, Li L, Wu X . Stacked convolutional sparse auto-encoders for representation learning. ACM Transactions on Knowledge Discovery from Data, 2021, 15( 2): 31

[8] Zhu Y, Wu X, Qiang J, Yuan Y, Li Y . Representation learning with collaborative autoencoder for personalized recommendation. Expert Systems with Applications, 2021, 186: 115825

[9] Yu M, Quan T, Peng Q, Yu X, Liu L . A model-based collaborate filtering algorithm based on stacked AutoEncoder. Neural Computing and Applications, 2022, 34( 4): 2503–2511

[10] Zhu H, Qian Z, Ye Z, Zhang D. An approach to rating prediction for personality recommendation via attention mechanism and denoising autoencoder. In: Proceedings of 2022 International Conference on Machine Learning, Cloud Computing and Intelligent Mining. 2022, 463−469

[11] Wu S, Sun F, Zhang W, Xie X, Cui B . Graph neural networks in recommender systems: a survey. ACM Computing Surveys, 2023, 55( 5): 97

[12] Yan Y, Cheng D, Feng J E, Li H, Yue J . Survey on applications of algebraic state space theory of logical systems to finite state machines. Science China Information Sciences, 2023, 66( 1): 111201

[13] Zhang L, Luo T, Zhang F, Wu Y . A recommendation model based on deep neural network. IEEE Access, 2018, 6: 9454–9463

[14] Hoyer P O. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 2004, 5(9): 1457−1469

[15] Koren Y. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2008, 426−434

[16] Rashed A, Grabocka J, Schmidt-Thieme L. Attribute-aware non-linear co-embeddings of graph features. In: Proceedings of the 13th ACM Conference on Recommender Systems. 2019, 314−321

[17] He X, Deng K, Wang X, Li Y, Zhang Y, Wang M. LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 639−648

[18] Lu Y, Fang Y, Shi C. Meta-learning on heterogeneous information networks for cold-start recommendation. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 1563−1573

[19] Yu Z, Lian J, Mahmoody A, Liu G, Xie X. Adaptive user modeling with long and short-term preferences for personalized recommendation. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019, 4213−4219

[20] Cheng H T, Koc L, Harmsen J, Shaked T, Chandra T, Aradhye H, Anderson G, Corrado G, Chai W, Ispir M, Anil R, Haque Z, Hong L, Jain V, Liu X, Shah H. Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. 2016, 7−10

[21] He X, Liao L, Zhang H, Nie L, Hu X, Chua T S. Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web. 2017, 173−182

[22] He X, Chua T S. Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2017, 355−364

[23] Mu R . A survey of recommender systems based on deep learning. IEEE Access, 2018, 6: 69009–69022

[24] Yang S, Wang Y, Chu X. A survey of deep learning techniques for neural machine translation. 2020, arXiv preprint arXiv: 2002.07526

[25] Subramanian A S, Weng C, Watanabe S, Yu M, Yu D . Deep learning based multi-source localization with source splitting and its effectiveness in multi-talker speech recognition. Computer Speech & Language, 2022, 75: 101360

[26] Zhu Y, Lin Q, Lu H, Shi K, Qiu P, Niu Z . Recommending scientific paper via heterogeneous knowledge embedding based attentive recurrent neural networks. Knowledge-Based Systems, 2021, 215: 106744

[27] Alamdari P M, Navimipour N J, Hosseinzadeh M, Safaei A A, Darwesh A . Image-based product recommendation method for E-commerce applications using convolutional neural networks. Acta Informatica Pragensia, 2022, 11( 1): 15–35

[28] Tahmasebi H, Ravanmehr R, Mohamadrezaei R . Social movie recommender system based on deep autoencoder network using Twitter data. Neural Computing and Applications, 2021, 33( 5): 1607–1623

[29] Askari B, Szlichta J, Salehi-Abari A. Variational autoencoders for Top-K recommendation with implicit feedback. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 2061−2065

[30] Zhu Y, Chen Z. Mutually-regularized dual collaborative variational auto-encoder for recommendation systems. In: Proceedings of the ACM Web Conference 2022. 2022, 2379−2387

[31] Zhang S, Yao L, Xu X, Wang S, Zhu L. Hybrid collaborative recommendation via semi-AutoEncoder. In: Proceedings of the 24th International Conference on Neural Information Processing. 2017, 185−193

[32] Yang Y, Zhu Y, Li Y . Personalized recommendation with knowledge graph via dual-autoencoder. Applied Intelligence, 2022, 52( 6): 6196–6207

[33] Nurmaini S, Darmawahyuni A, Mukti A N S, Rachmatullah M N, Firdaus F, Tutuko B . Deep learning-based stacked denoising and autoencoder for ECG heartbeat classification. Electronics, 2020, 9( 1): 135

[34] Zhang G, Liu Y, Jin X . A survey of autoencoder-based recommender systems. Frontiers of Computer Science, 2020, 14( 2): 430–450

[35] Xie Z, Liu C, Zhang Y, Lu H, Wang D, Ding Y. Adversarial and contrastive variational autoencoder for sequential recommendation. In: Proceedings of the Web Conference 2021. 2021, 449−459

[36] Jana D, Patil J, Herkal S, Nagarajaiah S, Duenas-Osorio L . CNN and Convolutional Autoencoder (CAE) based real-time sensor fault detection, localization, and correction. Mechanical Systems and Signal Processing, 2022, 169: 108723

[37] Zhu Y, Dong B, Sha Z. Personalized recommendation based on entity attributes and graph features. In: Proceedings of 2021 IEEE International Conference on Big Knowledge. 2021, 7−14

[38] Geng Y, Xiao X, Sun X, Zhu Y . Representation learning: Recommendation with knowledge graph via triple-autoencoder. Frontiers in Genetics, 2022, 13: 891265

[39] Dooms S, De Pessemier T, Martens L. MovieTweetings: a movie rating dataset collected from twitter. In: Proceedings of the Workshop on Crowdsourcing and Human Computation for Recommender Systems, Held in Conjunction with the 7th ACM Conference on Recommender Systems. 2013, 43

[40] Lee J, Sun M, Lebanon G . PREA: personalized recommendation algorithms toolkit. The Journal of Machine Learning Research, 2012, 13( 1): 2699–2703

Acknowledgements

This research was partially supported by the National Natural Science Foundation of China (Grant Nos. 61906060, 62076217, and 62120106008), National Key R&D Program of China (No. 2016YFC0801406), and the Natural Science Foundation of the Jiangsu Higher Education Institutions (No. 20KJB520007).

RIGHTS & PERMISSIONS

2024 Higher Education Press