Integrating Machine Learning and Evidential Reasoning for User Profiling and Recommendation

Abstract

User profiles representing users’ preferences and interests play an important role in many applications of personalized recommendation. With the rapid growth of social platforms, there is a critical need for efficient solutions to learn user profiles from the information they shared on social platforms so as to improve the quality of recommendation services. The problem of user profile learning is significantly challenging due to difficulty in handling data from multiple sources, in different formats and often associated with uncertainty. In this paper, we introduce an integrated approach that combines advanced Machine Learning techniques with evidential reasoning based on Dempster-Shafer theory of evidence for user profiling and recommendation. The developed methods for user profile learning and multi-criteria collaborative filtering are demonstrated with experimental results and analysis that show the effectiveness and practicality of the integrated approach. A proposal for extending multi-criteria recommendation systems by incorporating user profiles learned from different sources of data into the recommendation process so as to provide better recommendation capabilities is also highlighted.

Keywords

Machine learning / Dempster-Shafer theory of evidence / user profiles / personalized recommendation / preferences

References

[1] Adomavicius G, Kwon Y. New recommendation techniques for multicriteria rating systems. IEEE Intelligent Systems, 2007, 22(3): 48-55.

[2] Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(6): 734-749.

[3] Al-Ghuribi S M, Noah SAM. Multi-criteria review-based recommender system - The state of the art. IEEE Access, 2019, 7: 169446-169468.

[4] Alekseev A, Nikolenko S (2017). Word embeddings for user profiling in online social networks. Computación y Sistemas 21(2).

[5] Balog K. Expertise retrieval. Foundations and Trends in Information Retrieval, 2012, 6(2–3): 127-256.

[6] Balog K, de Rijke M. Determining expert profiles (with an application to expert finding). IJCAI’07: Proceedings of the 20th International Jjoint Conference on Artifical Intelligence, 2007, San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. 2657-2662.

[7] Barrios F, López F, Argerich L, Wachenchauzer R (2016). Variations of the similarity function of textrank for automated summarization. Proceedings of Argentine Symposium on Artificial Intelligence, ASAI, Argentina.

[8] Bergsma S, Durme B V. Using conceptual class attributes to characterize social media users. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2013, Sofia, Bulgaria: Association for Computational Linguistics 710-720.

[9] Blei D M, Ng A Y, Jordan M I. Latent dirichlet allocation. Journal of Machine Learning Research, 2003, 3: 993-1022.

[10] Boudin F. Unsupervised keyphrase extraction with multipartite graphs. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), 2018, New Orleans, Louisiana: Association for Computational Linguistics 667-672.

[11] Burke R. Hybrid recommender systems: Survey and experiments. User Modeling and User-Adapted Interaction, 2002, 12(4): 331-370.

[12] Campos R, Mangaravite V, Pasquali A, Jorge A, Nunes C, Jatowt A. YAKE! Keyword extraction from single documents using multiple local features. Information Sciences, 2020, 509: 257-289.

[13] Cimiano P, Völker J. Text2onto-Aframework for ontology learning and datadriven change discovery. Proceedings of the 10th International Conference on Applications of Natural Language to Information Systems (NLDB), volume 3513 of Lecture Notes in Computer Science, 2005, Alicante, Spain: Springer 227-238.

[14] Cohen R, Ruths D. Classifying political orientation on twitter: It’s not easy!. Proceedings of the 7th International Conference on Weblogs and Social Media, ICWSM 2013, 2013, USA: The AAAI Press 91-99.

[15] Craswell N, de Vries A P, Soboroff I. Overview of the TREC 2005 enterprise track. TREC, 2005, 5: 199-205.

[16] De Lathauwer L, De Moor B, Vandewalle J. A multilinear singular value d ecomposition. SIAM Journal on Matrix Analysis and Applications, 2000, 21(4): 1253-1278.

[17] Deitrick W, Miller Z, Valyou B, Dickinson B, Munson T, Hu W. Gender identification on Twitter using the modified balanced winnow. Communications and Network, 2012, 04(03): 189-195.

[18] Dempster A P. Upper and lower probabilities induced by a multivalued mapping. Classic Works of the Dempster-Shafer Theory of Belief Functions, 2008, Germany: Springer 57-72.

[19] Fang Y, Godavarthy A. Modeling the dynamics of personal expertise. SIGIR’ 14, 2014, New York, NY, USA: Association for Computing Machinery 1107-1110.

[20] Florescu C, Caragea C (2017). PositionRank: An unsuper-vised approach to keyphrase extraction from scholarly documents. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics.

[21] Fukushima K, Miyake S. Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. Competition and Cooperation in Neural Nets, 1982, Germany: Springer 267-285.

[22] García-Cumbreras M Á, Montejo-Ráez A, Díaz-Galiano M C. Pessimists and optimists: Improving collaborative filtering through sentiment analysis. Expert Systems with Applications, 2013, 40(17): 6758-6765.

[23] Girolami M, Kabán A. On an equivalence between PLSI and LDA. Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval - SIGIR’03, 2003, Canada: ACM Press 433-434.

[24] Glorot X, Bordes A, Bengio Y (2011). Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics: 315–323. JMLR Workshop and Conference Proceedings, USA.

[25] Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A C, Bengio Y (2014). Generative adversarial nets. In Ghahramani Z, Welling M, Cortes C, Lawrence N D, Weinberger K Q(eds.). NeurIPS 2014: 2672–2680, December 8–13, 2014, Montreal, Quebec, Canada.

[26] Green R, Sheppard J (2013). Comparing frequency and style-based features for Twitter author identification. In FLAIRS 2013 - Proceedings of the 26th International Florida Artificial Intelligence Research Society Conference, USA.

[27] Griffiths T L, Steyvers M. Finding scientific topics. Proceedings of the National Academy of Sciences, 2004, 101(Supplement1): 5228-5235.

[28] Han B, Cook P, Baldwin T. A stacking-based approach to Twitter user geolocation prediction. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2013, Sofia, Bulgaria: Association for Computational Linguistics 7-12.

[29] Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2015). Session-based recommendations with recurrent neural networks. arXiv Preprint arXiv:1511.06939.

[30] Kingma D P, Welling M (2014). Autoencoding variational bayes. In Bengio Y, LeCun Y(eds.): 2nd International Conference on Learning Representations, ICLR 2014, Conference Track Proceedings, Banff, AB, Canada, April 14–16, 2014.

[31] Koren Y (2008). Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining: 426–434.

[32] Koren Y, Bell R, Volinsky C. Matrix Factorization Techniques for Recommender Systems. Computer, 2009, 42(8): 30-37.

[33] Lampos V, Preot¸iuc-Pietro D, Cohn T. A user-centric model of voting intention from social media. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2013, Sofia, Bulgaria: Association for Computational Linguistics 993-1003.

[34] Le Q H, Mau T N, Tansuchat R, Huynh V N. A multi-criteria collaborative filtering approach using deep learning and dempster-shafer theory for hotel recommendations. IEEE Access, 2022, 10: 37281-37293.

[35] Li J, Ritter A, Hovy E. Weakly supervised user profile extraction from Twitter. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2014, Baltimore, Maryland: Association for Computational Linguistics 165-174.

[36] Liang S (2018). Dynamic user profiling for streams of short texts. In AAAI: 5860–5867.

[37] Liang S, Ren Z, Zhao Y, Ma J, Yilmaz E, Rijke M D. Inferring dynamic user interests in streams of short texts for user clustering. ACM Transactions on Information Systems, 2017, 36(1): 1-37.

[38] Liang S, Zhang X, Ren Z, Kanoulas E (2018). Dynamic em-beddings for user profiling in Twitter. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM.

[39] Mihalcea R, Tarau P. TextRank: Bringing order into text. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, 2004, Barcelona, Spain: Association for Computational Linguistics 404-411.

[40] Nassar N, Jafar A, Rahhal Y. A novel deep multi-criteria collaborative filtering model for recommendation system. Knowledge-Based Systems, 2020, 187: 104811.

[41] Nguyen V, Huynh V. Twoprobabilities focused combination in recommender systems. International Journal of Approximate Reasoning, 2017, 80: 225-238.

[42] Nguyen V, Huynh V, Sriboonchitta S. Integrating community context information into a reliably weighted collaborative filtering system using soft ratings. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020, 50(4): 1318-1330.

[43] Nguyen V, Sriboonchitta S, Huynh V. Using community preference for overcoming sparsity and cold-start problems in collaborative filtering system offering soft ratings. Electronic Commerce Research and Applications, 2017, 26: 101-108.

[44] Nilashi M, Ahani A, Esfahani M D, Yadegaridehkordi E, Samad S, Ibrahim O, Sharef N M, Akbari E. Preference learning for eco-friendly hotels recommendation: A multi-criteria collaborative filtering approach. Journal of Cleaner Production, 2019, 215: 767-783.

[45] Nilashi M, Ibrahim O B, Ithnin N, Zakaria R. A multi-criteria recommendation system using dimensionality reduction and Neuro-Fuzzy techniques. Soft Computing, 2015, 19(11): 3173-3207.

[46] Pennington J, Socher R, Manning C. Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, Doha, Qatar: Association for Computational Linguistics 1532-1543.

[47] Preot¸iuc-Pietro D, Lampos V, Aletras N. An analysis of the user occupational class through Twitter content. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2015, Beiing, China: Association for Computational Linguistics 1754-1764.

[48] Rahimi A, Cohn T, Baldwin T. Twitter user geolocation using a unified text and network prediction model. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2015, Beiing, China: Association for Computational Linguistics 630-636.

[49] Rajaraman A, Ullman J. Data mining. Mining of Massive Datasets, 2011, UK: Cambridge University Press 1-17.

[50] Rose S, Engel D, Cramer N, Cowley W. Automatic keyword extraction from individual documents. Text Mining, 2010, UK: John Wiley & Sons, Ltd 1-20.

[51] Rybak J, Balog K, Nørvåg, K. (2014). Temporal expertise profiling. In Advances in Information Retrieval: 36th European Conference on IR Research, ECIR 2014, Amsterdam, The Netherlands, April 13–16, 2014. Proceedings 36: 540–546. Springer.

[52] Sarwar B, Karypis G, Konstan J, Riedl J (2001). Item-based Collaborative Filtering Recommendation Algorithms. In Proceedings of the 10th International Conference on World Wide Web: 285–295.

[53] Shafer G. A Mathematical Theory of Evidence, 1976, USA: Princeton University Press. volume 42

[54] Shambour Q. A deep learning based algorithm for multi-criteria recommender systems. Knowledge-Based Systems, 2021, 211: 106545.

[55] Smets P. The combination of evidence in the transferable belief model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1990, 12(5): 447-458.

[56] Smets P. Decision making in the tbm: The necessity of the pignistic transformation. International Journal of Approximate Reasoning, 2005, 38(2): 133-147.

[57] Smets P, Kennes R. The transferable belief model. Classic Works of the Dempster-Shafer Theory of Belief Functions, 2008, Germany: Springer 693-736.

[58] Store R, Kangas J. Integrating spatial multi-criteria evaluation and expert knowledge for GIS-based habitat suitability modelling. Landscape and Urban Planning, 2001, 55(2): 79-93.

[59] Tallapally D, Sreepada R S, Patra B K, Babu K S (2018). User preference learning in multi-criteria recommendations using stacked auto encoders. In Proceedings of the 12th ACM Conference on Recommender Systems: 475–479.

[60] Velardi P, Faralli S, Navigli R. OntoLearn reloaded: A graph-based algorithm for taxonomy induction. Computational Linguistics, 2013, 39(3): 665-707.

[61] Vo D V, Karnjana J, Huynh V N. An integrated framework of learning and evidential reasoning for user profiling using short texts. Information Fusion, 2021, 70: 27-42.

[62] Vo D V, Tran T T, Shirai K, Huynh V N (2022). Deep generative networks coupled with evidential reasoning for dynamic user preferences using short texts. IEEE Transactions on Knowledge and Data Engineering (In press).

[63] Volkova S, Coppersmith G, Durme B V. Inferring user political preferences from streaming communications. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2014, Baltimore, Maryland: Association for Computational Linguistics 186-196.

[64] Wang F, Zhu H, Srivastava G, Li S, Khosravi M R, Qi L (2021). Robust collaborative filtering recommendation with useritem- trust records. IEEE Transactions on Computational Social Systems.

[65] Wong W, Liu W, Bennamoun M. Ontology learning from text. ACM Computing Surveys, 2012, 44(4): 1-36.

[66] Yang C, Yu X, Liu Y, Nie Y, Wang Y. Collaborative filtering with weighted opinion aspects. Neurocomputing, 2016, 210: 185-196.

[67] Yang T, Zhang J, Wang L, Zhang J. A novel customer-oriented recommendation system for paid knowledge products. Journal of Systems Science and Systems Engineering, 2022, 31(5): 515-533.

[68] Yin J, Wang J (2014). A dirichlet multinomial mixture model-based approach for short text clustering. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, USA.

[69] Zhang S, Yao L, Sun A, Tay Y. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys (CSUR), 2019, 52(1): 1-38.

[70] Zhao Z D, Shang M S. Userbased collaborative-filtering recommendation algorithms on hadoop. 2010 Third International Conference on Knowledge Discovery and Data Mining, 2010, USA: IEEE 478-481.