How graph convolutions amplify popularity bias for recommendation?

Abstract

Graph convolutional networks (GCNs) have become prevalent in recommender system (RS) due to their superiority in modeling collaborative patterns. Although improving the overall accuracy, GCNs unfortunately amplify popularity bias — tail items are less likely to be recommended. This effect prevents the GCN-based RS from making precise and fair recommendations, decreasing the effectiveness of recommender systems in the long run.

In this paper, we investigate how graph convolutions amplify the popularity bias in RS. Through theoretical analyses, we identify two fundamental factors: (1) with graph convolution (i.e., neighborhood aggregation), popular items exert larger influence than tail items on neighbor users, making the users move towards popular items in the representation space; (2) after multiple times of graph convolution, popular items would affect more high-order neighbors and become more influential. The two points make popular items get closer to almost users and thus being recommended more frequently. To rectify this, we propose to estimate the amplified effect of popular nodes on each node’s representation, and intervene the effect after each graph convolution. Specifically, we adopt clustering to discover highly-influential nodes and estimate the amplification effect of each node, then remove the effect from the node embeddings at each graph convolution layer. Our method is simple and generic — it can be used in the inference stage to correct existing models rather than training a new model from scratch, and can be applied to various GCN models. We demonstrate our method on two representative GCN backbones LightGCN and UltraGCN, verifying its ability in improving the recommendations of tail items without sacrificing the performance of popular items. Codes are open-sourced .

Graphical abstract

Keywords

recommendation / graph convolution networks / popularity bias

References

[1] Wu L, He X, Wang X, Zhang K, Wang M . A survey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation. IEEE Transactions on Knowledge and Data Engineering, 2023, 35( 5): 4425–4445

[2] Chen J, Dong H, Wang X, Feng F, Wang M, He X . Bias and debias in recommender system: a survey and future directions. ACM Transactions on Information Systems, 2023, 41( 3): 67

[3] Wang X, He X, Wang M, Feng F, Chua T S. Neural graph collaborative filtering. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 165–174

[4] Wu L, Sun P, Fu Y, Hong R, Wang X, Wang M. A neural influence diffusion model for social recommendation. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2019, 235−244

[5] Wu F, Souza Jr A H, Zhang T, Fifty C, Yu T, Weinberger K Q. Simplifying graph convolutional networks. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 6861–6871

[6] He X, Deng K, Wang X, Li Y, Zhang Y, Wang M. LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 639–648

[7] Liu F, Cheng Z, Zhu L, Gao Z, Nie L. Interest-aware message-passing GCN for recommendation. In: Proceedings of the Web Conference 2021. 2021, 1296−1305

[8] Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009, 452−461

[9] Wei T, Feng F, Chen J, Wu Z, Yi J, He X. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1791−1800

[10] Zheng Y, Gao C, Li X, He X, Li Y, Jin D. Disentangling user interest and conformity for recommendation with causal embedding. In: Proceedings of the Web Conference 2021. 2021, 2980−2991

[11] Zhang Y, Feng F, He X, Wei T, Song C, Ling G, Zhang Y. Causal intervention for leveraging popularity bias in recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 11−20

[12] Liang D, Charlin L, Blei D M. Causal inference for recommendation. In: Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence Workshop on Causation: Foundation to Application. 2016

[13] Abdollahpouri H, Burke R, Mobasher B. Managing popularity bias in recommender systems with personalized re-ranking. In: Proceedings of the 32nd International Florida Artificial Intelligence Research Society Conference. 2019, 413−418

[14] Mao K, Zhu J, Xiao X, Lu B, Wang Z, He X. UltraGCN: ultra simplification of graph convolutional networks for recommendation. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 1253−1262

[15] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations. 2017

[16] Johnson C C. Logistic matrix factorization for implicit feedback data. In: Proceedings of Advances in Neural Information Processing Systems. 2014

[17] Koh P W, Liang P. Understanding black-box predictions via influence functions. In: Proceedings of the 34th International Conference on Machine Learning. 2017, 1885−1894

[18] Xu K, Li C, Tian Y, Sonobe T, Kawarabayashi K I, Jegelka S. Representation learning on graphs with jumping knowledge networks. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 5453−5462

[19] Tang X, Yao H, Sun Y, Wang Y, Tang J, Aggarwal C, Mitra P, Wang S. Investigating and mitigating degree-related biases in graph convoltuional networks. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020, 1435–1444

[20] Wu J, Wang X, Gao X, Chen J, Fu H, Qiu T, He X. On the effectiveness of sampled softmax loss for item recommendation. 2022, arXiv preprint arXiv: 2201.02327

[21] Zhao Z, Chen J, Zhou S, He X, Cao X, Zhang F, Wu W. Popularity bias is not always evil: disentangling benign and harmful bias for recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022,

[22] Zhao M, Deng Q, Wang K, Wu R, Tao J, Fan C, Chen L, Cui P . Bilateral filtering graph convolutional network for multi-relational social recommendation in the power-law networks. ACM Transactions on Information Systems, 2021, 40( 2): 31

[23] Gruson A, Chandar P, Charbuillet C, McInerney J, Hansen S, Tardieu D, Carterette B. Offline evaluation to make decisions about playlistrecommendation algorithms. In: Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 2019, 420−428

[24] Schnabel T, Swaminathan A, Singh A, Chandak N, Joachims T. Recommendations as treatments: debiasing learning and evaluation. In: Proceedings of the 33rd International Conference on Machine Learning. 2016, 1670−1679

[25] Bonner S, Vasile F. Causal embeddings for recommendation. In: Proceedings of the 12th ACM Conference on Recommender Systems. 2018, 104−112

[26] Liu Z, Nguyen T K, Fang Y. Tail-GNN: tail-node graph neural networks. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1109−1119

[27] Kingma D P, Ba J. Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. 2015

[28] Zhu Z, He Y, Zhao X, Caverlee J. Popularity bias in dynamic recommendation. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 2439−2449

[29] Xv G, Lin C, Li H, Su J, Ye W, Chen Y. Neutralizing popularity bias in recommendation models. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2022, 2623−2628

[30] Gupta P, Sharma A, Malhotra P, Vig L, Shroff G. CauSeR: causal session-based recommendations for handling popularity bias. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021, 3048−3052

[31] Gupta S, Wang H, Lipton Z, Wang Y. Correcting exposure bias for link recommendation. In: Proceedings of the 38th International Conference on Machine Learning. 2021, 3953−3963

[32] Guo H, Yu J, Liu Q, Tang R, Zhang Y. PAL: a position-bias aware learning framework for CTR prediction in live recommender systems. In: Proceedings of the 13th ACM Conference on Recommender Systems. 2019, 452−456

[33] Dudík M, Langford J, Li L. Doubly robust policy evaluation and learning. In: Proceedings of the 28th International Conference on Machine Learning. 2011, 1097−1104

[34] Krishnan A, Sharma A, Sankar A, Sundaram H. An adversarial approach to improve long-tail performance in neural collaborative filtering. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2018, 1491−1494

[35] Chen J, Dong H, Qiu Y, He X, Xin X, Chen L, Lin G, Yang K. AutoDebias: learning to debias for recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2021, 21−30

[36] Zhang Y, Cheng D Z, Yao T, Yi X, Hong L, Chi E H. A model of two tales: dual transfer learning framework for improved long-tail item recommendation. In: Proceedings of the Web Conference 2021. 2021, 2220−2231

[37] Abdollahpouri H, Burke R, Mobasher B. Controlling popularity bias in learning-to-rank recommendation. In: Proceedings of the 11th ACM Conference on Recommender Systems. 2017, 42−46

[38] Wang Y, Liang D, Charlin L, Blei D M. The deconfounded recommender: a causal inference approach to recommendation. 2019, arXiv preprint arXiv: 1808.06581

[39] Sato M, Takemori S, Singh J, Ohkuma T. Unbiased learning for the causal effect of recommendation. In: Proceedings of the 14th ACM Conference on Recommender Systems. 2020, 378−387

[40] Qiu R, Wang S, Chen Z, Yin H, Huang Z. CausalRec: causal inference for visual debiasing in visually-aware recommendation. In: Proceedings of the 29th ACM International Conference on Multimedia. 2021, 3844–3852

[41] Wang W, Feng F, He X, Wang X, Chua T S. Deconfounded recommendation for alleviating bias amplification. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021, 1717−1725

[42] Dai H, Li H, Tian T, Huang X, Wang L, Zhu J, Song L. Adversarial attack on graph structured data. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1115−1124

[43] Xu K, Hu W, Leskovec J, Jegelka S. How powerful are graph neural networks? In: Proceedings of the 7th International Conference on Learning Representations. 2018

[44] Li Q, Han Z, Wu X M. Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 3538−3545

[45] Chen D, Lin Y, Li W, Li P, Zhou J, Sun X. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 3438−3445

[46] Chen M, Wei Z, Huang Z, Ding B, Li Y. Simple and deep graph convolutional networks. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 1725−1735

[47] Liu M, Gao H, Ji S. Towards deeper graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020, 338−348

[48] Sun K, Lin Z, Zhu Z. Multi-stage self-supervised learning for graph convolutional networks on graphs with few labeled nodes. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. 2020, 5892−5899

RIGHTS & PERMISSIONS

Higher Education Press