User behavior modeling for better Web search ranking

Abstract

Modern search engines record user interactions and use them to improve search quality. In particular, user click-through has been successfully used to improve clickthrough rate (CTR), Web search ranking, and query recommendations and suggestions. Although click-through logs can provide implicit feedback of users’ click preferences, deriving accurate absolute relevance judgments is difficult because of the existence of click noises and behavior biases. Previous studies showed that user clicking behaviors are biased toward many aspects such as “position” (user’s attention decreases from top to bottom) and “trust” (Web site reputations will affect user’s judgment). To address these problems, researchers have proposed several behavior models (usually referred to as click models) to describe users? practical browsing behaviors and to obtain an unbiased estimation of result relevance. In this study, we review recent efforts to construct click models for better search ranking and propose a novel convolutional neural network architecture for building click models. Compared to traditional click models, our model not only considers user behavior assumptions as input signals but also uses the content and context information of search engine result pages. In addition, our model uses parameters from traditional click models to restrict the meaning of some outputs in our model’s hidden layer. Experimental results show that the proposed model can achieve considerable improvement over state-of-the-art click models based on the evaluation metric of click perplexity.

Keywords

user behavior / click model / Web search

References

[1] Joachims T, Granka L, Pan B, Hembrooke H, Gay G. Accurately interpreting clickthrough data as implicit feedback. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2005, 154–161

[2] Craswell N, Zoeter O, Taylor M, Ramsey B. An experimental comparison of click position-bias models. In: Proceedings of ACM International Conference on Web Search and Data Mining. 2008, 87–94

[3] Yue Y S, Patel R, Roehrig H. Beyond position bias: examining result attractiveness as a source of presentation bias in clickthrough data. In: Proceedings of the 19th ACM International Conference onWorldWide Web. 2010, 1011–1018

[4] Wang C, Liu Y Q, Zhang M, Ma S P, Zheng M H, Qian J, Zhang K. Incorporating vertical results into search click models. In: Proceedings of the 36th ACM International ACM SIGIR Conference on Research and Development in Information Retrieval. 2013, 503–512

[5] Guo F, Liu C, Wang Y M. Efficient multiple-click models in web search. In: Proceedings of the 2nd ACM International Conference on Web Search and Data Mining. 2009, 124–131

[6] Dupret G E, Piwowarski B. A user browsing model to predict search engine click data from past observations. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2008, 331–338

[7] Chapelle O, Zhang Y. A dynamic Bayesian network click model for Web search ranking. In: Proceedings of the 18th ACM International Conference on World Wide Web. 2009, 1–10

[8] Liu Z Y, Liu Y Q, Zhou K, Zhang M, Ma S P. Influence of vertical result in Web search examination. In: Proceedings of the 38th International ACMSIGIR Conference on Research and Development in Information Retrieval. 2015, 193–202

[9] Wang C, Liu Y Q, Wang M, Zhou K, Nie J Y, Ma S P. Incorporating non-sequential behavior into click models. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2013, 283–292

[10] Kaisser M, Hearst M, Lowe J. Improving search results quality by customizing summary lengths. In : Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technlogies (ACLHLT’08). 2008

[11] Kanungo T, Orr D. Predicting the readability of short Web summaries. In: Proceedings of the International Conference on Web Search and Web Data Mining. 2009, 325–326

[12] Carbonell J, Goldstein J. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 1998, 335–336

[13] Clarke C L A, Kolla M, Cormack G V, Vechtomova O, Ashkan A, Büttcher S, MacKinnon I. Novelty and diversity in information retrieval evaluation. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2008, 659–666

[14] Wang H N, Zhai C X, Dong A L, Chang Y. Content-aware click modeling. In: Proceedings of the 23rd International World-Wide Web Conference. 2013, 175–176

[15] Schmidhuber J. Deep learning in neural networks: an overview. Neural Networks, 2015, 61: 85–117

[16] Yu L, Hermann K N, Blunsom P, Pulman S. Deep learning for answer sentence selection. In: Proceedings of NIPS Deep Learning and Representation Learning Workshop. 2014, 393–402

[17] Kim Y. Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2014, 1746–1751

[18] Huang P S, He X D, Gao J F, Deng L, Acero A, Heck L. Learning deep structured semantic models for Web search using clickthrough data. In: Proceedings of the 22nd ACM International Conference on Information and Knowledge Management. 2013, 2333–2338

[19] Severyn A, Moschitti A. Learning to rank short text pairs with convolutional deep neural networks. In: Proceedings of the 38th International ACMSIGIR Conference on Research and Development in Information Retrieval. 2015, 373–382

[20] Liu Q, Yu F, Wu S, Wang L. A convolutional click prediction model. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 2015, 1743–1746

[21] Guo F, Liu C, Kannan A, Minka T, Taylor M J, Wang Y M, Faloutsos C. Click chain model in Web search. In: Proceedings of International Conference on World Wide Web. 2009, 11–20

[22] Buscher G, Van Elst L, Dengel A. Segment-level display time as implicit feedback: a comparison to eye tracking. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2009, 67–74

[23] Smucker M D, Clarke C L A. Time-based calibration of effectiveness measures. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2012, 95–104

[24] Fox S, Karnawat K, Mydland M, Dumais S, White T. Evaluating implicit measures to improve Web search. ACM Transactions on Information Systems, 2005, 23(2): 147–168

[25] White R W, Kelly D. A study on the effects of personalization and task information on implicit feedback performance. In: Proceedings of the 15th ACM International Conference on Information and Knowledge Management. 2006, 297–306

[26] Agichtein E, Brill E, Dumais S. Improving Web search ranking by incorporating user behavior information. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 2006, 19–26

[27] Xu W H, Manavoglu E, Cantu-Paz E. Temporal click model for sponsored search. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2010, 106–113

[28] Wang K S, Gloy N, Li X L. Inferring search behaviors using partially observable Markov (POM) model. In: Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. 2010, 211–220

[29] Xu D Q, Liu Y Q, Zhang M, Ma S P, Ru L Y. Incorporating revisiting behaviors into click models. In: Proceedings of the 5th ACM International Conference on Web Search and Data Mining. 2012, 303–312

[30] Liu Y Q, Xie X H, Wang C, Nie J Y, Zhang M, Ma S P. Time-aware click model. ACM Transactions on Information Systems, 2016, 35(3): 24–34

[31] Salakhutdinov R, Hinton G. Semantic hashing. International Journal of Approximate Reasoning, 2009, 50(7): 969–978

[32] Socher R, Huval B, Manning C D, Ng A Y. Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2012, 1201–1211

[33] Tur G, Deng L, Hakkani-Tür D, He X D. Towards deeper understanding: deep convex networks for semantic utterance classification. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2012, 5045–5048

[34] Shen Y L, He X D, Gao J F, Deng L, Mesnil G. Learning semantic representations using convolutional neural networks for Web search. In: Proceedings of the 23rd International Conference onWorld WideWeb. 2014, 373–374

[35] Zhang Y Y, Dai H J, Xu C, Feng J, Wang T F, Bian J, Wang B, Liu T Y. Sequential click prediction for sponsored search with recurrent neural networks. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 133–134

[36] Borisov A, Markov I, de Rijke M, Serdyukov P. A neural click model for Web search. In: Proceedings of the 25th International Conference on World Wide Web. 2016, 531–541

[37] Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014

[38] Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J. Distributed representations of words and phrases and their compositionality. In: Proceedings of the Neural Information Processing Systems Conference. 2013, 3111–3119

[39] Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference onMachine Learning. 2010, 807–814

[40] Wan L, Zeiler M, Zhang S X, Cun Y L, Fergus R. Regularization of neural networks using dropconnect. In: Proceedings of the 30th International Conference on Machine Learning. 2013, 1058–1066

[41] Bordes A, Weston J, Usunier N. Open question answering with weakly supervised embedding models. In: Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 2014, 165–180

[42] Echihabi A, Marcu D. A noisychannel approach to question answering. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics. 2003, 16–23

[43] Chen D Q, Chen W Z, Wang J X, Chen Z, Yang Q. Beyond ten blue links: enabling user click modeling in federated web search. In: Proceedings of the 5th ACM International Conference on Web Search and Data Mining. 2012, 463–472

[44] Järvelin K, Kekäläinen J. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 2002, 20(4): 422–446

[45] Yang H, Mityagin A, Svore K M, Markov S. Collecting high quality overlapping labels at low cost. In Proceedings of the 33rd International ACMSIGIR Conference on Research and Development in Information Retrieval. 2010, 459–466

[46] Landis J R, Koch G G. The measurement of observer agreement for categorical data. Biometrics, 1977, 33(1): 159–174

RIGHTS & PERMISSIONS

2017 Higher Education Press and Springer-Verlag Berlin Heidelberg