Please wait a minute...

Frontiers of Computer Science

Front Comput Sci    2012, Vol. 6 Issue (1) : 76-87     DOI: 10.1007/s11704-011-1174-8
RESEARCH ARTICLE |
Mining the interests of Chinese microbloggers via keyword extraction
Zhiyuan LIU(), Xinxiong CHEN, Maosong SUN
Department of Computer Science and Technology, State Key Lab on Intelligent Technology and Systems, National Lab for Information Science and Technology, Tsinghua University, Beijing 100084, China
Download: PDF(547 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Microblogging provides a new platform for communicating and sharing information amongWeb users. Users can express opinions and record daily life using microblogs. Microblogs that are posted by users indicate their interests to some extent. We aim to mine user interests via keyword extraction from microblogs. Traditional keyword extraction methods are usually designed for formal documents such as news articles or scientific papers. Messages posted by microblogging users, however, are usually noisy and full of new words, which is a challenge for keyword extraction. In this paper, we combine a translation-based method with a frequency-based method for keyword extraction. In our experiments, we extract keywords for microblog users from the largest microblogging website in China, Sina Weibo. The results show that our method can identify users’ interests accurately and efficiently.

Keywords microblogging      Sina Weibo      Chinese keyword extraction      user interests     
Corresponding Authors: LIU Zhiyuan,Email:lzy.thu@gmail.com   
Issue Date: 01 February 2012
 Cite this article:   
Zhiyuan LIU,Xinxiong CHEN,Maosong SUN. Mining the interests of Chinese microbloggers via keyword extraction[J]. Front Comput Sci, 2012, 6(1): 76-87.
 URL:  
http://journal.hep.com.cn/fcs/EN/10.1007/s11704-011-1174-8
http://journal.hep.com.cn/fcs/EN/Y2012/V6/I1/76
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Zhiyuan LIU
Xinxiong CHEN
Maosong SUN
1 Kwak H, Lee C, Park H, Moon S. What is Twitter, a social network or a news media? In: Proceedings of the 19th International Conference on World Wide Web . 2010, 591-600
doi: 10.1145/1772690.1772751
2 Liu Z, Chen X, Zheng Y, Sun M. Automatic keyphrase extraction by bridging vocabulary gap. In: Proceedings of the 15th Conference on Computational Natural Language Learning . 2011, 135-144
3 Brown P F, Pietra S A D, Pietra V J D, Mercer R L. The mathematics of statistical machine translation: parameter estimation. Computational linguistics , 1993, 19(2): 263-311
4 Koehn P. Statistical Machine Translation. Cambridge: Cambridge University Press, 2010
5 Berger A L, Lafferty J D. Information retrieval as statistical translation. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 1999, 222-229
doi: 10.1145/312624.312681
6 Karimzadehgan M, Zhai C X. Estimation of statistical translation models based on mutual information for ad hoc information retrieval. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval . 2010, 323-330
7 Duygulu P, Barnard K, de Freitas J F G, Forsyth D A. Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In: Proceedings of the 7th European Conference on Computer Vision, Part IV . 2002, 97-112
8 Berger A L, Caruana R, Cohn D, Freitag D, Mittal V O. Bridging the lexical chasm: statistical approaches to answer-finding. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 2000, 192-199
doi: 10.1145/345508.345576
9 Echihabi A, Marcu D. A noisy-channel approach to question answering. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics . 2003, 16-23
10 Murdock V, Croft W B. Simple translation models for sentence retrieval in factoid question answering. In: Proceedings of SIGIR 2004 Workshop on Information Retrieval for Question Answering . 2004
11 Soricut R, Brill E. Automatic question answering using the web: beyond the factoid. Information Retrieval , 2006, 9(2): 191-206
doi: 10.1007/s10791-006-7149-y
12 Xue X, Jeon J, Croft W B. Retrieval models for question and answer archives. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 2008, 475-482
doi: 10.1145/1390334.1390416
13 Riezler S, Vasserman A, Tsochantaridis I, Mittal V, Liu Y. Statistical machine translation for query expansion in answer retrieval. In: Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics . 2007, 464-471
14 Riezler S, Liu Y, Vasserman A. Translating queries into snippets for improved query expansion. In: Proceedings of the 22nd International Conference on Computational Linguistics . 2008, 737-744
15 Riezler S, Liu Y. Query rewriting using monolingual statistical machine translation. Computational Linguistics , 2010, 36(3): 569-582
doi: 10.1162/coli_a_00010
16 Banko M, Mittal V O, Witbrock M J. Headline generation based on statistical translation. In: Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics . 2000, 318-325
17 Liu Z, Wang H, Wu H, Li S. Collocation extraction using monolingual word alignment method. In: Proceedings of 2009 Conference on Empirical Methods in Natural Language Processing . 2009, 487-495
18 Liu Z, Wang H, Wu H, Li S. Improving statistical machine translation with monolingual collocation. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics . 2010, 825-833
19 Quirk C, Brockett C, Dolan W B. Monolingual machine translation for paraphrase generation. In: Proceedings of 2004 Conference on Empirical Methods in Natural Language Processing . 2004, 142-149
20 Zhao S, Wang H, Liu T. Paraphrasing with search engine query logs. In: Proceedings of the 23rd International Conference on Computational Linguistics . 2010, 1317-1325
21 Frank E, Paynter G W, Witten I H, Gutwin C, Nevill-Manning C G. Domain-specific keyphrase extraction. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence . 1999, 668-673
22 Witten I H, Paynter G W, Frank E, Gutwin C, Nevill-Manning C G. Kea: practical automatic keyphrase extraction. In: Proceedings of 4th ACM conference on Digital Libraries . 1999, 254-255
doi: 10.1145/313238.313437
23 Turney P D. Learning algorithms for keyphrase extraction. Information Retrieval , 2000, 2(4): 303-336
doi: 10.1023/A:1009976227802
24 Salton G, Buckley C. Term-weighting approaches in automatic text retrieval. Information Processing and Management , 1988, 24(5): 513-523
doi: 10.1016/0306-4573(88)90021-0
25 Mihalcea R, Tarau P. Textrank: bringing order into texts. In: Proceedings of 2004 Conference on Empirical Methods in Natural Language Processing . 2004, 404-411
26 Page L, Brin S, Motwani R, Winograd T. The pagerank citation ranking: bringing order to the web. Technical Report, Stanford Digital Library Technologies Project , 1998
27 Landauer T K, Foltz PW, Laham D. An introduction to latent semantic analysis. Discourse Processes , 1998, 25(2&3): 259-284
doi: 10.1080/01638539809545028
28 Hofmann T. Probabilistic latent semantic indexing. In: Proceedings of 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval . 1999, 50-57
doi: 10.1145/312624.312649
29 Blei D M, Ng A Y, Jordan M I. Latent Dirichlet allocation. Journal of Machine Learning Research , 2003, 3: 993-1022
30 Heinrich G. Parameter estimation for text analysis. http://www.arbylon.net/publications/text-est
31 Blei D M, Lafferty J D. Topic Models. In: Srivastava A, Sahami M, eds. Text Mining: Classification, Clustering, and Applications . London: Chapman & Hall, 2009
32 Zhao D, Rosson MB. How and why people twitter: the role that microblogging plays in informal communication at work. In: Proceedings of ACM 2009 International Conference on Supporting Group Work . 2009, 243-252
doi: 10.1145/1531674.1531710
33 Savage N. Twitter as medium and message. Communications of the ACM , 2011, 54(3): 18-20
doi: 10.1145/1897852.1897860
34 Zhao W X, Jiang J, Weng J, He J, Lim E, Yan H, Li X. Comparing twitter and traditional media using topic models. In: Proceedings of the 33rd European Conference on IR Research . 2011, 338-349
35 Java A, Song X, Finin T, Tseng B. Why we Twitter: understanding microblogging usage and communities. In: Proceedings of 9th WebKDD and 1st SNA-KDD 2007 Workshop on Web Mining and Social Network Analysis . 2007, 56-65
36 Teevan J, Ramage D, Morris MR. #Twittersearch: a comparison of microblog search and web search. In: Proceedings of the 4th International Conference on Web Search and Web Data Mining . 2011, 35-44
37 Mustafaraj E, Metaxas P. From obscurity to prominence in minutes: political speech and real-time search. In: Proceedings of Web Science Conference . 2010.
38 Phelan O, McCarthy K, Smyth B. Using twitter to recommend realtime topical news. In: Proceedings of the 3rd ACM conference on Recommender systems . 2009, 385-388
39 Sakaki T, Okazaki M, Matsuo Y. Earthquake shakes twitter users: realtime event detection by social sensors. In: Proceedings of the 19th International Conference on World Wide Web . 2010, 851-860
doi: 10.1145/1772690.1772777
40 Culotta A. Detecting influenza outbreaks by analyzing twitter messages. In: Proceedings of KDD Workshop on Social Media Analytics . 2010
doi: 10.1145/1964858.1964874
41 Earle P S, Guy M, Ostrum C, Horvath S, Buckmaster R A. Omg earthquake! Can twitter improve earthquake response? In: Proceedings of 2009 AGU Fall Meeting Abstracts , Vol 1. 2009
42 Petrovic S, Osborne M, Lavrenko V. Streaming first story detection with application to Twitter. In: Proceedings of 2010 Human Language Technologies: Conference of the North American Chapter of the Association for Computational Linguistics . 2010, 181-189
43 Cha M, Haddadi H, Benevenuto F, Gummadi K P. Measuring user influence in Twitter: the million follower fallacy. In: Proceedings of the 4th International AAAI Conference on Weblogs and Social . 2010, 10-17
44 Tumasjan A, Sprenger T O, Sandner P G, Welpe I M. Predicting elections with Twitter: what 140 characters reveal about political sentiment. In: Proceedings of the 4th International AAAI Conference on Weblogs and Social Media . 2010, 178-185
45 OConnor B, Balasubramanyan R, Routledge B R, Smith N A. From tweets to polls: linking text sentiment to public opinion time series. In: Proceedings of the 4th International AAAI Conference onWeblogs and Social Media . 2010, 122-129
46 Pak A, Paroubek P. Twitter as a corpus for sentiment analysis and opinion mining. In: Proceedings of International Conference on Language Resources and Evaluation . 2010
47 Jiang L, Yu M, Zhou M, Liu X, Zhao T. Target-dependent twitter sentiment classification. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , Vol 1. 2011, 151-160
48 Agarwal A, Xie B, Vovsha I, Rambowand O, Passonneau R. Sentiment analysis of twitter data. In: Proceedings of Workshop on Language in Social Media . 2011, 30-38
49 Qu Z, Liu Y. Interactive group suggesting for twitter. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , Vol 2. 2011, 519-523
50 Huang J, Thornton K M, Efthimiadis E N. Conversational tagging in Twitter. In: Proceedings of the 21st ACM Conference on Hypertext and Hypermedia . 2010, 173-178
doi: 10.1145/1810617.1810647
51 Efron M. Hashtag retrieval in a microblogging environment. In: Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval . 2010, 787-788
52 Gimpel K, Schneider N, Oonnor B, Das D, Mills D, Eisenstein J, Heilman M, Yogatama D, Flanigan J, Smith N A. Part-of-speech tagging for twitter: annotation, features, and experiments. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , Vol 2. 2011, 42-47
53 Finin T, Murnane W, Karandikar A, Keller N, Martineau J, Dredze M. Annotating named entities in twitter data with crowdsourcing. In: Proceedings of NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk . 2010, 80-88
54 Liu X, Zhang S, Wei F, Zhou M. Recognizing named entities in tweets. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Human Language Technologies , Vol 1. 2011, 359-367
55 Ritter A, Clark S, Mausam 0000, Etzioni O. Named entity recognition in tweets: an experimental study. In: Proceedings of 2011 Conference on Empirical Methods in Natural Language Processing . 2011, 1524-1534
56 Han B, Baldwin T. Lexical normalisation of short text messages: makn sens a #twitter. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , Vol 1. 2011, 368-378
57 Wu W, Zhang B, Ostendorf M. Automatic generation of personalized annotation tags for twitter users. In: Proceedings of Human Language Technologies: Conference of the North American Chapter of the Association of . 2010, 689-692
58 Zhang K, Sun M. A stacked model based on word lattice for Chinese word segmentation and part-of-speech tagging. http://nlp.csai.tsinghua.edu.cn/thulac
59 Jiang W, Mi H, Liu Q. Word lattice reranking for Chinese word segmentation and part-of-speech tagging. In: Proceedings of the 22nd International Conference on Computational Linguistics . 2008, 385-392
60 Viegas F B, Wattenberg M, Feinberg J. Participatory visualization with Wordle. IEEE Transactions on Visualization and Computer Graphics , 2009, 15(6): 1137-1144
doi: 10.1109/TVCG.2009.171
61 Och F J, Ney H. A systematic comparison of various statistical alignment models. Computational linguistics , 2003, 29(1): 19-51
doi: 10.1162/089120103321337421
62 Wan X, Xiao J. Single document keyphrase extraction using neighborhood knowledge. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence . 2008, 855-860
63 Liu Y, Liu Q, Lin S. Discriminative word alignment by linear modeling. Computational Linguistics , 2010, 36(3): 303-339
doi: 10.1162/coli_a_00001
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed