Topics and trends of the on-line public concerns based on Tianya forum

Abstract

Many social events spread fast through the Internet and arouse wide community discussions. Those on-line public opinions emerge into diverse topics along the time. Moreover, the strength of the topics is fluctuating. How to catch both primary topics and trend of topics over the shifting on-line discussions are not only of theoretical importance for scientific research, but also of practical importance for societal management especially in current China. To try the cutting-edge text analytic technologies to deal with unstructured on-line public opinions and provide support for social problem-solving in the big data era is worth an endeavour. This paper applies dynamic topic model (DTM) to explore the changing topics of new posts collected from Tianya Zatan Board of Tianya Club, the most influential Chinese BBS in mainland China. By analysis of the hot and cold terms trends, we catch the topics shift of main on-line concerns with illustrations of topics of school bus and environment in December of 2011. An algorithm is proposed to compute the strength fluctuation of each topic. With visualized analysis of the respective main topics in several months of 2012, some patterns of the topics fluctuation on the board are summarized.

Keywords

Topic models / dynamic topic model / on-line topics evolution / Tianya Club / societal management

References

[1] Allan J, Carbonell J G, Doddington G, et al. Topic detection and tracking pilot study final report. Proceedings of the Broadcast News Transcription and Understanding Workshop (Sponsored by DARPA), Feb, 1998, 1998

[2] Alsumait L, Barbara D, Domeniconi C. On-line LDA: Adaptive topic models for mining text streams with applications to topic detection and tracking. Proceedings of the 8th IEEE International Conference on Data Mining (ICDM’08), 3–12, Pisa, December 15–19, 2008, 2008

[3] Blei D M, Lafferty J D. Cohen W W, Moore A. Dynamic topic models. Proceedings of the 23rd International Conference on Machine Learning, 113–120, Pittsburgh, PA, June 25–29, 2006, 2006

[4] Blei D M, Lafferty J D. A correlated topic model of Science. Annals of Applied Statistics, 2007, 1: 17-35.

[5] Blei D M, Ng A Y, Jordan M I. Latent Dirichlet Allocation. Journal of Machine Learning Research, 2003, 3: 993-1022.

[6] Cao B, Shen D, Sun J, et al. Veloso M M, et al. Detect and track latent factors with online nonnegative matrix factorization. Proceedings of the 20th International Joint Conference on Artificial Intelligence, 2007 2689-2694.

[7] Chu K M. The Research on Topic Evolution for News based on LDA Model, 2010

[8] Cui L J, He H, Liu W. Research on hot issues and evolutionary trends in network forums. International Journal of u- and e- Service, Science and Technology, 2013, 6(2): 89-97.

[9] Guha R, Kumar R, Sivakumar D, Jose S. Grossman R, Bayardo R J, Bennett K P. Unweaving a web of documents. Proceeding of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, Illinois, August 2005, 2005

[10] Hall D, Jurafsky D, Manning C D. Studying the history of ideas using topic models. Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2008 363-371.

[11] Hao X, Hu Y. Topic detection and tracking oriented to BBS. Proceedings of 2010 International Conference on Computer, Mechatronics, Control and Electronic Engineering, 2010, 4: 154-157.

[12] He Q, Chen B, Pei J, et al. Detecting topic evolution in scientific literature: how can citations help?. Proceedings of the 18th ACM conference on Information and Knowledge Management, 2009 957-966.

[13] Jin L. Chinese outline BBS sphere: what BBS has brought to China, 2008

[14] Kleinberg J. Bursty and hierarchical structure in streams. Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2002 91-101.

[15] Lu M, Yao X, Wei S. BBS hot topic mining algorithm based on fuzzy clustering. Journal of Dalian Maritime University, 2008, 34(04): 52-58.

[16] Meng C, Zhang M, Guo W. E volution of Movie Topics Over Time, 2012

[17] Nallapati R M, Ditmore S, Lafferty J D, et al. Berkhin P, Caruana R, Wu X D, et al. Multiscale topic tomography. Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2007 520-529.

[18] Rittel H W J, Webber M M. Dilemmas in a general theory of planning. Policy Sciences, 1973, 4(2): 155-169.

[19] Shi D W, Zhang H. LDA Model-based BBS topic evolution. Industrial Control Computer, 2012, 25(05): 82-84.

[20] Song X, Lin C Y, Tseng B L, et al. Grossman R, Bayardo R J, Bennett K P, et al. Modeling and predicting personal information dissemination behavior. Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2005 479-488.

[21] Tang G, Chen H. Text clustering method based on BBS hot topics discovery. Computer Engineering, 2010, 7: 31

[22] Wang C, Blei D, Heckerman D. McAllester D A, Myllym P. Continuous time dynamic topic models. Proceedings of the 24th Conference in Uncertainty in Artificial Intelligence, 2008 579-586.

[23] Wang X, McCallum A, et al. Eliassi-Rad T, Ungar L H, Craven M, et al. Topics over time: a Non-Markov continuous-time model of topical trends. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2006 424-433.

[24] Wayne C L. Multilingual topic detection and tracking: successful research enabled by corpora and evaluation. Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC), Greece, May 31–June 2, 2000, European Language Resources Association, 2000

[25] Wu J J, Sun H Y, Tan Y. Social media research: a review. Journal of Systems Science and Systems Engineering., 2013, 22(3): 257-282.

[26] Yang Y, Pierce T, Carbonell J. A study of retrospective and on-line event detection. Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1998 28-36.

[27] You L, Du Y, Ge J, et al. Su K Y, Tsujii J, Lee J H, et al. BBS based hot topic retrieval using back-propagation neural network. Proceedings of the 1st International Joint Conference on Natural Language Processing (IJCNLP 2004), 2005 139-148.

[28] Zhang J, Li F. LDA topic evolution based on global and local modeling. Automation Technique, Computer Technology., 2012, 46(11): 1753-1758.

[29] Zhang Z D, Tang X J. A preliminary study of web mining for Tianya forum. Proceedings of the 11th Youth Conference of Systems Science and Management Science and 7th Conference of Logistic Systems Technology, 2011, Wuhan: Wuhan University of Science and Engineering Press 199-204.

[30] Zhao Y L, Tang X J. Wang S Y, Nakamori Y, Jin W L. A preliminary research of pattern of users’ behavior based on Tianya forum. Proceedings of the 14th International Symposium on Knowledge and Systems Sciences, 2013 171-179.

[31] Zheng R, Shi K, Li S. Zhou J. The influence factors and mechanism of societal risk. First International Conference on Complex Sciences, Complex 2009, Part II, 2009 2266-2275.