An Empirical Feasibility Study of Societal Risk Classification Toward BBS Posts

Abstract

Societal risk classification is the fundamental issue for online societal risk monitoring. To show the challenge and feasibility of societal risk classification toward BBS posts, an empirical analysis is implemented in this paper. Through effectiveness analysis, Support Vector Machine based on Bag-Of-Words (BOW-SVM) is adopted for challenge validation, and the distributed document embeddings of BBS posts generated by Paragraph Vector are applied to feasibility study. Based on BOW-SVM, cross-validations of BBS posts labeled by different groups and annotators are conducted. The big fluctuation of cross-validation results indicates the differences of individual risk perceptions, which brings more challenges to societal risk classification. Furthermore, based on the distributed document embeddings of BBS posts, the pairwise similarities of more than 300 thousands BBS posts from different societal risk categories are compared. The higher similarities of BBS posts in the same societal risk category reveal that BBS posts in the same societal risk category share more features than BBS posts in different categories, which manifests the feasibility of societal risk classification of BBS posts, and also reflects the possibility to improve the performance of societal risk monitoring.

Keywords

Societal risk classification / Tianya Forum / cross validation / pairwise similarity / individual risk perception

References

[1] Bengio Y., Ducharme R., Vincent P., Jauvin C.. A Neural Probabilistic Language Model. Journal of Machine Learning Research, 2003, 3: 1137-1155.

[2] Cao L.N., Tang X.J.. Topics and Threads of the Online Public Concerns Based on Tianya Forum. Journal of Systems Science and Systems Engineering, 2014, 23(2): 212-230.

[3] Chen J.D., Tang X.J.. Exploring Societal Risk Classification of the Posts of Tianya Club. International Journal of Knowledge and Systems Science, 2014, 5(1): 36-48.

[4] Chen J.D., Tang X.J.. Wang S. Y., Nakamori Y., Huynh V. N.. Societal Risk Classification of Post Based on Paragraph Vector and KNN Method. the 15th International Symposium on Knowledge and Systems Sciences, 2014 117-123.

[5] Collobert R., Weston J., Bottou L., Karlen M. K. K., Kuksa P.. Natural Language Processing (Almost) from Scratch. Journal of Machine Learning Research, 2011, 12: 2461-2505.

[6] Cover T.M., Hart P.E.. Nearest Neighbor Pattern Classification. IEEE Transactions on Information Theory, 1967, 13(1): 21-27.

[7] Griffis E.S., Thomas J.G., Martha C.. Web-Based Surveys: A Comparison of Response, Data, and Cost. Journal of Business Logistics, 2003, 24(2): 237-258.

[8] Hao B.B., Li L., Gao R., Li A., Zhu T.S.. Sensing Subjective Well-Being from Social Media. International Conference on Active Media Technology, 2014, 8610: 324-335.

[9] Jeffrey P., Richard S., Christopher M.. Moschitti A., Pang B., Daelemans W.. Glove: Global Vectors for Word Representation. Proceedings of the 2014 Empirical Methods in Natural Language Processing, 2014 1532-1543.

[10] Le Q., Mikolov T.. Distributed Representations of Sentences and Documents. Proceedings of the 31st International Conference on Machine Learning, 2014 1188-1196.

[11] Liu B.. Sentiment Analysis and Opinion Mining (Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2012

[12] Lu Y.F., Hu X., Wang F., Kumar S., Liu H., Maciejewski R.. Gangemi A., Leonardi S., Panconesi A.. Visualizing Social Media Sentiment in Disaster Scenarios. Proceedings of the 24th International Conference on World Wide Web Companion, 2015 1211-1215.

[13] Mikolov T., Chen K., Corrado G., Dean J.. Efficient Estimation of Word Representations in Vector Space. nPaper presented at ICLR 2013: International Conference on Learning Representations, Scottsdale, May 2–4, 2013, 2013

[14] Qiu L., Cao Y., Nie Z.Q., Rui Y.. Brodley C.E., Stone P.. Learning Word Representation Considering Proximity and Ambiguity. Proceedings of the 28th AAAI Conference on Artificial Intelligence, 572–1578, Québec, July 27 – 31, 2014, AAAI Press, 2014

[15] Tai K.S., Socher R., Manning C.D.. Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks. Paper presented at the 53rd Annual Meeting of the Association for Computational Linguistics, Beijing, July 26–31, 2015, 2015

[16] Tang D.Y., Qin B., Liu T.. Màrquez L., Chris C.B., Su J., Pighin D., Marton Y.. Document Modeling with Gated Recurrent Neural Network for Sentiment Classification. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1422–1432, Lisbon, September 17–21, 2015

[17] Tang X.J.. Qualitative metasynthesis techniques for analysis of public opinions for in-depth study. the 1st International Conference on Complex Sciences: Theory and Applications II, LNICST, 2009, 5: 2338-2353.

[18] Tang X.J.. Exploring On-line Societal Risk Perception for Harmonious Society Measurement. Journal of Systems Science and Systems Engineering, 2013, 22(4): 469-486.

[19] Wen S.Y., Wan X.J.. Brodley C.E., Stone P.. Emotion Classification in Microblog Texts Using Class Sequential Rules. Proceedings of the 28th AAAI Conference on Artificial Intelligence, 187–193, Québec, July 27 – 31, 2014, AAAI Press, 2014

[20] Wu J.J., Sun H.Y., Tan Y.. Social media research: a review. Journal of Systems Science and Systems Engineering., 2013, 22(3): 257-282.

[21] Wu Y., Xiao K., Liu H., Tang H.. Evolution of BBS virtual community and its simulation[J]. Systems Engineering-Theory & Practice, 2010, 30(10): 1883-1890.

[22] Zhang W., Yoshida T., Tang X.J.. Text Classification Based on Multi-word with Support Vector Machine. Knowledge-Based Systems, 2008, 21(8): 879-886.

[23] Zhao Y.L., Tang X.J.. A Preliminary Research of Pattern of Users’ Behavior Based on Tianya Forum. Paper presented at the 14th International Symposium on Knowledge and Systems Sciences, 139–145 Ningbo, Octobrt 25–27, 2013, JAIST Press, 2013

[24] Zheng Y., Tok S.K.. “Harmonious Society” and “Harmonious World”: China’s Policy Discourse under Hu Jintao. Briefing Series, Issue 26, China Policy Institute, The University of Nottingham, UK, 2007

[25] Zheng R., Shi K., Li S.. Zhou J.. The InfluenceFactors and Mechanism of Societal Risk Perception. Proceedings of the 1st International Conference on Complex Sciences: Theory and Application, 2266–2275, Shanghai, February 23–25, 2009, Springer Berlin Heidelberg, 2009