Event analysis in social multimedia: a survey

Abstract

Recent years have witnessed the rapid growth of social multimedia data available over the Internet. The age of huge amount of media collection provides users facilities to share and access data, while it also demands the revolution of data management techniques, since the exponential growth of social multimedia requires more scalable, effective and robust technologies to manage and index them. The event is one of the most important cues to recall people’s past memory. The reminder value of an event makes it extremely helpful in organizing data. The study of event based analysis on social multimedia data has drawn intensive attention in research community. In this article, we provide a comprehensive survey on event based analysis over social multimedia data, including event enrichment, detection, and categorization. We introduce each paradigm and summarize related research efforts. In addition, we also suggest the emerging trends in this research area.

Keywords

event enrichment / event detection / event categorization

References

[1] Mor N. Social Multimedia: highlighting opportunities for search and mining of multimedia aata in social media applications. Multimedia Tools and Applications, 2012, 56(1): 9–34

[2] Luo J B, Dhiraj J, Yu J, Andrew G. Geotagging in multimedia and computer vision—a survey. Multimedia Tools and Applications, 2011, 51(1): 187–211

[3] Alessandro V, Maja P, Herve B. Social signal processing: survey of an emerging domain. Image and Vision Computing, 2009, 27(12): 1743–1759

[4] Mei T, Rui Y, Li S P, Tian Q. Multimedia search reranking: a literature survey. ACM Computing Surveys, 2014, 46(3)

[5] Wang M, Ni B B, Hua X S, Tat-Seng C. Assistive tagging: a survey of multimedia tagging with human-computer joint exploration. ACM Computing Surveys, 2012, 44(4)

[6] Yang Q. Three challenges in data mining. Frontiers of Computer Science in China, 2010, 4(3): 324–333

[7] Ma H X, Qian W N, Xia F, He X F, Xu J, Zhou A Y. Towards modeling popularity of microblogs. Frontiers of Computer Science, 2013, 7(2): 171–184

[8] Utz W, Ramesh J. Toward a common event model for multimedia applications. IEEE MultiMedia, 2007, 14(1): 19–29

[9] Liu X L, Raphaël T, Benoit H. Finding media illustrating events. In: Proceedings of ACM International Conference on Multimedia Retrieval, 2011

[10] Scherp A, Jain R, Kankanhalli M, Mezaris V. Modeling, detecting, and processing events in multimedia. In: Proceedings of the International Conference on Multimedia, 2010, 1739–1740

[11] Petkos G, Papadopoulos S, Mezaris V, Troncy R, Cimiano P, Reuter T, Kompatsiaris Y. Social event detection at mediaeval: a threeyear retrospect of tasks and results. In: Proceedings of ACM ICMR 2014 Workshop on Social Events in Web Multimedia, 2014

[12] Reuter T, Papadopoulos S, Petkos G, Mezaris V, Kompatsiaris Y, Cimiano P, de Vries C, Geva S. Social event detection at mediaeval 2013: challenges, datasets, and evaluation. In: Proceedings of the MediaEval 2013 Multimedia Benchmark Workshop, 2013

[13] Troncy R, Malocha B, Fialho A T S. Linking events with media. In: Proceedings of the 6th International Conference on Semantic Systems, 2010

[14] Becker H, Naaman M, Gravano L.Learning similarity metrics for event identification in social media. In: Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. 2010, 291–300

[15] Liu X LTroncy R, Huet B. Usingsocial media to identifyevents. In Proceedings of the ACM SIGMM Workshop on Social Media. 2011, 3–8

[16] Trabelsi C, Yahia S B. A probabilistic approach for events identification from social media RSS feeds. Database Systems for Advanced Applications, 2013, 7827: 139–152

[17] Firan C S, Georgescu M, Nejdl W, Paiu R. Bringing order to your photos: event-driven classification of Flickr images based on social knowledge. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, 2010, 189–198

[18] Liu X L, Huet B. Heterogeneous features and model selection for event-based media classification. In: Proceedings of International Conference on Multimedia Retrieval. 2013, 151–158

[19] Billsus D, Pazzani M J. A hybrid user model for news story classification. In: Proceedings of the 7th International Conference on User Modeling. 1999, 99–108

[20] Toda H, Kataoka R. A clustering method for news articles retrieval system. In: Proceedings of the International Conference on World Wide Web. 2005

[21] Delgado D, Magalhães J, Correia N. Assisted news reading with automated illustrations. In: Proceedings of ACM Conference on Multimedia. 2010, 1647–1650

[22] Joshi D, Wang J Z, Jia L. The story picturing engine—a system for automatic text illustration. ACM Transactions on Multimedia Computing Communications and Applications, 2006, 2(1): 68–89

[23] Cooper M, Foote J, Girgensohn A, Wilcox L. Temporal event clustering for digital photo collections. In: Proceedings of ACM International Conference on Multimedia, 2003

[24] Graham A, Garcia-Molina H, Paepcke A, Winograd T. Time as essence for photo browsing through personal digital libraries. In: Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries. 2002, 326–335

[25] Jou B, Li H Z, Ellis J G, Morozoff-Abegauz D, Chang S F. Structured exploration of who, what, when, and where in heterogeneous multimedia news sources. In: Proceedings of the ACM International Conference on Multimedia. 2013, 357–360

[26] Kim M, Xie L X, Christen P. Event diffusion patterns in social media. In: Proceedings of the International AAAI Conference onWeblogs and Social Media. 2012

[27] Burton K, Kasch N, Soboroff I. The icwsm 2011 Spinn3r dataset. In: Proceedings of the 5th Annual Conference on Weblogs and Social Media. 2011

[28] Mattivi R, Uijlings J, De Natale F, Sebe N. Categorization of a collection of pictures into structured events. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

[29] Mattivi R, Uijlings J, De Natale F G B, Sebe N. Exploitation of Time Constraints for (sub-)Event Recognition. In: Proceedings of the 2011 Joint ACM Workshop on Modeling and Representing Events. 2011, 7–12

[30] Cooper M, Foote J, Girgensohn A, Wilcox L. Temporal event clustering for digital photo collections. ACM Transactions on Multimedia Computing, Communications, and Applications, 2005, 1(3): 269–288

[31] Sinha P, Jain R. Extractive summarization of personal photos from life events. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2011

[32] Kennedy L, Naaman m. Less talk, more rock: automated organization of community-contributed collections of concert videos. In: Proceedings of the 18th ACM International Conference on World Wide Web. 2009, 311–320

[33] Diakopoulos N, Naaman M, Kivran-Swaine F. Diamonds in the rough: social media visual analytics for journalistic inquiry. In: Proceedings of 2010 IEEE Symposium on Visual Analytics Science and Technology. 2010, 115–122

[34] Gao M Y, Hua X S, Jain R. WonderWhat: real-time event determination from photos. In: Proceedings of the 20th World Wide Web Conference. 2011

[35] Liu X L, Huet B. Event representation and visualization from social media. Lecture Notes in Computer Science, 2013, 8294: 740–749

[36] Tang J L, Wang X F, Gao H J, Hu X, Liu H. Enriching short text representation in microblog for clustering. Frontiers of Computer Science, 2012, 6(1): 88–101

[37] Zheng X L, Zhong Y G, Daniel Z, Wang F Y. Social influence and spread dynamics in social networks. Frontiers of Computer Science, 2012, 6(5): 611–620

[38] Liu X L, Huet B. Automatic concept detector refinement for large-scale video semantic annotation. In: Proceedings of the 4th IEEE International Conference on Semantic Computing. 2010, 97–100

[39] Bird S, Klein E, Loper E. Natural language processing with python. Sebastopol: O’Reilly Media, Inc., 2009

[40] Auer S, Bizer C, Kobilarov G, Lehmann J, Ives Z. Dbpedia: a nucleus for a Web of open data. In: Proceedings of the 6th International Semantic Web Conference. 2007, 11–15

[41] Sun C J, Guan Y. A statistical approach for content extraction from Web page. Journal of Chinese Information Processing, 18(5): 17–22, 2004

[42] Mei T, Yang B, Yang S Q, Hua X S. Video collage: presenting a video sequence using a single image. The Visual Computer, 2009, 25(1): 39–51

[43] Aouiche K, Lemire D, Godin R. Web 2.0 OLAP: from data cubes to tag clouds. Lecture Notes in Business Information Processing, 2009

[44] Quack T, Leibe B, Van Gool L. World-scale mining of objects and events from community photo collections. In: Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, 2008, 47–56

[45] Papadopoulos S, Zigkolis C, Kompatsiaris Y, Vakali A. Cluster-based landmark and event detection for tagged photo collections. IEEE Multimedia, 2011, 18(1): 52–63

[46] Becker H, Naaman H, Gravano L. Event identification in social media. In: Proceedings of the 12th International Workshop on the Web and Databases. 2009

[47] Petkos G, Papadopoulos S, Schinas E, Kompatsiaris Y. Graph-based multimodal clustering for social event detection in large collections of images. Lecture Notes in Computer Science, 2014, 8325: 146–158

[48] Petkos G, Papadopoulos S, Kompatsiaris Y. Social event detection using multimodal clustering and integrating supervisory signals. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

[49] Rattenbury T, Good N, Naaman M. Towards automatic extraction of event and place semantics from flickr tags. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval. 2007, 103

[50] Liu Z Y, Chen X X, Sun M S. Mining the interests of chinese microbloggers via keyword extraction. Frontiers of Computer Science, 2012, 6(1): 76–87

[51] Chen L, Abhishek R. Event detection from Flickr data through waveletbased spatial analysis. In: Proceedings of ACM conference on CIKM. 2009

[52] Weng J S, Lee B S. Event detection in twitter. In: Proceedings of International AAAI Conference on Weblogs and Social Media. 2011

[53] Deerwester S, Dumais S T, Furnas G W, Landauer T K, Harshman R. Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 1990, 41(6): 391–407

[54] Hofmann T. Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 1999, 50–57

[55] Blei D M, Ng A Y, Jordan M I. Latent Dirichlet Allocation. Journal of Machine Learning Research. 2003, 3(4–5): 993–1022

[56] Pan C C, Mitra P. Event detection with spatial latent dirichlet allocation. In: Proceeding of the 11th Annual International ACM/IEEE joint Conference on Digital Libraries. 2011, 349

[57] Liu X L, Huet B. Social event discovery by topic inference. In: Proceeding of IEEE International Workshop on Image Analysis for Multimedia Interactive Services. 2012, 1–4

[58] Wang X, Zhu F, Jiang J, Li S J.Real time event detection in twitter. Lecture Notes in Computer Science, 2013, 7923: 502–513

[59] The Y W, Jordan M I, Beal M J, Blei D M. Hierarchical dirichlet processes. Journal of the American Statistical Association, 2004, 101

[60] Cheng T, Wicks T. Event detection using Twitter: a spatio-temporal approach. PLoS ONE, 2014, 9(6): e97807

[61] Reuter T, Cimiano P. Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, 2012

[62] Wang Y X, Sundaram H, Xie L X. Social event detection with interaction graph modeling. In: Proceedings of the 20th ACM International Conference on Multimedia, 2012, 865–868

[63] Brenner M, Izquierdo E. Social event detection and retrieval in collaborative photo collections. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

[64] Liu X L, Huet B, Troncy R. Eurecom mediaeval 2011 social event detection task. MediaEval, 2011

[65] Li R H, Liu J Q, Yu J X, Chen H X, Kitagawa H. Co-occurrence prediction in a large location-based social network. Frontiers of Computer Science, 2013, 7(2): 185–194

[66] Makkonen J, Ahonen-Myka H, Salmenkivi M. Simple semantics in topic detection and tracking. Information Retrieval, 2004, 3(7): 347–368

[67] Reuter T, Cimiano P. Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. 2012

[68] Gu J M, Wu Y L, Hung W C, Tang C Y. Personal photo organization using event annotation. In: Proceedings of the 9th International Conference on Information, Communications and Signal Processing. 2013, 1–4

[69] Qian S H, Zhang T Z, Xu C S. Multimodal supervised latent dirichlet allocation for event classification in social media. In: Proceedings of International Conference on Internet Multimedia Computing and Service, 2014, 152–157

[70] Suchanek F M, Kasneci G, Weikum G. Yago: a large ontology from Wikipedia and WordNet. Web Semantics: Science, Services and Agents on the World Wide Web, 2008, 6(3): 203–217

[71] Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten I H. The WEKA data mining software: an update. ACM SIGKDD Explorations Newsletter, 2009, 11(1): 10–18

[72] Chang C C, Lin C J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 27:1–27

[73] Breiman L, Friedman J, Olshen R, Stone C. Classification and regression trees. Monterey: Wadsworth and Brooks, 1984

[74] Breiman L. Random forests. Machine Learning, 2001, 45: 5–32

[75] Hays J, Efros A A. IM2GPS: estimating geographic information from a single image. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2008, 1–8

[76] Lo H Y, Chang K W, Chen S T, Chiang S H, Ferng C S, Hsieh C J, Ko Y K, Kuo T T, Lai H C, Lin K Y, Wang C H, Yu H F, Lin C J, Lin H T, Lin S. An ensemble of three classifiers for KDD Cup 2009: expanded linear model, heterogeneous boosting, and selective naïve bayes. Journal of Machine Learning Research—Proceedings Track, 2009, 7: 57–64

[77] Manning C D, Raghavan P, Schütze H. Introduction to information retrieval. Cambridge: Cambridge University Press, 2008

[78] Chua T S, Tang J H, Hong R C, Li H J, Luo Z P, Zheng Y T. Nus-wide: a real-world Web image database from national university of singapore. In: Proceedings of ACM Conferrence on Image and Video Retrieval. 2009

[79] Hebeler J, Fisher M, Blace R, Perez-Lopez A. Semantic Web programming. Indianapolis: Wiley Publishing, 2009

[80] Petkos G, Papadopoulos S, Mezaris V, Kompatsiaris Y. Social event detection at mediaeval 2014: challenges, datasets, and evaluation. MediaEval, 2013

[81] Over P, Awad G, Michel M, Fiscus J, Sanders G, Kraaij W, Smeaton A F, Quvenot G. TRECVID 2014—an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID. 2010, 15–17

RIGHTS & PERMISSIONS

2014 Higher Education Press and Springer-Verlag Berlin Heidelberg