Visual narratives of disaster on Instagram: a multimodal AI-assisted analysis

Abstract

This paper explores Instagram as a platform for visual and emotional narratives of disaster across the response and recovery phases. Focusing on the extreme flooding that affected western Germany in July 2021, the study analyzes approximately 2,700 Instagram posts collected over a three-year period. It employs a mixed-methods approach combining generative artificial intelligence with qualitative and quantitative analyses. Images were interpreted by ChatGPT-4 under human supervision, and the resulting classifications were validated against manual coding, yielding an agreement rate of 80.73% and a Cohen’s kappa of 0.74. Six visual themes were identified: flood damage, rescue operations, solidarity and resilience, media coverage, post-disaster recovery, and government involvement. The temporal distribution of these themes traces a movement from immediate impacts toward reconstruction and remembrance, while the emotional tone shifts from despair and anxiety toward more consistent expressions of hope. A statistically significant association was found between visual themes and emotional tones. A complementary thematic examination of hashtags shows how public discourse surrounding the disaster was framed, and indicates that images convey a more nuanced understanding of disaster impacts that hashtags alone cannot capture. The study demonstrates how AI-assisted visual analysis can facilitate the large-scale interpretation of social media image corpora and generate insights relevant to disaster response and recovery strategies.

Keywords

Social media / Disaster response / Disaster recovery / Visual analysis / Sentiment analysis / AI-assisted analysis

References

[1] Abdulhamid NG, Ayoung DA, Kashefi A, Sigweni B. A survey of social media use in emergency situations: A literature review. Information Development, 2021, 372274291.

[2] Agarwal, N., & Yiliyasi, Y. (2010). Information quality challenges in social media. Proceedings of the 2010 International Conference on Information Quality (ICIQ). 12–14 November 2010 Little Rock, Arkansas, SA

[3] Alam, F., Ofli, F., Imran, M., Alam, T., & Qazi, U. (2020). Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response. 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 151–158. https://doi.org/10.1109/ASONAM49781.2020.9381294

[4] Al-Shabi, M. (2020). Evaluating the performance of the most important Lexicons used to Sentiment analysis and opinions Mining.

[5] Arif, Amin MA, Ali AA, Rahman AKM. Visual attention-based comparative study on disaster detection from social media images. Innovations in Systems and Software Engineering, 2020, 163–4309319.

[6] Banikalef A, Al Bataineh K, Atoum M. Exploring facebook affordances in natural disaster: a case study of the 2018 dead sea flash floods in Jordan. International Journal of Engineering and Technology, 2018, 4: 5001-5006.

[7] Belal, M., She, J., & Wong, S. (2023). Leveraging ChatGPT As Text Annotation Tool For Sentiment Analysis (arXiv:2306.17177). arXiv. https://doi.org/10.48550/arXiv.2306.17177

[8] Bell JE, Brown CL, Conlon K, Herring S, Kunkel KE, Lawrimore J, Luber G, Schreck C, Smith A, Uejio C. Changes in extreme events and the potential impacts on human health. Journal of the Air & Waste Management Association, 2018, 684265287.

[9] Bica, M., Palen, L., & Bopp, C. (2017). Visual Representations of Disaster. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. https://www.academia.edu/97930921/Visual_Representations_of_Disaster

[10] Blackwood, G., Hardy, A., Dodd, B., Ooi, C. S., & Williams, K. (2023). Natural disaster on Instagram: Examining representations of the 2018–2019 Tasmanian fires. Annals of Tourism Research Empirical Insights, 4(1). Scopus. https://doi.org/10.1016/j.annale.2022.100082

[11] Bossetta M. The digital architectures of social media: Comparing political campaigning on Facebook, Twitter, Instagram, and Snapchat in the 2016 U.S. election. Journalism & Mass Communication Quarterly, 2018, 952471496.

[12] Brandt HM, Turner-Mcgrievy G, Friedman DB, Gentile D, Schrock C, Thomas T, West D. Examining the role of Twitter in response and recovery during and after historic flooding in South Carolina. Journal of Public Health Management and Practice, 2019, 255E6E12.

[13] Brown S, Davidovic J, Hasan A. The algorithm audit: Scoring the algorithms that score us. Big Data & Society, 2021, 812053951720983865.

[14] Bryman A. Social research methods, 2016Oxford University Press

[15] Bui, T. X. (Ed.). (2019). Proceedings of the 52nd Annual Hawaii International Conference on System Sciences: January 8–11, 2019, Maui, Hawaii. University of Hawaii at Manoa, Hamilton Library, ScholarSpace. Hawaii International Conference on System Sciences.

[16] Chen Q, Ragusa E, Chaturvedi I, Cambria E, Zunino R. Gelbukh A. Text-image sentiment analysis. Computational linguistics and intelligent text processing, 2023Springer Nature Switzerland169180.

[17] Dahal L, Idris MS, Bravo V. It helped us, and it hurt us” The role of social media in shaping agency and action among youth in post-disaster Nepal. Journal of Contingencies and Crisis Management, 2021, 292217225.

[18] Dar, S. S., Rehman, M. Z.U., Bais, K., Haseeb, M. A., & Kumar, N. (2024). A social context-aware graph-based multimodal attentive learning framework for disaster content classification during emergencies. Expert Systems with Applications, 259, 125337. https://doi.org/10.1016/j.eswa.2024.125337

[19] David CC, Ong JC, Legara EFT. Tweeting Supertyphoon Haiyan: Evolving functions of Twitter during and after a disaster event. PLoS ONE, 2016, 113e0150190.

[20] de Albuquerque JP, Herfort B, Brenning A, Zipf A. A geographic approach for combining social media and authoritative data towards identifying useful information for disaster management. International Journal of Geographical Information Science, 2015, 294667689.

[21] Eriksson M. Lessons for crisis communication on social media: A systematic review of what research tells the practice. International Journal of Strategic Communication, 2018, 125526551.

[22] Fang, J., Hu, J., Shi, X., & Zhao, L. (2019). Assessing disaster impacts and response using social media data in China: A case study of 2016 Wuhan rainstorm. International Journal of Disaster Risk Reduction, 34, 275–282. Scopus. https://doi.org/10.1016/j.ijdrr.2018.11.027

[23] Filieri R, Yen DA, Yu Q. #ILoveLondon: An exploration of the declaration of love towards a destination on Instagram. Tourism Management, 2021, 85: 104291.

[24] Ginzarly M, Teller J, Dujardin S. Social media use in disaster response: Empowering community resilience. International Journal of Digital Earth, 2025.

[25] Glasgow K, Fink C. Greenberg AM, Kennedy WG, Bos ND. Hashtag lifespan and social networks during the London riots. Social computing, behavioral-cultural modeling and prediction, 2013Springer Berlin Heidelberg311320. Vol. 7812

[26] Glasgow, K., Fink, C., Vitak, J., & Tausczik, Y. (2017). “Our hearts go out”: Social support and gratitude after disaster. 463–469. Scopus. https://doi.org/10.1109/CIC.2016.069

[27] Gopal LS, Prabha R, Thirugnanam H, Ramesh MV, Malamud BD. Review article: Social media for managing disasters triggered by natural hazards: A critical review of data collection strategies and actionable insights. Natural Hazards and Earth System Sciences, 2026, 261215250.

[28] Hao H, Wang Y. Leveraging multimodal social media data for rapid disaster damage assessment. International Journal of Disaster Risk Reduction, 2020, 51: 101760.

[29] Hara Y. Behaviour analysis using tweet data and geo-tag data in a natural disaster. Transportation Research Procedia, Transport Survey Methods: Embracing Behavioural and Technological Changes Selected Contributions from the 10th International Conference on Transport Survey Methods 16–21 November 2014, Leura, Australia., 2015, 11: 399-412.

[30] Hassan, S. Z., Ahmad, K., Hicks, S., Halvorsen, P., Al-Fuqaha, A., Conci, N., & Riegler, M. (2020). Visual Sentiment Analysis from Disaster Images in Social Media (arXiv:2009.03051). arXiv. https://doi.org/10.48550/arXiv.2009.03051

[31] Hong, J. Y. J., Kim, N., Lee, S., & Kim, J. H. (2018). Community Disaster Resilience and Social Solidarity on Social Media: A Semantic Network Analysis of the Sewol Ferry Disaster. Information Research: An International Electronic Journal, 23(3). https://eric.ed.gov/?id=EJ1196300

[32] Houston JB, Hawthorne J, Perreault MF, Park EH, Goldstein Hode M, Halliwell MR, Turner McGowen SE, Davis R, Vaid S, McElderry JA, Griffith SA. Social media and disasters: A functional framework for social media use in disaster planning, response, and research. Disasters, 2015, 391122.

[33] Huang Q, Xiao Y. Geographic situational awareness: Mining tweets for disaster preparedness, emergency response, impact, and recovery. ISPRS International Journal of Geo-Information, 2015.

[34] Hughes, A. L., St. Denis, L. A. A., Palen, L., & Anderson, K. M. (2014). Online public communications by police & fire services during the 2012 Hurricane Sandy. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1505–1514. https://doi.org/10.1145/2556288.2557227

[35] Ilyas B, Sharifi A. A systematic review of social media-based sentiment analysis in disaster risk management. International Journal of Disaster Risk Reduction, 2025, 123: 105487.

[36] Jaeger PT, Shneiderman B, Fleischmann KR, Preece J, Qu Y, Wu PF. Community response grids: E-government, social networks, and effective emergency management. Telecommunications Policy, 2007, 3110592604.

[37] Johannes BL, Ortiz SE. How Facebook is used to promote ENDS products near four Big 10 universities: A qualitative analysis. Journal of American College Health, 2024, 721210218.

[38] Johnson, O. V., Alyasiri, O. M., Akhtom, D., & Johnson, O. E. (2023). Image Analysis through the lens of ChatGPT-4. Journal of Applied Artificial Intelligence, 4(2), Article 2. https://doi.org/10.48185/jaai.v4i2.870

[39] Karimiziarani M, Jafarzadegan K, Abbaszadeh P, Shao W, Moradkhani H. Hazard risk awareness and disaster management: Extracting the information content of Twitter data. Sustainable Cities and Society, 2022, 77: 103577.

[40] Kaur, H. J., & Kumar, R. (2015). Sentiment analysis from social media in crisis situations. Communication & Automation International Conference on Computing, 251–256. https://doi.org/10.1109/CCAA.2015.7148383

[41] Khanmohammadi S, Golafshani E, Bai Y, Li H, Bazli M, Arashpour M. Multi-modal mining of crowd-sourced data: Efficient provision of humanitarian aid to remote regions affected by natural disasters. International Journal of Disaster Risk Reduction, 2023, 96: 103972.

[42] Kogan, M., Palen, L., & Anderson, K. M. (2015). Think Local, Retweet Global: Retweeting by the Geographically-Vulnerable during Hurricane Sandy. Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, CSCW ’15, 981–993. https://doi.org/10.1145/2675133.2675218

[43] Lazreg, M. B., Chakraborty, N. R., Stieglitz, S., Potthoff, T., Ross, B., & Majchrzak, T. A. (2018, August 24). Social Media Analysis in Crisis Situations: Can Social Media be a Reliable Information Source for Emergency Management Services? Designing Digitalization (ISD2018 Proceedings). 27th International Conference on Information Systems Development. https://www.research.ed.ac.uk/en/publications/social-media-analysis-in-crisis-situations-can-social-media-be-a-

[44] Lubiana T, Lopes R, Medeiros P, Silva JC, Goncalves ANA, Maracaja-Coutinho V, Nakaya HI. Ten quick tips for harnessing the power of ChatGPT in computational biology. PLoS Computational Biology, 2023, 198e1011319.

[45] Luna S, Pennock MJ. Social media applications and emergency management: A literature review and research agenda. International Journal of Disaster Risk Reduction, 2018, 28: 565-577.

[46] Ma Z, Li L, Mao Y, Wang Y, Patsy OG, Bensi MT, Hemphill L, Baecher GB. Surveying the use of social media data and natural language processing techniques to investigate natural disasters. Natural Hazards Review, 2024, 25403124003.

[47] Manandhar B, Cui S, Wang L, Shrestha S. Post-flood resilience assessment of July 2021 flood in Western Germany and Henan, China. Land, 2023, 123Article 3.

[48] Mao Y, Liu Q, Zhang Y. Sentiment analysis methods, applications, and challenges: A systematic literature review. Journal of King Saud University - Computer and Information Sciences, 2024, 364102048.

[49] Mavrodieva AV, Shaw R. Shaw R, Kakuchi S, Yamaji M. Social media in disaster management. Media and disaster risk reduction: Advances, challenges and potentials, 2021Springer5573.

[50] Mihunov VV, Jafari NH, Wang K, Lam NSN, Govender D. Disaster impacts surveillance from social media with topic modeling and feature extraction: Case of hurricane harvey. International Journal of Disaster Risk Science, 2022, 135729742.

[51] Mishra A, Danzy B, Soni U, Arunkumar A, Huang J, Kwon BC, Bryan C. PromptAid: Visual prompt exploration, perturbation, testing and iteration for large language models. IEEE Transactions on Visualization and Computer Graphics, 2025.

[52] Moghadas M, Fekete A, Rajabifard A, Kötter T. The wisdom of crowds for improved disaster resilience: A near-real-time analysis of crowdsourced social media data on the 2021 flood in Germany. GeoJournal, 2023, 88442154241.

[53] Murthy D, Gross A, McGarry M. Visual social media and big data. Interpreting Instagram images posted on Twitter. Digital Culture & Society, 2016, 22113134.

[54] Nguyen, D. T., Ofli, F., Imran, M., & Mitra, P. (2017). Damage Assessment from Social Media Imagery Data During Disasters. Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, 569–576. https://doi.org/10.1145/3110025.3110109

[55] NRCFacing hazards and disasters: Understanding human dimensions, 2006National Academies Press

[56] Ofli, F., Alam, F., & Imran, M. (2020). Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response. In A. L. Hughes, F. McNeill, & C. Zobel, (Eds.), Social Media for Disaster Response and Resilience.

[57] Ogie, R. I., Clarke, R. J., Forehead, H., & Perez, P. (2019). Crowdsourced social media data for disaster management: Lessons from the PetaJakarta.org project. Computers, Environment and Urban Systems, 73, 108–117. Scopus. https://doi.org/10.1016/j.compenvurbsys.2018.09.002

[58] Ogie RI, James S, Moore A, Dilworth T, Amirghasemi M, Whittaker J. Social media use in disaster recovery: A systematic literature review. International Journal of Disaster Risk Reduction, 2022, 70: 102783.

[59] Ogie R, Moore A, Wickramasuriya R, Amirghasemi M, James S, Dilworth T. Twitter data from the 2019–20 Australian bushfires reveals participatory and temporal variations in social media use for disaster recovery. Scientific Reports, 2022, 12116914.

[60] Olson MK, Sutton J, Vos SC, Prestley R, Renshaw SL, Butts CT. Build community before the storm: The National Weather Service’s social media engagement. Journal of Contingencies and Crisis Management, 2019, 274359373.

[61] Oulahen G, Vogel B, Gouett-Hanna C. Quick response disaster research: Opportunities and challenges for a new funding program. International Journal of Disaster Risk Science, 2020, 115568577.

[62] Palen L, Hughes AL. Rodríguez H, Donner W, Trainor JE. Social media in disaster communication. Handbook of disaster research, 2018Springer International Publishing497518.

[63] Pearce, W., Özkula, S. M., Greene, A. K., Teeling, L., Bansard, J. S., Omena, J. J., & Rabello, E. T. (2018). Visual coss-platform analysis: Digital methods to research social media images. Inf Commun Soc, 1–20. Scopus.

[64] Ragini JR, Anand PMR, Bhaskar V. Big data analytics for disaster response and recovery through sentiment analysis. International Journal of Information Management, 2018, 42: 13-24.

[65] Rasmussen J, Ihlen Ø. Risk, crisis, and social media: A systematic review of seven years’ research. Nordicom Review, 2017, 382117.

[66] Reuter C, Kaufhold M. Fifteen years of social media in emergencies: A retrospective review and future directions for crisis informatics. Journal of Contingencies and Crisis Management, 2018, 2614157.

[67] Samofalova, Y. (2024). Strategies and challenges for constructing and collecting visual corpora from image-based social media platforms. Studies in Communication Sciences, 1–16. https://doi.org/10.24434/j.scoms.2024.01.3881

[68] Semaan, B., & Mark, G. (2012). “Facebooking” towards crisis recovery and beyond: Disruption as an opportunity. 27–36. Scopus. https://doi.org/10.1145/2145204.2145214

[69] Shibuya, Y. (2017). Mining social media for disaster management: Leveraging social media data for community recovery. 2018-January, 3111–3118. Scopus. https://doi.org/10.1109/BigData.2017.8258286

[70] Stolero N, Elkady S, Labaka L, Verlin J, Branlat M, Adini B. Using social media in disaster management: The perceptions of emergency responders versus the public. Risk, Hazards & Crisis in Public Policy, 2024, 152128161.

[71] Subba, R. (2016). Online Convergence Behavior, Social Media Communications and Crisis Response: An Empirical Study of the 2015 Nepal Earthquake Police Twitter Project. https://doi.org/10.24251/HICSS.2017.034

[72] UNISDR. (2015). Sendai Framework for Disaster Risk Reduction 2015–2030—English. https://climate-adapt.eea.europa.eu/en/metadata/publications/sendai-framework-for-disaster-risk-reduction-2015-2030

[73] Valdez D, ten Thij M, Bathina K, Rutter LA, Bollen J. Social media insights into US mental health during the COVID-19 pandemic: Longitudinal analysis of Twitter data. Journal of Medical Internet Research, 2020, 2212e21418.

[74] Vieweg, S., Hughes, A. L., Starbird, K., & Palen, L. (2010). Microblogging during two natural hazards events: What twitter may contribute to situational awareness. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’10, 1079–1088. https://doi.org/10.1145/1753326.1753486

[75] Willson, G., Wilk, V., Sibson, R., & Morgan, A. (2021). Twitter content analysis of the Australian bushfi res disaster2019-2020: Futures implications. Journal of Tourism Futures, 7(3), 350–355. (world). https://doi.org/10.1108/JTF-10-2020-0183

[76] Zhang C, Fan C, Yao W, Hu X, Mostafavi A. Social media for intelligent public information and warning in disasters: An interdisciplinary review. International Journal of Information Management, 2019, 49: 190-207.

[77] Zhang J, Liao M, Wang Y, Huang Y, Chen F, Makiko C. Multi-modal deep learning framework for damage detection in social media posts. PeerJ Computer Science, 2024, 10: e22622.

Funding

HORIZON-MSCA-2022-PF-01(101106194)

Rights & permissions

The Author(s)