Modified Usability Framework for Disaster Apps: A Qualitative Thematic Analysis of User Reviews

Abstract

The public has access to a range of mobile applications (apps) for disasters. However, there has been limited academic research conducted on disaster apps and how the public perceives their usability. This study explores end-users’ perceptions of the usability of disaster apps. It proposes a conceptual framework based on insights gathered from thematically analyzing online reviews. The study identifies new usability concerns particular to disaster apps’ use: (1) content relevance depends on the app’s purpose and the proximate significance of the information to the hazard event’s time and location; (2) app dependability affects users’ perceptions of usability due to the life-safety association of disaster apps; (3) users perceive advertisements to contribute to their cognitive load; (4) users expect apps to work efficiently without unnecessary consumption of critical phone resources; (5) appropriate audio interface can improve usability, as sounds can boost an app’s alerting aspect; and, finally (6) in-app browsing may potentially enhance users’ impression of the structure of a disaster app. As a result, this study argues for focussed research and development on public-facing disaster apps. Future research should consider the conceptual framework and concerns presented in this study when building design guidelines and theories for disaster apps.

Keywords

Disaster apps / End-user perspective / Human–computer interaction / Usability

References

[1] Apple Inc. 2016. The app store. https://www.apple.com/nz/ios/app-store/. Accessed 9 Aug 2019.

[2] Appleby-Arnold S, Brockdorff N, Fallou L, Bossu R. Truth, trust, and civic duty: Cultural factors in citizens’ perceptions of mobile phone apps and social media in disasters. Journal of Contingencies and Crisis Management, 2019, 29(4): 293-305

[3] Bachmann DJ, Jamison NK, Martin A, Delgado J, Kman NE. Emergency preparedness and disaster response: There’s an app for that. Prehospital and Disaster Medicine, 2015, 30(5): 486-490

[4] Balapour, A., and S.M. Walton. 2017. Usability of apps and websites: A meta-regression study. In Proceedings of the Twenty-third Americas Conference on Information Systems, 10–12 August 2017, Boston, MA, USA, ed. AMCIS 2017 organization, 1–10. Atlanta, GA: AIS

[5] Bopp, E., J. Douvinet, and D. Serre. 2019. Sorting the good from the bad smartphone applications to alert residents facing disasters—Experiments in France. In Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management (ISCRAM), ed. Z. Franco, J.J. Gonzales, and J.H. Canós, 435–449. València: ISCRAM

[6] Brajnik G, Gabrielli S. A review of online advertising effects on the user experience. International Journal of Human-Computer Interaction, 2010, 26(10): 971-997

[7] Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology, 2006, 3(2): 77-101

[8] Braun V, Clarke V. Braun V, Clarke V. Moving towards analysis. Successful qualitative research: A practical guide for beginners, 2013, London: SAGE 173-200.

[9] Cosgrove S. Exploring usability and user-centered design through emergency management websites. Communication Design Quarterly Review, 2018, 6(2): 93-102

[10] Coursaris C, Kim D. A meta-analytical review of empirical mobile usability studies. Journal of Usability Studies, 2011, 6(3): 117-171.

[11] Ding Y, Chai KH. Emotions and continued usage of mobile applications. Industrial Management & Data Systems, 2015, 115(5): 833-852

[12] Dou W, Lim KH, Su C, Zhou N, Cui N. Brand positioning strategy using search engine marketing. MIS Quarterly, 2010, 34(2): 261-279

[13] Estuar, M.R.J., M. De Leon, M.D. Santos, J.O. Ilagan, and B.A. May. 2014. Validating UI through UX in the context of a mobile-web crowdsourcing disaster management application. In Proceedings of 2014 International Conference on IT Convergence and Security (ICITCS), 28–30 October 2014, Beijing, China, ed. D. Lee, F. He, and H. Kim, 1–4. Red Hook, NY: IEEE.

[14] Ettinger KM, Pharaoh H, Buckman RY, Conradie H, Karlen W. Building quality mHealth for low resource settings. Journal of Medical Engineering & Technology, 2016, 40(7–8): 431-443

[15] Fallou, L., L. Petersen, and F. Roussel. 2019. Efficiently allocating safety tips after an earthquake—Lessons learned from the smartphone application LastQuake. In Proceedings of the 16th International Conference on Information Systems for Crisis Response and Management (ISCRAM), ed. Z. Franco, J.J. Gonzales, and J.H. Canós, 263–275. València: ISCRAM.

[16] Felt, A.P., S. Egelman, and D. Wagner. 2012. I’ve got 99 problems, but vibration ain’t one: A survey of smartphone users’ concerns. In Proceedings of the Second ACM Workshop on Security and Privacy in Smartphones and Mobile Device, ed. T. Yu, W. Enck, and X. Jiang, 33–44. Raleigh: ACM.

[17] Ferreira D, Dey AK, Kostakos V. Lyons K, Hightower J, Huang EM. Understanding human-smartphone concerns: A study of battery life. International Conference on Pervasive Computing, 2011, Berlin, Heidelberg: Springer 19-33

[18] Gebauer J, Tang Y, Baimai C. User requirements of mobile technology: Results from a content analysis of user reviews. Information Systems and E-Business Management, 2008, 6(4): 361-384

[19] Gómez, D., A.M. Bernardos, J.I. Portillo, P. Tarrío, and J.R. Casar. 2013. A review on mobile applications for citizen emergency management. In Proceedings of International Conference on Practical Applications of Agents and Multi-Agent Systems–PAAMS, ed. J.M. Corchado, J. Bajo, J. Kozlak, P. Pawlewski, J.M. Molina, V. Julian, R.A. Silveira, R. Unland, and S. Giroux, 190–201. Berlin: Springer.

[20] Google. 2016. Google play. https://play.google.com/store/apps?hl=en. Accessed 9 Aug 2019.

[21] Gu B, Hu F, Liu H. Sampling and its application in data mining: A survey, 2000, Singapore: National University of Singapore

[22] Harrison, R., D. Flood, and D. Duce. 2013. Usability of mobile applications: Literature review and rationale for a new usability model. Journal of Interaction Science 1: Article 1.

[23] Hedegaard, S., and J.G. Simonsen. 2013. Extracting usability and user experience information from online user reviews. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems—CHI’13, ed. W.E. Mackay, S. Brewster, and S. Bødker, 2089–2098. New York: Association for Computing Machinery.

[24] Hertzum M. Images of usability. International Journal of Human-Computer Interaction, 2010, 26(6): 567-600

[25] Hiltz SR, de Walle BV, Turoff M. de Walle BV, Turoff M, Hiltz SR. The domain of emergency management information. Information systems for emergency management, 2010, New York: Routledge 3-20.

[26] Hoehle H, Venkatesh V. Mobile application usability: Conceptualization and instrument development. MIS Quarterly, 2015, 39(2): 435-472

[27] Houston JB, Hawthorne J, Perreault MF, Park EH, Hode MG, Halliwell MR, McGowen SET, Davis R Social media and disasters: A functional framework for social media use in disaster planning, response, and research. Disasters, 2015, 39(1): 1-22

[28] ISO (International Organization for Standardization) ISO 9241-11:1998 Ergonomic requirements for office work with visual display terminals (VDTs)–Part 11: Guidance on usability, 1998, Geneva: ISO

[29] Karl I, Rother K, Nestler S. Crisis-related apps: Assistance for critical and emergency situations. International Journal of Information Systems for Crisis Response and Management, 2015, 7(2): 19-35

[30] Korhonen, H., J. Holm, and M. Heikkinen. 2007. Utilizing sound effects in mobile user interface design. In Human-Computer Interaction–INTERACT 2007: Proceedings of the 11th IFIP TC 13 International Conference, Rio de Janeiro, Brazil, 10–14 September 2007, ed. C. Baranauskas, P. Palanque, J. Abascal, S.D.J. Borbosa, 283–296. Berlin: Springer.

[31] Kwee-Meier ST, Wiessmann M, Mertens A. Harris D. Integrated information visualization and usability of user interfaces for safety-critical contexts. Lecture notes in computer science, 2017, Berlin: Springer 71-85.

[32] Landis JR, Koch GG. The measurement of observer agreement for categorical data. International Biometric Society, 1977, 33(1): 159-174

[33] Mashhadi, A., A. Mathur, and F. Kawsar. 2014. The myth of subtle notifications. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct publication, ed. A.J. Brush, and A. Friday, 111–114. New York: Association for Computing Machinery.

[34] McCurdie T, Taneva S, Casselman M, Yeung M, Mcdaniel C, Ho W, Cafazzo J. mHealth consumer apps: The case for user-centered design. Biomedical Instrumentation & Technology: Mobile Health, 2012, 46(s2): 49-56

[35] McIlroy S, Ali N, Khalid H, Hassan AE. Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews. Empirical Software Engineering, 2015, 21(3): 1067-1106

[36] Mills, A., R. Chen, J. Lee, and H.R. Rao. 2009. Web 2.0 emergency applications: How useful can Twitter be for emergency response? Journal of Information Privacy & Security 5(3): 3–26.

[37] Nielsen J. Usability engineering, 1994, Amsterdam: Elsevier

[38] Oxford Dictionary Definition of usability in English, 2019, Oxford: Oxford University Press

[39] Palomba, F., M. Linares-Vasquez, G. Bavota, R. Oliveto, M.D. Penta, D. Poshyvanyk, and A. De Lucia. 2015. User reviews matter! Tracking crowdsourced reviews to support evolution of successful apps. In Proceedings of IEEE 31st International Conference on Software Maintenance and Evolution, 29 September–1 October 2015, Bremen, Germany, ed. R. Koschke, J. Krinke, and M. Robillard, 291–300. Bassau: IEEE.

[40] Reuter C, Spielhofer T. Towards social resilience: A quantitative and qualitative survey on citizens’ perception of social media in emergencies in Europe. Technological Forecasting and Social Change, 2017, 121: 168-180

[41] Reuter, C., M.-A. Kaufhold, T. Spielhofer, and A.S. Hahne. 2017. Social media in emergencies: A representative study on citizens’ perception in Germany. PACM on Human-Computer Interaction 1(2): Article 90.

[42] Rosenthal, S., A.K. Dey, and M. Veloso. 2011. Using decision-theoretic experience sampling to build personalized mobile phone interruption models. In Proceedings of the 9th International Conference on Pervasive Computing, 12–15 June 2011, San Francisco, CA, USA, ed. K. Lyons, J. Hightower, and E.M. Huang, 170–187. Berlin: Springer.

[43] Sarshar P, Nunavath V, Radianti J. Kurosu M. On the usability of smartphone apps in emergencies: An HCI analysis of GDACSmobile and SmartRescue apps. Human-computer interaction: Interaction technologies, 2015, Cham: Springer 765-774

[44] Schimak, G., D. Havlik, and J. Pielorz. 2015. Crowdsourcing in crisis and disaster management—Challenges and considerations. In Environmental software systems. Infrastructures, services and applications, ed. R. Denzer, R.M. Argent, G. Schimak, and J. Hřebíček, 56–70. Cham: Springer.

[45] Stratmann TC, Boll S. Bogdan C, Gulliksen J, Sauer S, Forbrig P, Winckler M, Johnson C, Palanque P, Bernhaupt R, Kis F. Demon hunt—The role of Endsley’s demons of situation awareness in maritime accidents. Human-centered and error-resilient systems development, 2016, Berlin: Springer 203-212

[46] Tan ML, Prasanna R, Stock K, Hudson-Doyle E, Leonard G, Johnston D. Mobile applications in crisis informatics literature: A systematic review. International Journal of Disaster Risk Reduction, 2017, 24: 297-311

[47] Westermann T. Westermann T. Permission requests and notification settings. User acceptance of mobile notifications, 2017, Singapore: Springer 81-102

[48] Zahra, F., A. Hussain, and H. Mohd. 2017. Usability evaluation of mobile applications; where do we stand? In Proceedings of the 2nd International Conference on Applied Science and Technology, ed. F.A.A. Nifa, K.L. Chong, and A. Hussain, Vol. 1891, Article 020056. Melville, NY: AIP Publishing.

[49] Zhang D, Adipat B. Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction, 2005, 18(3): 293-308