Enhancing chatbot usability and UX through activity theory: A case of the road sign chatbot

Abstract

Many chatbots fail to meet user expectations and are often perceived as not useful due to design, technical, and usability shortcomings. Usability is a critical factor in the design of effective chatbots because it ensures that users can achieve their goals efficiently, effectively, and satisfactorily. A chatbot with high usability enhances the user experience (UX), builds trust, and promotes engagement. UX also plays a pivotal role in the design of effective chatbots, as it directly influences user satisfaction, engagement, and the overall success of interactions. Both usability and UX are critical factors in the design of effective chatbots, as they influence how easily and satisfactorily users can interact with the system. Activity theory provides a robust framework for understanding and designing usability for effective chatbots by focusing on human activities, the tools mediating these activities, and the context in which they occur. It also provides a structured approach for designing UXs by focusing on the interaction between users, tools (e.g., chatbots), and their environment. This paper describes how activity theory has been used to design a road sign chatbot that offers information on road signs in Malaysia to road users. The Road Sign Chatbot was evaluated through a User Acceptance Test and the results revealed that users found the system is user-friendly, satisfactory, and enjoyable to use.

Keywords

Activity theory / Chatbot / Road sign chatbot / Driving license / User experience

References

[1] Manawadu M, Wijenayake U. Voice-Assisted Real-Time Traffic Sign Recognition System Using Convolutional Neural Network. arXiv; 2024. Available from: https://www.arxiv.org/abs/2404.07807 [Last accessed on 2025 Jul 05].

[2] Norman DA. The Design of Everyday Things. New York: Basic Books; 2013.

[3] Nielsen J. Usability inspection methods. In: Conference Companion on Human Factors in Computing Systems. New York: ACM; 1994. p. 413-414.

[4] Abras C, Maloney-Krichmar D, Preece J. User-centered design. In: BainbridgeW,editor. Encyclopedia of Human- Computer Interaction. California: Sage Publications; 2004. p. 445-456.

[5] Shneiderman B. The Eight Golden Rules of Interface Design. Department of Computer Science, University of Maryland; 2016. Available from: https://www.cs.umd.edu/-ben/goldenrules.html [Last accessed on 2025 Jul 05].

[6] Norman DA. Emotional Design. Perché Amiamo (o Odiamo) Gli Oggetti Della Vita Quotidiana. Italy: Apogeo Editore; 2004.

[7] Fogg BJ. Persuasive technology: Using computers to change what we think and do. Ubiquity. 2002; 2002:5. doi: 10.1145/764008.763957

[8] Shevat A. Designing Bots: Creating Conversational Experiences. United States: O’Reilly Media, Inc.; 2017.

[9] Kaptelinin V, Nardi BA. Acting with Technology: Activity Theory and Interaction Design. United States: MIT Press; 2009.

[10] Miola L, Muffato V, Sella E, Meneghetti C, Pazzaglia F. GPS use and navigation ability: A systematic review and meta-analysis. J Environ Psychol. 2024; 99:102417. doi: 10.1016/j.jenvp.2024.102417

[11] Oviedo-Trespalacios O, Haque MM, King M, Washington S. Understanding the impacts of mobile phone distraction on driving performance: A systematic review. Transp Res Part C Emerg Technol. 2016; 72:360-380. doi: 10.1016/j.trc.2016.10.006

[12] Kaplan ED, Hegarty C, editors. Understanding GPS/GNSS: Principles and Applications. New York: Artech House; 2017.

[13] Kasinathan V, Mustapha A, Rani MFCA, Mostafa SA. The role of chatterbots in enhancing tourism: A case study of Penang tourism spots. IAES Int J Artif Intell. 2020; 9(4):569-575. doi: 10.11591/ijai.v9.i4.pp569-575

[14] Kasinathan V, Mustapha A, Dhanoa CK. Shadow puppet chatbot:Preserving cultural tourism. In: International Workshop on Learning Technology for Education Challenges. Switzerland: Springer Nature; 2024. p. 143-154.

[15] Vo NN, Vu QT, Vu NH, Vu TA, Mach BD, Xu G. Domain-specific NLP system to support learning path and curriculum design at tech universities. Comput Educ Artif Intell. 2022; 3:100042. doi: 10.1016/j.caeai.2021.100042

[16] Georgiev P, Lei VI, Burnell R, et al. Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context. arXiv; 2024. Available from: https://www.arxiv.org/abs/2403.05530 [Last accessed on 2025 Jul 05].

[17] Huang YSS, Dootson P. Chatbots and service failure: When does it lead to customer aggression. J Retail Consum Serv. 2022; 68:103044. doi: 10.1016/j.jretconser.2022.103044

[18] Rodríguez Cardona D, Janssen A, Guhr N, Breitner MH, Milde J. A Matter of Trust? Examination of Chatbot Usage in Insurance Business; 2021. Available from: http://hdl.handle.net/10125/70679 [Last accessed on 2025 Jul 05].

[19] Feine J, Morana S, Gnewuch U. Measuring Service Encounter Satisfaction with Customer Service Chatbots using Sentiment Analysis; 2019. Available from: https://aisel.aisnet.org/wi2019/track10/papers/2/ [Last accessed on 2025 Jul 05].

[20] Rese A, Ganster L, Baier D. Chatbots in retailers’ customer communication: How to measure their acceptance? J Retail Consum Serv. 2020; 56:102176. doi: 10.1016/j.jretconser.2020.102176

[21] Aditya EW, Ismail S, Allias N. Implementation of intelligent chatbot in student portal:A systematic literature review. In: 2022 International Visualization, Informatics and Technology Conference (IVIT). United States: IEEE; 2022. p. 47-51.

[22] Norman DA, Draper SW. User Centered System Design: New Perspectives on Human-Computer Interaction. New Jersey: L. Erlbaum Associates Inc.; 1986.

[23] Engeström Y. An Activity-Theoretical Approach to Developmental Research. New Delhi: Orienta-Konsultit; 1987.

[24] Kuutti K. Activity theory as a potential framework for human-computer interaction research. In: Context and Consciousness: Activity Theory and Human-Computer Interaction. United States: MIT Press; 1996.

[25] Bødker S. Activity theory as a challenge to systems design. In: Proceedings of the IFIP TC. Berlin: Springer; 1991.

[26] Bødker S, Grønbæk K. Cooperative prototyping: Users and designers in mutual activity. Int J Man Mach Stud. 1991; 34(3):453-478. doi: 10.1016/0020-7373(91)90030-B

[27] Engeström Y. Developmental studies of work as a test bench of activity theory:The case of primary care medical practice. In: Understanding Practice: Perspectives on Activity and Context. Cambridge: Cambridge University Press; 1993. p. 64-103.

[28] Jonassen DH, Rohrer-Murphy L. Activity theory as a framework for designing constructivist learning environments. Educ Technol Res Dev. 1999; 47(1):61-79. doi: 10.1007/BF02299477

[29] Bødker S. Computers in mediated human activity. Mind Cult Act. 1997; 4(3):149-158. doi: 10.1207/s15327884mca0403_2

[30] Bernard RG, Enyedy ND. Activity centred design:Towards a theoretical framework for CSCL. In: Paper Presented at the Computer Supported Collaborative Learning (CSCL) Conference. United States: University of California at Berkeley; 1999. p. 12-15.

[31] Engeström Y, Brown K, Engeström R, Koistinen K. Organizational Forgetting: An Activity-Theoretical Perspective. London: Sage Publications Inc.; 1990

[32] Kaptelinin V. Creating computer-based work environments:An empirical study of Macintosh users. In: Proceedings of the 1996 ACM SIGCPR/SIGMIS Conference on Computer Personnel Research. New York: ACM; 1996. p. 360-366.

[33] McMichael H. An Activity Based Perspective for Information Systems Research. In: 10th Amsterdam Conference on Information Systems; 1999.

[34] Cameron G, Cameron D, Megaw G, et al. Back to the Future:Lessons from Knowledge Engineering Methodologies for Chatbot Design and Development. In: British HCI Conference 2018. BCS Learning and Development Ltd; 2018.

[35] Weber I. Low-code from frontend to backend:Connecting conversational user interfaces to backend services via a low-code IoT platform. In: Proceedings of the 3rd Conference on Conversational User Interfaces. New York: ACM; 2021. p. 1-5.

[36] Wan S. Developing an engati-based library chatbot to improve reference services. In: Innovation and Experiential Learning in Academic Libraries: Meeting the Needs of Twenty-First Century Students. Urban: University of Alberta; 2022. p. 189.

[37] Engeström Y, Miettinen R, Punamäki-Gitai RL, editors. Perspectives on Activity Theory. Cambridge: Cambridge University Press; 1999.

[38] Kaptelinin V, Nardi BA, Macaulay C. Methods & tools: The activity checklist: A tool for representing the “space” of context. Interactions. 1999; 6(4):27-39. doi: 10.1145/306412.306431

[39] Kaenampornpan M, O’Neill E, Ay BB. An Integrated Context Model: Bringing Activity to Context. In: Proceedings of Workshop; 2004.

[40] Radziwill NM, Benton MC. Evaluating Quality of Chatbots and Intelligent Conversational Agents. arXiv; 2017. Available from: https://www.//arxiv.org/abs/1704.04579 [Last accessed on 2025 Jul 05].

[41] Cooper A, Reimann R, Cronin D, Noessel C. About Face: The Essentials of Interaction Design. United States: John Wiley and Sons; 2014.

[42] Munday P. Duolingo: Gamified learning through translation. J Span Lang Teach. 2017; 4(2):194-198. doi: 10.1080/23247797.2017.1396071

[43] Compare Aivo and Engati on Revoyant. See Side-by-Side Comparisons of Features, Pricing, and Customer Reviews.Available from: https://www.revoyant.com/compare/aivo-vs-engati [Last accessed on 2025 Jul 05].

[44] Kofod-Petersen A, Cassens J. Using activity theory to model context awareness. In: International Workshop on Modeling and Retrieval of Context. Berlin: Springer; 2005. p. 1-17.

[45] Engeström Y. Learning by Expanding: An Activity- Theoretical Approach to Developmental Research. Cambridge University Press; 2014.