Using virtual reality to enhance attention for autistic spectrum disorder with eye tracking

Abstract

Attention deficit disorder is a frequently observed symptom in individuals with autism spectrum disorder (ASD). This condition can present significant obstacles for those affected, manifesting in challenges such as sustained focus, task completion, and the management of distractions. These issues can impede learning, social interactions, and daily functioning. This complexity of symptoms underscores the need for tailored approaches in both educational and therapeutic settings to support individuals with ASD effectively. In this study, we have expanded upon our initial virtual reality (VR) prototype, originally created for attention therapy, to conduct a detailed statistical analysis. Our objective was to precisely identify and measure any significant differences in attention-related outcomes between sessions and groups. Our study found that heart rate (HR) and electrodermal activity (EDA) were more responsive to attention shifts than temperature. The ‘Noise’ and ‘Score’ strategies significantly affected eye openness, with the ASD group showing more responsiveness. The control group had smaller pupil sizes, and the ASD group’s pupil size increased notably when switching strategies in Session 1. Distraction log data showed that both ‘Noise’ and ‘Object Opacity’ strategies influenced attention patterns, with the ‘Red Vignette’ strategy showing a significant effect only in the ASD group. The responsiveness of HR and EDA to attention shifts and the changes in pupil size could serve as valuable physiological markers to monitor and guide these interventions. These findings further support evidence that VR has positive implications for helping those with ASD, allowing for more tailored personalized interventions with meaningful impact.

Keywords

Virtual reality / Attention training / Autistic spectrum disorders / Human-computer interaction

CRediT authorship contribution statement

Rehma Razzak: Formal analysis, Investigation, Data curation, Writing - original draft. Yi (Joy) Li: Methodology, Software, Validation, Investigation, Writing - review & editing, Supervision. Jing (Selena) He: Validation, Formal analysis, Data curation, Writ- ing - review & editing, Supervision. Sungchul Jung: Writing - review & editing, Supervision. Chao Mei: Conceptualization, Visualization, Funding acquisition. Yan Huang: Writing - review & editing, Supervision.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was partially supported by the US National Science Foundation (2244221 and 2315595).

References

[1] B. Keehn, R. Müller, J. Townsend, Atypical attentional networks and the emergence of autism, Neurosci. Biobehav. Rev. 37 (2013) 164-183.

[2] B. Keehn, M. Westerfield, R. Müller, J. Townsend, Autism, attention, and alpha oscillations: An electrophysiological study of attentional capture, Biol. Psychiatry: Cogn. Neurosci. Neuroimaging 2 (2017) 528-536.

[3] S. Bryson, N. Garon, T. McMullen, J. Brian, L. Zwaigenbaum, V. Armstrong, W. Roberts, I. Smith, P. Szatmari, Impaired disengagement of attention and its relationship to emotional distress in infants at high-risk for autism spectrum disorder, J. Clin. Exp. Neuropsychol. 40 (2018) 487-501.

[4] M. Muller Spaniol, L. Shalev, C. Mevorach, Reduced distractor interference in neurotypical adults with high expression of autistic traits irrespective of stimulus type, Autism Res. 11 (2018) 1345-1355.

[5] Y. Morimoto, A. Imamura, N. Yamamoto, S. Kanegae, H. Ozawa, R. Iwanaga, Atypical sensory characteristics in autism spectrum disorders, Autism Spectrum Disord.[Internet] (2021).

[6] C. Mallory, B. Keehn, Implications of sensory processing and attentional differences associated with autism in academic settings: An integrative review, Front. Psychiatry 12 (2021) 695825.

[7] A. Fridhi, F. Benzarti, A. Frihida, H. Amiri, Application of virtual reality and augmented reality in psychiatry and neuropsychology, in particular in the case of autistic spectrum disorder (ASD), Neurophysiology 50 (2018) 222-228.

[8] L. Escobedo, D. Nguyen, L. Boyd, S. Hirano, A. Rangel, D. Garcia-Rosas, M. Tentori, G. Hayes, MOSOCO: A mobile assistive tool to support children with autism practicing social skills in real-life situations,in:Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2012, pp. 2589-2598.

[9] R. Joseph, B. Keehn, C. Connolly, J. Wolfe, T. Horowitz, Why is visual search superior in autism spectrum disorder? Dev. Sci. 12 (2009) 1083-1096.

[10] H. Meadan, M. Ostrosky, B. Triplett, A. Michna, A. Fettig, Using visual supports with young children with autism spectrum disorder, Teach. Except. Child. 43 (2011) 28-35.

[11] A. Vahabzadeh, N. Keshav, R. Abdus-Sabur, K. Huey, R. Liu, N. Sahin, Improved socio-emotional and behavioral functioning in students with autism following school-based smartglasses intervention: Multi-stage feasibility and controlled efficacy study, Behav. Sci. 8 (2018) 85.

[12] L. Escobedo, M. Tentori, E. Quintana, J. Favela, D. Garcia-Rosas, Using augmented reality to help children with autism stay focused, IEEE Pervasive Comput. 13 (2014) 38-46.

[13] M. Spaniol, C. Mevorach, L. Shalev, M. Teixeira, R. Lowenthal, C. Paula, Attention training in children with autism spectrum disorder improves academic performance: A double-blind pilot application of the computerized progressive attentional training program, Autism Res. 14 (2021) 1769-1776.

[14] M. Spaniol, L. Shalev, L. Kossyvaki, C. Mevorach, Attention training in autism as a potential approach to improving academic performance: A school-based pilot study, J. Autism Dev. Disord. 48 (2018) 592-610.

[15] M. Spaniol, Attentional atypicalities in autism spectrum disorder and the broader autism phenotype, Cadernos Pós-Graduação Em Distúrbios Do Desenvolvimento 18 (2018).

[16] E. McDougal, D. Riby, M. Hanley, Profiles of academic achievement and attention in children with and without autism spectrum disorder, Res. Dev. Disabil. 106 (2020) 103749.

[17] B. Banire, D. Thani, M. Qaraqe, B. Mansoor,A systematic review: Attention assessment of virtual reality based intervention for learning in children with autism spectrum disorder, in: 2017 7th IEEE International Conference on Control System, Computing and Engineering, (ICCSCE), 2017, pp. 97-103.

[18] H. Yazdanian, A. Vakili, A. Soltani, Z. Bagheri Rekhne, S. Zareii, T. Zarifian, Virtual/augmented reality for joint attention skills improvement in autism spectrum disorder: A systematic review, Int. J. Dev. Disabil. (2023) 1-16.

[19] E. Özkan, S. Belhan Çelik, M. Yaran, G. Bumin, Joint attention-based occupational therapy intervention in preschoolers with autism spectrum disorder: A randomized controlled trial, Am. J. Occup. Therapy 77 (2023).

[20] P. Satu, L. Minna, S. Satu, Immersive VR assessment and intervention research of individuals with neurodevelopmental conditions is dominated by ASD and ADHD: A scoping review, Rev. J. Autism Dev. Disord. (2023) 1-19.

[21] K. Wang, S. Julier, Y. Cho, Attention-based applications in extended reality to support ASD users: A systematic review, IEEE Access 10 (2022) 15574-15593.

[22] L. Adamson, R. Bakeman, K. Suma, D. Robins, An expanded view of joint attention: Skill, engagement, and language in typical development and autism, Child Dev. 90 (2019) e1-e18.

[23] L. Ronconi, M. Devita, M. Molteni, S. Gori, A. Facoetti, Brief report: When large becomes slow: Zooming-out visual attention is associated to orienting deficits in autism, J. Autism Dev. Disord. 48 (2018) 2577-2584.

[24] A. Richard, R. Lajiness-O’Neill, Visual attention shifting in autism spectrum disorders, J. Clin. Exp. Neuropsychol. 37 (2015) 671-687.

[25] T. Alloway, A. Lepere, Sustained attention and working memory in children with autism spectrum disorder, Int. J. Disabil., Dev. Educ. 68 (2019) 1-9.

[26] Y. Chien, S. Gau, Y. Chiu, W. Tsai, C. Shang, Y. Wu, Impaired sustained attention, focused attention, and vigilance in youths with autistic disorder and Asperger’s disorder, Res. Autism Spectrum Disord. 8 (2014) 881-889.

[27] B. Yerys, B. Tunç, T. Satterthwaite, L. Antezana, M. Mosner, J. Bertollo, L. Guy, R. Schultz, J. Herrington, Functional connectivity of frontoparietal and salience/ventral attention networks have independent associations with co-occurring attention-deficit/hyperactivity disorder symptoms in children with autism, Biol. Psychiatry: Cogn. Neurosci. Neuroimaging 4 (2019) 343-351.

[28] Y. Kado, S. Sanada, S. Oono, T. Ogino, S. Nouno, Children with autism spectrum disorder comorbid with attention-deficit/hyperactivity disorder examined by the wisconsin card sorting test: Analysis by age-related differences, Brain Dev. 42 (2020) 113-120.

[29] S. Boxhoorn, E. Lopez, C. Schmidt, D. Schulze, S. Hänig, C. Freitag, Attention profiles in autism spectrum disorder and subtypes of attention-deficit/hyperactivity disorder, Eur. Child Adolesc. Psychiatry 27 (2018) 1433-1447, http://dx.doi.org/10.1007/s00787-018-1138-8.

[30] C. Murphy, A. Christakou, E. Daly, C. Ecker, V. Giampietro, M. Brammer, A. Smith, P. Johnston, D. Robertson, M. Consortium, et al., Abnormal functional activation and maturation of fronto-striato-temporal and cerebellar regions during sustained attention in autism spectrum disorder, Am. J. Psychiatry 171 (2014) 1107-1116.

[31] M. Granovetter, C. Burlingham, N. Blauch, N. Minshew, D. Heeger, M. Behrmann, Uncharacteristic task-evoked pupillary responses implicate atypical locus ceruleus activity in autism, J. Neurosci. 40 (2020) 3815-3826.

[32] K. Wang, B. Zhang, Y. Cho, Using mobile augmented reality to improve attention in adults with autism spectrum disorder, in:Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1-9.

[33] J. Zhao, X. Zhang, Y. Lu, X. Wu, F. Zhou, S. Yang, L. Wang, X. Wu, F. Fei, Virtual reality technology enhances the cognitive and social communication of children with autism spectrum disorder, Front. Public Health 10 (2022) 1029392.

[34] B. Cho, J. Lee, J. Ku, D. Jang, J. Kim, I. Kim, J. Lee, S. Kim, Attention enhancement system using virtual reality and EEG biofeedback, Proc. IEEE Virtual Real. 2002 (2002) 156-163.

[35] Y. Tan, D. Zhu, H. Gao, T. Lin, H. Wu, S. Yeh, T. Hsu,Virtual classroom: An ADHD assessment and diagnosis system based on virtual reality, in: 2019 IEEE International Conference on Industrial Cyber Physical Systems, ( ICPS), 2019, pp. 203-208.

[36] M. Simões, P. Carvalho, M. Castelo-Branco, Virtual reality and braincomputer interface for joint-attention training in autism, in: Intl Conf. Disability, Virtual Reality & Associated Technologies, Laval, France (10-12 Sept. 2012), 2012.

[37] V. Jyoti, U. Lahiri, Virtual reality based joint attention task platform for children with autism, IEEE Trans. Learn. Technol. 13 (2020) 198-210.

[38] S. Vidhusha, B. Divya, A. Kavitha, R. Viswath Narayanan, D. Yaamini,Cognitive attention in autism using virtual reality learning tool, in: 2019 IEEE 18th International Conference on Cognitive Informatics & Cognitive Computing, ( ICCI*CC), 2019, pp. 159-165.

[39] A. Amat, H. Zhao, A. Swanson, A. Weitlauf, Z. Warren, N. Sarkar, Design of an interactive virtual reality system, invirs, for joint attention practice in ASD children, IEEE Trans. Neural Syst. Rehabil. Eng. 29 (2021) 1866-1876.

[40] S. Sarker, A. Md Linkon, F. Bappy, M. Rabbi, M. Nahid, Improving joint attention in children with autism: A VR-AR enabled game approach, Int. J. Eng. Adv. Technol. 10 (3) (2021).

[41] J. Zhao, X. Zhang, C. Wang, J. Yang, Effect of cognitive training based on virtual reality on the children with autism spectrum disorder, Curr. Res. Behav. Sci. 2 (2021) 100013.

[42] C. Mei, R. Guo, Enable an innovative prolonged exposure therapy of attention deficits on autism spectrum through adaptive virtual environments, in:2018 10th International Conference on Virtual Worlds and Games for Serious Applications, (VS-Games), 2018, pp. 1-4.

[43] K. Lu, K. Yueh, H. Hu, M. Guo, Y. Liu,A novel neurofeedback attentional enhancement approach based on virtual reality, in: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, ( EMBC), 2022, pp. 5140-5143.

[44] C. Ji, J. Yang, Effects of physical exercise and virtual training on visual attention levels in children with autism spectrum disorders, Brain Sci. 12 (2021) 41.

[45] S. Lee, Virtual speech-language therapy for individuals with communication disorders: Current evidence, limitations, and benefits, Curr. Dev. Disord. Rep. 6 (2019) 119-125.

[46] M. Reitman, Effectiveness of Music Therapy Interventions on Joint Attention in Children Diagnosed with Autism: A Pilot Study, Carlos Albizu University, 2005.

[47] A. LaGasse, R. Manning, J. Crasta, W. Gavin, P. Davies, Assessing the impact of music therapy on sensory gating and attention in children with autism: A pilot and feasibility study, J. Music Ther. 56 (2019) 287-314.

[48] J. Kim, T. Wigram, C. Gold, The effects of improvisational music therapy on joint attention behaviors in autistic children: A randomized controlled study, J. Autism Dev. Disord. 38 (2008) 1758-1766.

[49] X. Liu, Q. Wu, W. Zhao, X. Luo, Technology-facilitated diagnosis and treatment of individuals with autism spectrum disorder: An engineering perspective, Appl. Sci. 7 (2017) 1051.

[50] S. Rezayi, M. Tehrani-Doost, L. Shahmoradi, Features and effects of computer-based games on cognitive impairments in children with autism spectrum disorder: an evidence-based systematic literature review, BMC Psychiatry 23 (2023) 1-32.

[51] A. Ryan, N. Files, The efficacy of neurofeedback and traditional therapies with autism spectrum disorder in a clinic setting, Asia Pac. J. Neurother. 1 (2019) 14-20.

[52] M. Tabrizi, G. Manshaee, A. Ghamarani, J. Rasti, Comparison of the effec- tiveness of virtual reality with medication on the memory of attention deficit hyperactivity disorder students, Int. Arch. Health Sci. 7 (2020) 37-42.

[53] A. Ruttledge, Attention autism^TM training for occupational therapists: A pilot study, Irish J. Occup. Ther. (2024).

[54] V. Pandey, A Training Workshop for Guiding Occupational Therapy Practice:Use of Environments in Virtual Reality for Social Skills Training in Autism, Boston University, 2020.

[55] S. Dominguez-Lucio, L. Compan-Gabucio, L. Torres-Collado, M. Hera, Occupational therapy interventions using new technologies in children and adolescents with autism spectrum disorder: A scoping review, J. Autism Dev. Disord. 53 (2023) 332-358.

[56] B. Kadir, M. Sahid, M. Noviana, P. Denny, Effectiveness of virtual reality (VR) in improving social skills in autism spectrum disorder cases: A literature review,in:The 6th International Conference on Vocational Education Applied Science and Technology, (ICVEAST 2023), 2023, pp. 45-71.

[57] N. Josman, H. Ben-Chaim, S. Friedrich, P. Weiss, Effectiveness of virtual reality for teaching street-crossing skills to children and adolescents with autism, Int. J. Disabil. Hum. Dev. 7 (2008) 49-56.

[58] L. Lamash, N. Josman, A metacognitive intervention model to promote independence among individuals with autism spectrum disorder: Implementation on a shopping task in the community, Neuropsychol. Rehabil. 31 (2021) 189-210.

[59] S. Parsons, et al., The use and effects of virtual reality within occupational therapy intervention for children and Young people with autism spectrum disorder: A mixed methods study, Disabil. Rehabil. 39 (2017) 1579-1586.

[60] L. Shahmoradi, S. Rezayi, Cognitive rehabilitation in people with autism spectrum disorder: A systematic review of emerging virtual reality-based approaches, J. NeuroEng. Rehabil. 19 (2022) 91.

[61] P. Mesa-Gresa, H. Gil-Gómez, J. Lozano-Quilis, J. Gil-Gómez, Effectiveness of virtual reality for children and adolescents with autism spectrum disorder: an evidence-based systematic review, Sensors 18 (2018) 2486.

[62] B. Sanku, Y. Li, S. Jung, C. Mei, S. He, Enhancing attention in autism spectrum disorder: Comparative analysis of virtual reality-based training programs using physiological data, Front. Comput. Sci. 5, 1250652.

[63] M. Patriquin, E. Hartwig, B. Friedman, S. Porges, A. Scarpa, Autonomic response in autism spectrum disorder: Relationship to social and cognitive functioning, Biol. Psychol. 145 (2019) 185-197.