Online assessment in young children: Challenges and considerations

Abstract

In the years following the COVID-19 pandemic, online assessment has emerged as an increasingly popular method for testing children. This review aims to review key advantages and disadvantages of online assessment in young children, specifically in the area of literacy and mathematics assessment. We first consider the recent history of online assessment and give evidence from our own experiences with online assessments in both Cebu, Philippines and Hong Kong. We then discuss the benefits of conducting assessments online. One of the key strengths of online assessments is the efficient use of resources, such as ease of staff training, increased participation and decreased costs for both participants and researchers. Difficulties related to testing environment, parental interference and exclusion of certain populations are also highlighted as focal challenges that must be considered when designing and implementing online assessment. The reliability and validity of online assessments are then specifically discussed in relation to the question of how to best ensure that online measures function as intended, stressing the importance of establishing separate norms for the various administration modes. Notably, online assessments delivered via phone screen must have separate norms than those delivered on a computer screen, as device differences may account for differences in performance. Finally, we present broader considerations for designing online assessments with young children. This includes practical tips for pilot testing, user interface design, data confidentiality and automatic scoring, both to enhance the efficiency of future online assessments and increase participation. Specifically, online assessments for children must account for their developing attentional abilities, meaning assessments should be brief, engaging and contain clear task instructions. The implementation of these considerations will support the development of more reliable, valid and inclusive assessments which will allow researchers and practitioners to support equitable and feasible online testing moving forward.

Keywords

assessment design / digital assessment / online assessment

References

[1] Aishath, W., Fathimath, M., Aishath, S., & Fathimath, S. (2023). Benefits of online assessments in higher education institutions: Lessons from COVID-19 pandemic. Journal of Business and Social Sciences, 2023(3), 1–18.

[2] Asprilia, M. T., Qodariah, L., & Purba, F. D. (2020). First grader’s attention span during in-class activity. Jurnal Ilmu Pendidikan, Psikologi, Bimbingan dan Konseling, 10(2), 144–150.

[3] Bakar, A., & Cagiltay, K. (2006). Evaluation of children’s web sites: What do they prefer? (pp. 569–574). EdMedia+ Innovate Learning.

[4] Beckman, M. E., Plummer, A. R., Munson, B., & Reidy, P. F. (2017). Methods for eliciting, annotating, and analyzing databases for child speech development. Computer Speech & Language, 45, 278–299.

[5] Bennett, P. (2023). Pandemic fallout: Learning loss, collateral damage, and recovery in Canada’s schools. Cardus. https://www.cardus.ca/wp-content/uploads/2023/11/Pandemic-Fallout-Learning-Loss-Collateral-Damage-and-Recovery-in-Canadas-Schools.pdf

[6] Boksem, M. A., Meijman, T. F., & Lorist, M. M. (2005). Effects of mental fatigue on attention: An ERP study. Cognitive Brain Research, 25(1), 107–116.

[7] Cheah, Z. R. E., McBride, C., Dulay, K. M., Cheung, S. K., Reyes, P., & Dillenbourg, P. (2024). Effects of online parent coaching on Filipino children’s skills. Early Childhood Research Quarterly, 68(3), 191–202.

[8] Chen, L. K., Dorn, E., Sarakatsannis, J., & Wiesinger, A. (2021). Teacher survey: Learning loss is global—and significant. Public & social sector practice (pp. 1999–2003). McKinsey & Company.

[9] Corbillé S., Fromont, E., Anquetil, E., & Nerdeux, P. (2020). Integrating writing dynamics in CNN for online children handwriting recognition. In J, Guerrero (Ed.), 2020 17th International Conference on Frontiers in Handwriting Recognition (ICFHR) (pp. 270–275). IEEE.

[10] Cubilla-Bonnetier, D., Grajales-Barrios, M., Ortega-Espinosa, A., Puertas, L., & De León Sautú N. (2023). Unequal literacy development and access to online education in public versus private Panamanian schools during COVID-19 pandemic. In M. A., Queiruga-Dios (Ed.), Frontiers in education (Vol. 8, 989872). Frontiers Media SA.

[11] Dong, C., Cao, S., & Li, H. (2020). Young children’s online learning during COVID-19 pandemic: Chinese parents’ beliefs and attitudes. Children and Youth Services Review, 118, 105440.

[12] Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2021). COVID-19 and education: The lingering effects of unfinished learning. McKinsey & Company.

[13] Dotan, S., Katzir, T., Lipka, O., & Shaul, S. (2021). Elementary school literacy teachers’ perceptions and challenges during COVID-19. Academia Letters, 898, 1–7.

[14] Dulay, K. M., Cheung, S. K., Reyes, P., & McBride, C. (2019). Effects of parent coaching on Filipino children’s numeracy, language, and literacy skills. Journal of Educational Psychology, 111(4), 641–662.

[15] Dykman, C. A., & Davis, C. K. (2008). Part one — The shift toward online education. Journal of Information Systems Education, 19(1), 11–16.

[16] Early, J., & Hernandez, A. (2021). Digital disenfranchisement and COVID-19: Broadband internet access as a social determinant of health. Health Promotion Practice, 22(5), 605–610.

[17] Engzell, P., Frey, A., & Verhagen, M. D. (2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences, 118(17), e2022376118.

[18] Fitzner, K. (2007). Reliability and validity a quick review. The Diabetes Educator, 33(5), 775–780.

[19] Gossen, T., Hempel, J., & Nürnberger, A. (2013). Find it if you can: Usability case study of search engines for young users. Personal and Ubiquitous Computing, 17, 1593–1603.

[20] Hautala, J., Heikkilä R., Nieminen, L., Rantanen, V., Latvala, J. M., & Richardson, U. (2020). Identification of reading difficulties by a digital game-based assessment technology. Journal of Educational Computing Research, 58(5), 1003–1028.

[21] Ho, J. C., McBride, C., & Lui, K. F. H. (2024). What explains children’s digital word reading performance in L2. Reading and Writing, 37(7), 1827–1848.

[22] Ho, J. C., McBride, C., Lui, K. F. H., & Łockiewicz, M. (2024). WordSword: An efficient online word reading assessment for global English. Assessment, 31(4), 875–891.

[23] Hodge, M. A., Sutherland, R., Jeng, K., Bale, G., Batta, P., Cambridge, A., Detheridge, J., Drevensek, S., Edwards, L., Everett, M., Ganesalingam, K., Geier, P., Kass, C., Mathieson, S., McCabe, M., Micallef, K., Molomby, K., Ong, N., Pfeiffer, S., … Silove, N. (2019). Agreement between telehealth and face-to-face assessment of intellectual ability in children with specific learning disorder. Journal of Telemedicine and Telecare, 25(7), 431–437.

[24] Joshi, A., Vinay, M., & Bhaskar, P. (2021). Impact of coronavirus pandemic on the Indian education sector: Perspectives of teachers on online teaching and assessments. Interactive Technology and Smart Education, 18(2), 205–226.

[25] Kabali, H. K., Irigoyen, M. M., Nunez-Davis, R., Budacki, J. G., Mohanty, S. H., Leister, K. P., & Bonner, R. L., Jr. (2015). Exposure and use of mobile media devices by young children. Pediatrics, 136(6), 1044–1050.

[26] Khaleghi, A., Aghaei, Z., & Mahdavi, M. A. (2021). A gamification framework for cognitive assessment and cognitive training: Qualitative study. JMIR Serious Games, 9(2), e21900.

[27] Kiefer, M., & Velay, J. L. (2016). Writing in the digital age. Trends in Neuroscience and Education, 5(3), 77–81.

[28] Lee, S. A. S., Hall, B., & Sancibrian, S. (2017). Feasibility of a supplemental phonological awareness intervention via telepractice for children with hearing loss: A preliminary study. International Journal of Telerehabilitation, 9(1), 23–38.

[29] Lee, Y.-H., Hsiao, C., & Ho, C.-H. (2014). The effects of various multimedia instructional materials on students’ learning responses and outcomes: A comparative experimental study. Computers in Human Behavior, 40, 119–132.

[30] Leech, N. L., Gullett, S., Howland Cummings, M., & Haug, C. (2020). Challenges of remote teaching for K-12 teachers during COVID-19. Journal of Educational Leadership in Action, 7(1), 1.

[31] Lenhard, W., Schroeders, U., & Lenhard, A. (2017). Equivalence of screen versus print reading comprehension depends on task complexity and proficiency. Discourse Processes, 54(5–6), 427–445.

[32] Ling, G., Attali, Y., Finn, B., & Stone, E. A. (2017). Is a computerized adaptive test more motivating than a fixed-item test? Applied Psychological Measurement, 41(7), 495–511.

[33] McLaughlin, T., & Yan, Z. (2017). Diverse delivery methods and strong psychological benefits: A review of online formative assessment. Journal of Computer Assisted Learning, 33(6), 562–574.

[34] Middleton, F. (2023, June 22). The 4 types of validity in research | definitions & examples. Scribbr. https://www.scribbr.com/methodology/types-of-validity/

[35] Mohammed, S., Flores, L., Deveau, J., Cohen Hoffing, R., Phung, C. M., Parlett, C., Sheehan, E., Lee, D., Au, J., & Buschkuehl, M. (2017). The benefits and challenges of implementing motivational features to boost cognitive training outcome. Journal of Cognitive Enhancement, 1, 491–507.

[36] Muirhead, W. D. (2000). Online education in schools. International Journal of Educational Management, 14(7), 315–324.

[37] Muravyeva, E., Janssen, J., Dirkx, K., & Specht, M. (2019). Students’ attitudes towards personal data sharing in the context of e-assessment: Informed consent or privacy paradox? In S. Draaijer, D. Joosten-ten Brinke., & E. Ras (Eds.), Technology Enhanced Assessment:21st International Conference, TEA 2018, Amsterdam, The Netherlands, December 10–11, 2018, Revised Selected Papers 21 (pp. 16–26). Springer International Publishing.

[38] Ngan, V. S. H., Mo, J., Wong, O. K. P., Ng, M. C. Y., Ho, J. C.-S., Inoue, T., McBride, C., & Maurer, U. (2024). Creating online cognitive assessment platform for children’s literacy and math skills: Illustrated with the ACE Kids Platform. Department of Psychology, the Chinese University of Hong Kong. Manuscript submitted for publication.

[39] Paleczek, L., Seifert, S., & Schöfl, M. (2021). Comparing digital to print assessment of receptive vocabulary with GraWo-KiGa in Austrian kindergarten. British Journal of Educational Technology, 52(6), 2145–2161.

[40] Sanchez, C. A., & Wiley, J. (2009). To scroll or not to scroll: Scrolling, working memory capacity, and comprehending complex texts. Human Factors, 51(5), 730–738.

[41] Sari, T., & Nayır, F. (2020). Challenges in distance education during the (Covid-19) pandemic period. Qualitative Research in Education, 9(3), 328–360.

[42] Schrank, F. A., Mather, N., & McGrew, K. S. (2014). Woodcock-Johnson IV tests of achievement. Riverside.

[43] Scott, K., & Schulz, L. (2017). Lookit (part 1): A new online platform for developmental research. Open Mind, 1(1), 4–14.

[44] Shamir, H. (2018). Assessing reading in young learners: Using a computerized adaptive Reading test for pre-kindergarten through 2nd grade. Journal of Educational Multimedia and Hypermedia, 27(4), 507–527.

[45] Simon, A. J., Gallen, C. L., Ziegler, D. A., Mishra, J., Marco, E. J., Anguera, J. A., & Gazzaley, A. (2023). Quantifying attention span across the lifespan. Frontiers in Cognition, 2, 1–11.

[46] Sobers, S. M., Whitehead, H. L., N’Goh, K. N. A., Ball, M. C., Tanoh, F., Akpé H., & Jasińska, K. K. (2023). Is a phone-based language and literacy assessment a reliable and valid measure of children’s reading skills in low-resource settings? Reading Research Quarterly, 58(4), 733–754.

[47] Trust, T., & Whalen, J. (2020). Should teachers be trained in emergency remote teaching? Lessons learned from the COVID-19 pandemic. Journal of Technology and Teacher Education, 28(2), 189–199.

[48] Twenge, J. M., Martin, G. N., & Spitzberg, B. H. (2019). Trends in US adolescents’ media use, 1976–2016: The rise of digital media, the decline of TV, and the (near) demise of print. Psychology of Popular Media Culture, 8(4), 329–345.

[49] Typing.com. (2022, August 29). At what age are kids developmentally ready for typing? Typing blog. Retrieved February 7, 2024, from https://www.typing.com/blog/age-kids-developmentally-reading-typing/

[50] UNESCO. (2021). Education: From disruption to recovery. UNESCO. https://en.unesco.org/covid19/educationresponse

[51] UNESCO, UNICEF, World Bank, & OECD. (2021). What’s next? Lessons on education recovery: Findings from a survey of ministries of education amid the COVID-19 pandemic. OECD.

[52] UNICEF. (2020, November 30). Two thirds of the world’s school-age children have no internet access at home, new UNICEF-ITU report says. [Press release]. https://www.unicef.org/press-releases/two-thirds-worlds-school-age-children-have-no-internet-access-home-new-unicef-itu

[53] Waite, M. C., Theodoros, D. G., Russell, T. G., & Cahill, L. M. (2010). Assessment of children’s literacy via an Internet-based telehealth system. Telemedicine and e-Health, 16(5), 564–575.

[54] Waters, L. H., Barbour, M. K., & Menchaca, M. P. (2014). The nature of online charter schools: Evolution and emerging concerns. Journal of Educational Technology & Society, 17(4), 379–389.

[55] Wilson, L. (2024, January 4). How big is a house? Average house size by country – 2024. Shrink That Footprint. https://shrinkthatfootprint.com/how-big-is-a-house/

[56] Wright, A. J. (2020). Equivalence of remote, digital administration and traditional, in-person administration of the Wechsler Intelligence Scale for Children, (WISC-V). Psychological Assessment, 32(9), 809–817.

[57] Wu, Z., Lin, T., & Li, M. (2019). Automated detection of children at risk of Chinese handwriting difficulties using handwriting process information: An exploratory study. IEICE Transactions on Information and Systems, 102(1), 147–155.

[58] Yeatman, J. D., Tang, K. A., Donnelly, P. M., Yablonski, M., Ramamurthy, M., Karipidis, I. I., Caffarra, S., Takada, M. E., Kanopka, K., Ben-Shachar, M., & Domingue, B. W. (2021). Rapid online assessment of reading ability. Scientific Reports, 11(1), 6396.

RIGHTS & PERMISSIONS

2024 The Author(s). PsyCh Journal published by Institute of Psychology, Chinese Academy of Sciences and John Wiley & Sons Australia, Ltd.