Assessing AI-based Summarizing and Paraphrasing Tools for a CLIL Intercultural Communication Academic Writing Class

Abstract

This paper aimed to examine the benefits and drawbacks of AI-based summarizing and paraphrasing tools that were employed in a CLIL academic writing course on intercultural communication at a national Japanese university. Students summarized an internet article of their choice that had to include indirect citations by paraphrasing the author’s ideas. First, they did this without the support of AI-based tools. On their second and third attempt, they used the AI-based summarizing and paraphrasing tools Quillbot and SpinBot respectively. A one-way ANOVA was conducted to evaluate the three summaries for grammar, readability and plagiarism. Results showed that although the AI-based tools can improve grammar, they have limited capability in improving readability and preventing plagiarism. A survey on student perceptions of the tools also highlighted the drawbacks of the software and that dependence on such tools to produce academic writing content needs to be treated with caution.

Keywords

AI-based tools / summarizing and paraphrasing / academic writing / plagiarism

References

[1] Bailey, Daniel; Southam, Ashleigh, & Costley, Jamie. (2021). Digital storytelling with chatbots: mapping L2 participation and perception patterns. Interactive Technology and Smart Education, 18(1), 85-103.

[2] Bailin, Alan. (1987). Artificial intelligence and computer—assisted language instruction: A perspective. CALICO Journal, 5(3), 25-45.

[3] Basilaia, Giorgi, & Kvavadze, David. (2020). Transition to online education in schools during a SARS—CoV—2 Coronavirus (COVID—19) pandemic in Georgia. Pedagogical Research, 5(4), em0060.

[4] Burleson, Winslow, & Lewis, Armanda. (2016). Optimists’ creed: Brave new cyberlearning, evolving utopias (Circa 2041). International Journal of Artificial Intelligence in Education, 26(2), 796-80.

[5] Coyle, Do.; Hood, Philip, & Marsh, David. (2010). Content and language integrated learning. Cambridge: CUP.

[6] Educause. (2018). Horizon report: 2018 higher education edition. Retrieved from EDUCAUSE Learning Initiative and The New Media Consortium website: https://library.educause.edu/~/media/files/library/2018/8/2018horizonreport.pdf

[7] El Shazly, Reham. (2021). Effects of artificial intelligence on English speaking anxiety and speaking performance: A case study. Expert Systems, 38(3), e12667.

[8] Gallacher, Andrew; Thompson, Andrew, & Howarth, Mark. (2018). “My robot is an idiot!” — Students’ perceptions of AI in the L2 classroom. In P. Taalas, J. Jalkanen, L. Bradley & S. Thouësny (Eds), Future—proof CALL: Language learning as exploration and encounters — short papers from EUROCALL 2018 (pp. 70-76).

[9] Istenic, Andreja; Bratko, Ivan, & Rosanda, Violeta. (2021). Pre—service teachers’ concerns about social robots in the classroom: A model for development. Education and Self Development, 16, 60-87.

[10] Kaplan, Andreas, & Haenlein, Michael. (2019). Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business Horizons, 62, 15-25.

[11] Kaban, Ayseguk Liman, & Ergul, Isil Boy. (2020). Teachers’ attitudes towards the use of tablets in six EFL classrooms. In P. Eva (ed.), Examining the Roles of Teachers and Students in Mastering New Technologies (pp. 284-298). Pennsylvania: IGI Global.

[12] Keck, Casey. (2006). The use of paraphrase in summary writing: A comparison of L1 and L2 writers. Journal of Second Language Writing 15(4), 261-278.

[13] Keck, Casey. (2014). Copying, paraphrasing, and academic writing development: A re—examination of L1 and L2 summarization practices. Journal of Second Language Writing, 25, 4-22.

[14] Matthews, Clive. (1993). Grammar frameworks in intelligent call. CALICO Journal, 11(1), 5-27.

[15] Mulenga, Eddie, & Marbán, Jose. (2020). Is COVID—19 the gateway for digital learning in mathematics education? Contemporary Educational Technology, 12(2), ep269, 1—11.

[16] Naciri, Aziz; Baba, Mohamed; Baba, Amine; Achbani, Achbani, & Kharbach, Ahmed. (2020). Mobile learning in higher education: Unavoidable alternative during COVID—19. Aquademia, 4(1), ep20016.

[17] Nilsson, Nils. J. (2011). The quest for artificial intelligence: A history of ideas and achievements. Cambridge University Press.

[18] Niño, Ana. (2009). Machine translation in foreign language learning: Language learners’ and tutors’ perceptions of its advantages and disadvantages. ReCALL, 21(02), 241-258.

[19] Ohmori, Ai. (2014). Exploring the potential of CLIL in English Language Teaching in Japanese universities: An innovation for the development of effective teaching and global awareness. The Journal of Rikkyo University Language Center Bulletin, 32, 39-51.

[20] Pathways to Academic English. (2021). The Center for Culture and Language Education English Section, Institute for Excellence in Higher Education, Tohoku University, Tohoku University Press.

[21] Prensky, Mark. (2007). How to teach with technology: Keeping both teachers and students comfortable in an era of exponential change. Emerging Technologies for Learning, 2, 40-46.

[22] Prensky, Mark. (2008). Backup education? Too many teachers see education as preparing kids for the past, not the future. Education Technology, 48, 1-3.

[23] Rogerson, Ann M., & McCarthy, Grace. (2017). Using internet based paraphrasing tools: Original work, patchwriting or facilitated plagiarism? International Journal for Educational Integrity, 13(2), 1-15.

[24] Salaberry, M. Rafael. (1996). A theoretical foundation for the development of pedagogical tasks in computer mediated communication. CALICO Journal, 14(1), 5-34.

[25] Shi, Ling. (2012). Rewriting and paraphrasing source texts in second language writing. Journal of Second Language Writing, 21, 134-148.

[26] Sintema, Edgar John. (2020). E—learning and smart revision portal for Zambian primary and secondary school learners: A digitalized virtual classroom in the COVID—19 era and beyond. Aquademia, 4(2), ep20017.

[27] Tafazoli, Dara; María, Elena Gomez, & Huetas Abrill, Cristina Aranzaza. (2019). Intelligent language tutoring system: Integrating intelligent computer—assisted language learning into language education. International Journal of Information and Communication Technology Education, 15(3), 60-74.

[28] Wartman, Steven. A., & Combs, Donald. (2018). Medical education must move from the information age to the age of artificial intelligence. Academic Medicine, 93(8), 1107-1109.

[29] Yin, Jiaqi; Goh, Tiong—Thye; Yang, Bing, & Yang, Xiaobin. (2021). Conversation technology with micro—learning: The impact of chatbot—based learning on students’ learning motivation and performance. Journal of Educational Computing Research, 59(1), 154-177.