Warning: Artificial intelligence chatbots can generate inaccurate medical and scientific information and references

Abstract

The use of generative artificial intelligence (AI) chatbots, such as ChatGPT and YouChat, has increased enormously since their release in late 2022. Concerns have been raised over the potential of chatbots to facilitate cheating in education settings, including essay writing and exams. In addition, multiple publishers have updated their editorial policies to prohibit chatbot authorship on publications. This article highlights another potentially concerning issue; the strong propensity of chatbots in response to queries requesting medical and scientific information and its underlying references, to generate plausible looking but inaccurate responses, with the chatbots also generating nonexistent citations. As an example, a number of queries were generated and, using two popular chatbots, demonstrated that both generated inaccurate outputs. The authors thus urge extreme caution, because unwitting application of inconsistent and potentially inaccurate medical information could have adverse outcomes.

Keywords

ChatGPT / YouChat / artificial intelligence / inaccurate information

References

[1] Hu K. ChatGPT sets record for fastest-growing user base - analyst note [Internet].Reuters; 2023 [cited 2023 Feb 2]. Available from:https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/

[2] ChatGPT passes MBA exam given by a Wharton professor [Internet].NBC UNIVERSAL; c2023 [cited 2023 Jan 23]. Available from:https://www.nbcnews.com/tech/tech-news/chatgpt-passes-mba-exam-wharton-professor-rcna67036

[3] Varanasi L. GPT-4 can ace the bar, but it only has a decent chance of passing the CFA exams. Here’s a list of difficult exams the ChatGPT and GPT-4 have passed. [Internet].Insider Inc; 2023 [cited 2023 Feb 19]. Available from:https://www.businessinsider.com/list-here-are-the-exams-chatgpt-has-passed-so-far-2023-1?op=1#us-medical-licensing-exam-2

[4] Thorp HH. ChatGPT is fun, but not an author. Science. 2023; 379:313.

[5] Owens B. How Nature readers are using ChatGPT. Nature. 2023; 615:20.

[6] Mastrangelo M, Cesario S. Update on the treatment of vitamin B6 dependent epilepsies. Expert Rev Neurother. 2019; 19:1135-47.

[7] Stockler S, Plecko B, Gospe SM Jr, Coulter-Mackie M, Connolly M, van Karnebeek C, et al. Pyridoxine dependent epilepsy and antiquitin deficiency: clinical and molecular characteristics and recommendations for diagnosis, treatment and follow-up. Mol Genet Metab. 2011; 104:48-60.

[8] Kožich V, Sokolová J, Morris AAM, Pavlíková M, Gleich F, Kölker S, et al. ; E-HOD consortium. Cystathionine β-synthase deficiency in the E-HOD registry-part I: pyridoxine responsiveness as a determinant of biochemical and clinical phenotype at diagnosis. J Inherit Metab Dis. 2021; 44:677-92.

[9] Crabtree GW, Park AJ, Gordon JA, Gogos JA. Cytosolic accumulation of L-proline disrupts GABA-ergic transmission through GAD blockade. Cell Rep. 2016; 17:570-82.