Evaluating the efficacy of major language models in providing guidance for hand trauma nerve laceration patients: a case study on Google’s AI BARD, Bing AI, and ChatGPT

Abstract

This study evaluated three prominent Large Language Models (LLMs)-Google’s AI BARD, Bing’s AI, and ChatGPT-4 in providing patient advice for hand laceration. Five simulated patient inquiries on hand trauma were prompted to them. A panel of Board-certified plastic surgical residents evaluated the responses for accuracy, comprehensiveness, and appropriate sources. Responses were also compared against existing literature and guidelines. This study suggests that ChatGPT outperforms BARD and Bing AI in providing reliable, evidence-based clinical advice, but they still face limitations in depth and specificity. Healthcare professionals are essential in interpreting LLM recommendations, and future research should improve LLM performance by integrating specialized databases and human expertise to advance nerve injury management and optimize patient-centred care.

Keywords

Artificial intelligence / ChatGPT / BARD / Bings AI / large language model / nerve injury

References

[1] Xie Y,Hunter-Smith DJ,Ross R.Aesthetic surgery advice and counseling from artificial intelligence: a rhinoplasty consultation with ChatGPT.Aesthetic Plast Surg2023:online ahead of print

[2] Cox A,Xie Y,Rozen WM.Utilizing ChatGPT-4 for providing medical information on blepharoplasties to patients.Aesthet Surg J2023;43:NP658-62

[3] Menorca RM,Elfar JC.Nerve physiology: mechanisms of injury and recovery.Hand Clin2013;29:317-30

[4] Ducic I,Iorio ML.Innovative treatment of peripheral nerve injuries: combined reconstructive concepts.Ann Plast Surg2012;68:180-7

[5] Mhlanga D. Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. SSRN Journal 2023:online ahead of print.Available from: https://www.researchgate.net/profile/David-Mhlanga-2/publication/368476294_Open_AI_in_Education_the_Responsible_and_Ethical_Use_of_ChatGPT_Towards_Lifelong_Learning/links/63eb1c91bd7860764366f597/Open-AI-in-Education-the-Responsible-and-Ethical-Use-of-ChatGPT-Towards-Lifelong-Learning.pdf [Last accessed on 30 Aug 2023]

[6] Newman RK.Laceration. Treasure Island (FL): StatPearls Publishing; 2022.

[7] Seddon HJ.A classification of nerve injuries.Br Med J1942;2:237-9.

[8] Bhandari PS.Management of peripheral nerve injury.J Clin Orthop Trauma2019;10:862-6 PMCID:PMC6739245

[9] Griffin MF,Hindocha S.Peripheral nerve injury: principles for repair and regeneration.Open Orthop J2014;8:199-203

[10] Campbell WW.Evaluation and management of peripheral nerve injury.Clin Neurophysiol2008;119:1951-65

[11] Dębski T,Noszczyk BH.Long-term sensibility outcomes of secondary digital nerve reconstruction with sural nerve autografts: a retrospective study.Eur J Trauma Emerg Surg2022;48:2341-7 PMCID:PMC9192371

[12] Dunlop RLE,Jain A.Outcome of surgical repair of adult digital nerve injury: a systematic review.BMJ open2019;9:e025443 PMCID:PMC6429897

[13] Jain A,Hems T.Outcomes of surgical repair of a single digital nerve in adults.J Hand Surg Eur Vol2019;44:560-5

[14] Höke A.A (heat) shock to the system promotes peripheral nerve regeneration.J Clin Invest2011;121:4231-4 PMCID:PMC3204849

[15] Sulaiman W.Neurobiology of peripheral nerve injury, regeneration, and functional recovery: from bench top research to bedside application.Ochsner J2013;13:100-8 PMCID:PMC3603172

[16] Cheng AS,Wong JM,Chan J.A prospective study of early tactile stimulation after digital nerve repair.Clin Orthop Relat Res2001;169-75.

[17] al-Ghazal SK,Khan K.Results of clinical assessment after primary digital nerve repair.J Hand Surg Br1994;19:255-7

[18] Rinker B.A prospective randomized study comparing woven polyglycolic acid and autogenous vein conduits for reconstruction of digital nerve gaps.J Hand Surg Am2011;36:775-81

[19] Cheung K,Thoma A.Approach to traumatic hand injuries for primary care physicians.Can Fam Physician2013;59:6148. PMCID:PMC3681444

[20] Risitano G,Merrino T,Ruggeri F.Clinical results and thoughts on sensory nerve repair by autologous vein graft in emergency hand reconstruction.Chir Main2002;21:194-7

[21] Wong JN,Morhart MJ.Electrical stimulation enhances sensory recovery: a randomized controlled trial.Ann Neurol2015;77:996-1006

[22] Modrak M,Gurgenashvili K,Elfar JC.Peripheral nerve injury and myelination: potential therapeutic strategies.J Neurosci Res2020;98:780-95. PMCID:PMC7072007

[23] Lopes B,Alvites R.Peripheral nerve injury treatments and advances: one health perspective.Int J Mol Sci2022;23:918 PMCID:PMC8779751

[24] Maugeri G,Trovato B.The role of exercise on peripheral nerve regeneration: from animal model to clinical application.Heliyon2021;7:e08281. PMCID:PMC8571504

[25] Chhabra A,Belzberg A.Peripheral nerve injury grading simplified on MR neurography: as referenced to Seddon and Sunderland classifications.Indian J Radiol Imaging2014;24:217-24 PMCID:PMC4126136

[26] Kaya Y.Sir Herbert Seddon (1903-1977) and his classification scheme for peripheral nerve injury.Childs Nerv Syst2015;31:177-80.