Evaluating the Performance of ChatGPT-4o in Risk of Bias Assessments

Ilari Kuitunen; Ville T. Ponkilainen; Rasmus Liukkonen; Lauri Nyrhi; Oskari Pakarinen; Matias Vaajala; Mikko M. Uimonen

doi:10.1111/jebm.12662

Journal of Evidence-Based Medicine ›› 2024, Vol. 17 ›› Issue (4) : 700 -702. DOI: 10.1111/jebm.12662

LETTER

Evaluating the Performance of ChatGPT-4o in Risk of Bias Assessments

Author information +

History +

PDF

Cite this article

Download citation ▾

Ilari Kuitunen, Ville T. Ponkilainen, Rasmus Liukkonen, Lauri Nyrhi, Oskari Pakarinen, Matias Vaajala, Mikko M. Uimonen. Evaluating the Performance of ChatGPT-4o in Risk of Bias Assessments. Journal of Evidence-Based Medicine, 2024, 17(4): 700-702 DOI:10.1111/jebm.12662

登录浏览全文

4963

注册一个新账户忘记密码

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	M. H.Murad,N.Asi, M.Alsawas, and F. Alahdab,“New Evidence Pyramid, ” BMJ Evidence-Based Medicine 21, no. 4 (2016):125–127.

[2]	G. H.Guyatt,A. D.Oxman, G.Vist, et al.,“GRADE Guidelines:4. Rating the Quality of Evidence–Study Limitations (Risk of Bias), ” Journal of Clinical Epidemiology 64, no. 4 (2011):407–415.

[3]	J. A. CSterne,J.Savović,M. J.Page, et al.,“RoB 2: A Revised Tool for Assessing Risk of Bias in Randomised Trials, ” Bmj 28, no. 366 (2019): l4898.

[4]	J. P. THiggins,D. G.Altman, P. C.Gotzsche, et al.,“The Cochrane Collaboration’s Tool for Assessing Risk of Bias in Randomised Trials, ” Bmj 343 (2011): d5928.

[5]	T. F.Crocker,N.Lam, M.Jordão, et al.,“Risk-of-bias Assessment Using Cochrane’s Revised Tool for Randomized Trials (RoB 2) Was Useful but Challenging and Resource-Intensive: Observations From a Systematic Review, ” Journal of Clinical Epidemiology 161 (2023):39–45.

[6]	I.Kuitunen,V. T.Ponkilainen, and M.Uimonen,“Incorrect Blinding Assessments Are Common in Meta-Analyses Published in High Impact Journals, ” Journal of Evidence-Based Medicine 17, no. 3 (2024):471–473.

[7]	I.Kuitunen,K.Räsänen,M. R.Gualano, and D.De Luca,“Blinding Assessments in Neonatal Ventilation Meta-Analyses: A Systematic Meta-Epidemiological Review, ” Neonatology (2024):1–8.

[8]	L.Puljak,I.Ramic, C.Arriola Naharro, et al.,“Cochrane Risk of Bias Tool Was Used Inadequately in the Majority of Non-Cochrane Systematic Reviews, ” Journal of Clinical Epidemiology 123 (2020):114–119.

[9]	S.Minozzi,M.Cinquini,S.Gianola,M.Gonzalez-Lorenzo, and R. Banzi,“The Revised Cochrane Risk of Bias Tool for Randomized Trials (RoB 2) Showed Low Interrater Reliability and Challenges in Its Application, ” Journal of Clinical Epidemiology 126 (2020):37–44.

[10]	B.Mohammad,T.Supti, M.Alzubaidi, et al.,“The Pros and Cons of Using ChatGPT in Medical Education: A Scoping Review, ” Studies in Health Technology and Informatics 305 (2023):644–647.

[11]	B.Hasan,S.Saadi, N. S.Rajjoub, et al.,“Integrating Large Language Models in Systematic Reviews: A Framework and Case Study Using ROBINS-I for Risk of Bias Assessment, ” BMJ Evidence-Based Medicine 29, no. 6 (2024):394–398.

[12]	J.Barsby,S.Hume, H. A.Lemmey,J. Cutteridge,R.Lee, and K. D.Bera,“Pilot Study on Large Language Models for Risk-of-bias Assessments in Systematic Reviews: A (I) New Type of Bias?, ” BMJ Evidence-Based Medicine (2024): bmjebm-2024-112990. Advance online publication.

[13]	S.Minozzi,M.Gonzalez-Lorenzo,M.Cinquini, et al.,“Adherence of Systematic Reviews to Cochrane RoB2 Guidance Was Frequently Poor: A Meta Epidemiological Study, ” Journal of Clinical Epidemiology 152 (2022):47–55.

RIGHTS & PERMISSIONS

2024 The Author(s). Journal of Evidence-Based Medicine published by Chinese Cochrane Center, West China Hospital of Sichuan University and John Wiley & Sons Australia, Ltd.