Leveraging artificial intelligence for resident recruitment: can the dream of holistic review be realized?

Abstract

Aim: The purpose of this study was to investigate if principles of Artificial Intelligence (AI), specifically Natural Language Processing (NLP), could be applied to the personal statements of general surgery residency applicants in order to gain valuable insight into the candidates and facilitate a more comprehensive assessment.

Methods: The personal statements from individuals applying for a general surgery residency position during the 2021/22 application cycle (n = 1792) were analyzed using AI technology. Comparison groups were drawn from a database of documents from the general population and the personal statements of current general surgery residents (n = 64) at a single academic center. The study was conducted in collaboration with a leading language psychology and natural language processing organization.

Results: Applicants exhibited a language-based personality that was highly self-assured (P < 0.0001) and trusting (P < 0.0001), and less stress-prone (P < 0.0001) and impulsive (P < 0.0001) than that of the general population. Compared to the general applicant pool, current residents were significantly more emotionally aware (P < 0.001) and organized (P < 0.001) and less self-assured (P < 0.001) and less driven by power (P < 0.001).

Conclusion: Natural language processing technology can be utilized to assess the unique characteristics of general surgery resident applicants based on the content of their personal statements. In addition, candidates who successfully gain admission to a single academic program display different language-based personalities and drives compared to the general applicant pool. Incorporating these principles of artificial intelligence into the residency selection process could facilitate a more holistic evaluation of candidates.

Keywords

Artificial Intelligence / natural language processing / general surgery / resident selection / holistic review

References

[1] Chowdhary KR.Natural language processing. In: fundamentals of artificial intelligence. New Delhi: Springer India, 2020. pp. 603-49.

[2] Boyd RL.Natural language analysis and the psychology of verbal behavior: the past, present, and future states of the field.J Lang Soc Psychol2021;40:21-41 PMCID:PMC8373026

[3] Nehemiah A,Song Y.Looking beyond the numbers: increasing diversity and inclusion through holistic review in general surgery recruitment.J Surg Educ2021;78:763-9

[4] Sungar WG,Mccormick T.Implementation of holistic review into emergency medicine residency application screening to improve recruitment of underrepresented in medicine applicants.AEM Educ Train2021;5:S10-8 PMCID:PMC8480502

[5] Receptiviti. Available from: https://www.receptiviti.com/ [Last accessed on 21 Dec 2022]

[6] Cobb-clark DA.The stability of big-five personality traits.Econ Lett2012;115:11-5

[7] He J,Xu J,Zhou X.The practical implementation of artificial intelligence technologies in medicine.Nat Med2019;25:30-6 PMCID:PMC6995276

[8] Kaul V,Gross SA.History of artificial intelligence in medicine.Gastrointest Endosc2020;92:807-12

[9] Kumar Y,Singla R.Artificial intelligence in disease diagnosis: a systematic literature review, synthesizing framework and future research agenda.J Ambient Intell Human Comput2022:1-28 PMCID:PMC8754556

[10] Johnson KB,Weeraratne D.Precision medicine, AI, and the future of personalized health care.Clin Transl Sci2021;14:86-93 PMCID:PMC7877825

[11] Tokuyasu T,Matsunobu Y.Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy.Surg Endosc2021;35:1651-8 PMCID:PMC7940266

[12] Paranjape K,Nannan Panday R,Nanayakkara P.Introducing artificial intelligence training in medical education.JMIR Med Educ2019;5:e16048 PMCID:PMC6918207

[13] Chan KS.Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review.JMIR Med Educ2019;5:e13930 PMCID:PMC6598417

[14] Kron FW,Fetters MD.Response to “emerging roles of virtual patients in the age of AI”.AMA J Ethics2019;21:E920-5 PMCID:PMC7294847

[15] Chen X,Zou D.Application and theory gaps during the rise of artificial intelligence in education.Comput Educ Artif Intell2020;1:100002

[16] Humphrey HJ.The pass/fail decision for USMLE step 1-next steps.JAMA2020;323:2022-3

[17] Pontell ME,Ganesh Kumar N.The change of USMLE step 1 to pass/fail: perspectives of the surgery program director.J Surg Educ2021;78:91-8 PMCID:PMC7347473

[18] Association of program directors in surgery. APDS position statements. Available from: https://apds.org/about/apds-position-statements/ [Last accessed on 21 Dec 2022]

[19] Crane JT.Selection criteria for emergency medicine residency applicants.Acad Emerg Med2000;7:54-60

[20] Demzik A,Chew C.Gender-based differences in urology residency applicant personal statements.Urology2021;150:2-8 PMCID:PMC7536514

[21] Flanigan KL,Morrell DS.Value of personal statements to dermatology programs: a survey-based critical review.Dermatology Online J2020;26:5

[22] Bird E,Jenner N,Weitkamp E.The ethics of artificial intelligence: issues and initiatives: study panel for the future of science and technology. Brussels : European Union, 2020.

[23] Hovy D.Five sources of bias in natural language processing.Lang Linguist Compass2021;15:e12432 PMCID:PMC9285808

[24] List JM.Honesty and fairness in the residency match.Virtual Mentor2012;14:1011-5

[25] Bowe SN.“Fit” Has a broader meaning: recognizing the utility of person-environment fit theory in residency recruitment and selection.J Grad Med Educ2020;12:637-8 PMCID:PMC7594772

[26] Caretta-Weyer HA.An outcomes-oriented approach to residency selection: implementing novel processes to align residency programs and applicants.Acad Med2022;97:626-30

[27] Stephenson-Famy A,Oberoi S,Chiang S.Use of the interview in resident candidate selection: a review of the literature.J Grad Med Educ2015;7:539-48 PMCID:PMC4675409

[28] Labiner HE,Maloney Patel N.Virtual recruitment in surgical residency programs.Curr Surg Rep2021;9:25 PMCID:PMC8488912

[29] Jayaratne M.Predicting personality using answers to open-ended interview questions.IEEE Access2020;8:115345-5

[30] Ahmad N.Personality assessment using twitter tweets.Procedia Comput Sci2017;112:1964-73

[31] Rathi S,Jain R,Thakur N.Psychometric profiling of individuals using Twitter profiles: a psychological natural language processing based approach.Concurr Comput2022;34:e7029