A Large Language Model Pipeline for Stroke Staging Using Electronical Medical Records

Xiaochao Luo; Yanmei Liu; Yu Ma; Chuwei Li; Minghong Yao; Hunong Xiang; Xuan Qin; Jiali Liu; Xiaoping Zhan; Chen Zhang; Songtao Zang; Ke Deng; Ling Li; Xin Sun

doi:10.1111/jebm.70159

Journal of Evidence-Based Medicine ›› 2026, Vol. 19 ›› Issue (2) :e70159 DOI: 10.1111/jebm.70159

ARTICLE

A Large Language Model Pipeline for Stroke Staging Using Electronical Medical Records

Xiaochao Luo ¹^,²^,³^,⁴^,⁵
, Yanmei Liu ¹^,²^,³^,⁴^,⁵
, Yu Ma ¹^,²^,³^,⁴^,⁵
, Chuwei Li ⁶
, Minghong Yao ¹^,²^,³^,⁴^,⁵
, Hunong Xiang ¹^,²^,³^,⁴^,⁵
, Xuan Qin ⁷
, Jiali Liu ⁸
, Xiaoping Zhan ⁹
, Chen Zhang ⁹
, Songtao Zang ⁹
, Ke Deng ¹^,²^,³^,⁴^,⁵
, Ling Li ¹^,²^,³^,⁴^,⁵^,¹⁰
, Xin Sun ¹^,²^,³^,⁴^,⁵^,¹¹

Author information +

¹. Clinical Epidemiology and Evidence-Based Medicine Center, West China Hospital, Sichuan University, Chengdu, China

². Department of Neuro surgery, West China Hospital, Sichuan University, Chengdu, China

³. Cochrane China and MAGIC China Center, West China Hospital, Sichuan University, Chengdu, China

⁴. NHC Key Laboratory of Clinical Epidemiology and Evidence-Based Medicine, West China Hospital, Sichuan University, Chengdu, China

⁵. Sichuan Center of Technology Innovation for Real World Data, Chengdu, China

⁶. School of International Pharmaceutical Business, China Pharmaceutical University, Nanjing, China

⁷. Yibin University, Yibin, China

⁸. State Key Laboratory of Southwestern Chinese Medicine Resources, School of Pharmacy, Chengdu University of Traditional Chinese Medicine, Chengdu, China

⁹. Beijing Baymax Technology Co. Ltd, Beijing, China

¹⁰. Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Canada

¹¹. Department of Epidemiology and Biostatistics, West China School of Public Health and West China Fourth Hospital, Sichuan University, Chengdu, China

liling@wchscu.cn

sunxin@wchscu.cn

Show less

History +

PDF

Abstract

Objective: Identifying stroke patients at different disease stages is a prerequisite for clinical research using electronic medical records (EMRs), whereas an artificial intelligence-based model that can be directly applied remains lacking. We therefore develop a large language model (LLM) pipeline for stroke staging model (StrokeSM) in retrospective clinical research.

Methods: StrokeSM was developed using a Chinese national stroke database comprising EMRs from 33,637 patients. A total of 2000 patients were randomly selected from the Tianjin regional stroke database for external validation. StrokeSM comprised three phases: stroke hospitalization identification based on BERT and a bidirectional cross-attention network to fuse present illness history and discharge diagnosis, symptom–time extraction based on chief complaint through a UIE-base LLM, and stroke staging classification according to the predefined rules.

Results: On the test set, StrokeSM achieved accuracy, F1 score, precision, and recall of 0.90, 0.91, 0.91, and 0.90, respectively. The F1 score, precision, and recall of StrokeSM for acute phase was 0.91, 0.89, and 0.93, respectively. On the external validation set, StrokeSM had an accuracy, F1 score, precision, and recall of 0.92, 0.93, 0.93, and 0.92, respectively. Moreover, StrokeSM performed remarkably well in acute phase, with F1 score, precision, and recall of 0.97, 0.98, and 0.96, respectively.

Conclusions: StrokeSM had achieved state-of-the-art performance, providing an accurate method of classifying stroke populations with different disease stages in EMRs, especially in the acute phase. StrokeSM heralds automatic and accurate identification of disease stage phenotypes based on LLM in EMRs, laying the foundation for drawing reliable conclusions in clinical research.

Keywords

classification / electronic health record / large language model / natural language processing / stroke

Cite this article

Download citation ▾

Xiaochao Luo, Yanmei Liu, Yu Ma, Chuwei Li, Minghong Yao, Hunong Xiang, Xuan Qin, Jiali Liu, Xiaoping Zhan, Chen Zhang, Songtao Zang, Ke Deng, Ling Li, Xin Sun. A Large Language Model Pipeline for Stroke Staging Using Electronical Medical Records. Journal of Evidence-Based Medicine, 2026, 19 (2) : e70159 DOI:10.1111/jebm.70159