Double-reading mammograms using artificial intelligence technologies: A new model of mass preventive examination organization

Abstract

BACKGROUND: In recent years, the availability of medical datasets and technologies for software development based on artificial intelligence technology has resulted in a growth in the number of solutions for medical diagnostics, particularly mammography. Registered as a medical device, this program can interpret digital mammography, significantly saving time, material, and human resources in healthcare while ensuring the quality of mammary gland preventive studies.

AIM: This study aims to justify the possibility and effectiveness of artificial intelligence-based software for the first interpretation of digital mammograms while maintaining the practice of a radiologist’s second description of X-ray images.

MATERIALS AND METHODS: A dataset of 100 digital mammography studies (50 — “absence of target pathology” and 50 ― “presence of target pathology,” with signs of malignant neoplasms) was processed by software based on artificial intelligence technology that was registered as a medical device in the Russian Federation. Receiver operating characteristic analysis was performed. Limitations of the study include the values of diagnostic accuracy metrics obtained for software based on artificial intelligence technology versions, relevant at the end of 2022.

RESULTS: When set to 80.0% sensitivity, artificial intelligence specificity was 90.0% (95% CI, 81.7–98.3), and accuracy was 85.0% (95% CI, 78.0–92.0). When set to 100% specificity, artificial intelligence demonstrated 56.0% sensitivity (95% CI, 42.2–69.8) and 78.0% accuracy (95% CI, 69.9–86.1). When the sensitivity was set to 100%, the artificial intelligence specificity was 54.0% (95% CI, 40.2–67.8), and the accuracy was 77.0% (95% CI, 68.8–85.2). Two approaches have been proposed, providing an autonomous first interpretation of digital mammography using artificial intelligence. The first approach is to evaluate the X-ray image using artificial intelligence with a higher sensitivity than that of the double-reading mammogram by radiologists, with a comparable level of specificity. The second approach implies that artificial intelligence-based software will determine the mammogram category (“absence of target pathology” or “presence of target pathology”), indicating the degree of “confidence” in the obtained result, depending on the corridor into which the predicted value falls.

CONCLUSIONS: Both proposed approaches for using artificial intelligence-based software for the autonomous first interpretation of digital mammograms can provide diagnostic quality comparable to, if not superior to, double-image reading by radiologists. The economic benefit from the practical implementation of this approach nationwide can range from 0.6 to 5.5 billion rubles annually.

Keywords

artificial intelligence / diagnostic accuracy / mammography / preventive medicine

References

[1] Malignant neoplasms in Russia in 2021 (morbidity and mortality). Ed by A.D. Kaprin, V.V. Starinsky, A.O. Shahzadova. Мoscow; 2022. 252 р. (In Russ).

[2] Злокачественные новообразования в России в 2021 году (заболеваемость и смертность) / под ред. А.Д. Каприна, В.В. Старинского, А.О. Шахзадовой. Москва, 2022. 252 с.

[3] The state of oncological assistance to the population of Russia in 2021. Ed by A.D. Kaprin, V.V. Starinsky, A.O. Shahzadova. Мoscow; 2022. 239 р. (In Russ).

[4] Состояние онкологической помощи населению России в 2021 году / под ред. А.Д. Каприна, В.В. Старинского, А.О. Шахзадовой. Москва, 2022. 239 с.

[5] Chen Y, James JJ, Michalopoulou E, et al. Performance of radiologists and radiographers in double reading mammograms: The UK national health service breast screening program. Radiology. 2023;306(1):102–109. doi: 10.1148/radiol.212951

[6] Chen Y., James J.J., Michalopoulou E., et al. Performance of radiologists and radiographers in double reading mammograms: The UK national health service breast screening program // Radiology. 2023. Vol. 306, N 1. P. 102–109. doi: 10.1148/radiol.212951

[7] Euler-Chelpin MV, Lillholm M, Napolitano G, et al. Screening mammography: Benefit of double reading by breast density. Breast Cancer Res Treat. 2018;171(3):767–776. doi: 10.1007/s10549-018-4864-1

[8] Euler-Chelpin M.V., Lillholm M., Napolitano G., et al. Screening mammography: Benefit of double reading by breast density // Breast Cancer Res Treat. 2018. Vol. 171, N 3. P. 767–776. doi: 10.1007/s10549-018-4864-1

[9] Hickman SE, Woitek R, Le EP, et al. Machine learning for workflow applications in screening mammography: Systematic review and meta-analysis. Radiology. 2022;302(1):88–104. doi: 10.1148/radiol.2021210391

[10] Hickman S.E., Woitek R., Le E.P., et al. Machine learning for workflow applications in screening mammography: Systematic review and meta-analysis // Radiology. 2022. Vol. 302, N 1. P. 88–104. doi: 10.1148/radiol.2021210391

[11] Liu J, Lei J, Ou Y, et al. Mammography diagnosis of breast cancer screening through machine learning: A systematic review and meta-analysis. Clin Exp Med. 2022. doi: 10.1007/s10238-022-00895-0

[12] Liu J., Lei J., Ou Y., et al. Mammography diagnosis of breast cancer screening through machine learning: A systematic review and meta-analysis // Clin Exp Med. 2022. doi: 10.1007/s10238-022-00895-0

[13] Rozhkova NI, Rojtberg PG, Varfolomeeva AA, et al. Neural network-based segmentation model for breast cancer X-ray screening. Sechenov medical journal. 2020;11(3):4–14 (In Russ). doi: 10.47093/2218-7332.2020.11.3.4-14

[14] Рожкова Н.И., Ройтберг П.Г., Варфоломеева А.А., и др. Cегментационная модель скрининга рака молочной железы на основе нейросетевого анализа рентгеновских изображений // Сеченовский вестник. 2020. Т. 11, № 3. С. 4–14. doi: 10.47093/2218-7332.2020.11.3.4-14

[15] Vasilev JA, Vladzimirskyy AV. Computer vision in radiology: The first stage of the Moscow experiment: Monograph. Moscow: Izdatel’skie resheniya; 2022. 388 р. (In Russ).

[16] Компьютерное зрение в лучевой диагностике: первый этап Московского эксперимента. Монография / под ред. Ю.А. Васильева, А.В. Владзимирского. Москва: Издательские решения, 2022. 388 с.

[17] Patent RUS № 2022617324/05.04.2022. Byul. № 4. Morozov SP, Andreichenko AE, Chetverikov SF, et al. A web-based tool for performing ROC analysis of diagnostic test results. Available from: https://www.elibrary.ru/item.asp?id=48373757. Accessed: 10.03.2023. (In Russ).

[18] Патент РФ на изобретение № 2022617324/05.04.2022. Бюл. № 4. Морозов С.П., Андрейченко А.Е., Четвериков С.Ф., и др. Веб-инструмент для выполнения ROC-анализа результатов диагностических тестов. Режим доступа: https://www.elibrary.ru/item.asp?id=48373757. Дата обращения: 10.03.2023.

[19] Morozov SP, Vladzimirsky AV, Klyashtornyy VG, et al. Clinical acceptance of software based on artificial intelligence technologies (radiology). Moscow; 2019. 45 p. (Ser. Best practices in medical imaging).

[20] Morozov S.P., Vladzimirsky A.V., Klyashtornyy V.G., et al. Clinical acceptance of software based on artificial intelligence technologies (radiology). Moscow, 2019. 45 р. (Сер. Best practices in medical imaging).

[21] Schaffter T, Buist DS, Lee CI, et al. Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms. JAMA Netw Open. 2020;3(3):e200265. doi: 10.1001/jamanetworkopen.2020.0265

[22] Schaffter T., Buist D.S., Lee C.I., et al. Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms // JAMA Netw Open. 2020. Vol. 3, N 3. P. e200265. doi: 10.1001/jamanetworkopen.2020.0265

[23] Wan Y, Tong Y, Liu Y, et al. Evaluation of the combination of artificial intelligence and radiologist assessments to interpret malignant architectural distortion on mammography. Front Oncol. 2022;(12):880150. doi: 10.3389/fonc.2022.880150

[24] Wan Y., Tong Y., Liu Y., et al. Evaluation of the combination of artificial intelligence and radiologist assessments to interpret malignant architectural distortion on mammography // Front Oncol. 2022. N 12. P. 880150. doi: 10.3389/fonc.2022.880150

[25] Leibig C, Brehmer M, Bunk S, et al. Combining the strengths of radiologists and AI for breast cancer screening: A retrospective analysis. Lancet Digit Health. 2022;4(7):e507–e519. doi: 10.1016/S2589-7500(22)00070-X

[26] Leibig C., Brehmer M., Bunk S., et al. Combining the strengths of radiologists and AI for breast cancer screening: A retrospective analysis // Lancet Digit Health. 2022. Vol. 4, N 7. P. e507–e519. doi: 10.1016/S2589-7500(22)00070-X

RIGHTS & PERMISSIONS

Eco-Vector