Multi-Scale Memory-Enhanced Transformer for Optical Chemical Structure Recognition

Rui Wang , Yujin Ji , Youyong Li , Shuit-Tong Lee

Chemical Research in Chinese Universities ›› 2026, Vol. 42 ›› Issue (5) : 1604 -1614.

PDF
Chemical Research in Chinese Universities ›› 2026, Vol. 42 ›› Issue (5) :1604 -1614. DOI: 10.1007/s40242-026-6126-9
Research Article
research-article
Multi-Scale Memory-Enhanced Transformer for Optical Chemical Structure Recognition
Author information +
History +
PDF

Abstract

Optical Chemical Structure Recognition (OCSR) is a key technology at the intersection of chemistry and computer science, aiming to convert molecular structure images into machine-readable sequence representations. However, existing OCSR models still fall short in building long-range dependencies within sequences, making it difficult to accurately generate long sequences with complex structures. To address this, we propose a Multi-Scale Memory-Enhanced Transformer (MSMET) model and design two modules: the Layer-wise Hierarchical Fusion (LHF) module and the Cross-Attention Memory Augmentation (CAMA) module. The LHF module employs adaptive feature recalibration across encoder blocks through learnable gating, effectively alleviating feature forgetting phenomena in deep architectures while maintaining multi-scale visual discriminability. The CAMA module equips the decoder with a learnable memory matrix that dynamically enriches cross-modal attention, significantly enhancing the model’s capacity to capture long-distance topological interactions in molecular graph structures. We evaluated the performance of MSMET, and the experimental results demonstrate that the model achieved strong overall state-of-the-art performance.

Keywords

Optical chemical structure recognition / Transformer / Attention / Deep learning

Cite this article

Download citation ▾
Rui Wang, Yujin Ji, Youyong Li, Shuit-Tong Lee. Multi-Scale Memory-Enhanced Transformer for Optical Chemical Structure Recognition. Chemical Research in Chinese Universities, 2026, 42 (5) : 1604-1614 DOI:10.1007/s40242-026-6126-9

登录浏览全文

4963

注册一个新账户 忘记密码

References

[1]

Weininger DJ. Chem. Inf. Comput. Sci., 1988, 2831.

[2]

Heller S R, McNaught A, Pletnev I, Stein S, Tchekhovskoi DJ. Cheminform., 2015, 723.

[3]

Krenn M, Häse F, Nigam A, Friederich P, Aspuru-Guzik AMach. Learn.: Sci. Technol., 2020, 1045024.

[4]

Wang R, Ji Y, Li Y, Lee S-TJ. Phys. Chem. Lett., 2025, 16421.

[5]

Gao T, Ji Y, Liu C, Li YSci. China Mater., 2024, 671243.

[6]

Wang H, Sun J, Li Y, Deng WSci. Data, 2025, 12648.

[7]

Chen S, Jin L, Ji Y, Li YJ. Mater. Chem. A, 2025, 139841.

[8]

Merchant A, Batzner S, Schoenholz S S, Aykol M, Cheon G, Cubuk E DNature, 2023, 62480.

[9]

Santos-Júnior C D, Torres M D T, Duan Y, Rodríguez del Río Á, Schmidt T S B, Chong H, Fullam A, Kuhn M, Zhu C, Houseman A, Somborski J, Vines A, Zhao X-M, Bork P, Huerta-Cepas J, de la Fuente-Nunez C, Coelho L PCell, 2024, 1873761.

[10]

McDaniel J R, Balmuth J RJ. Chem. Inf. Comput. Sci., 1992, 32373.

[11]

Filippov I V, Nicklaus M CJ. Chem. Inf. Model., 2009, 49740.

[12]

Peryea T, Katzel D, Zhao T, Southall N, Nguyen D-TAbstracts of Papers, 258th ACS National Meeting & Exposition, 2019

[13]

Valko A T, Johnson A PJ. Chem. Inf. Model., 2009, 49780.

[14]

Smolov V, Zentsev F, Rybalkin MProceedings of the Twentieth Text REtrieval Conference (TREC 2011), 2011

[15]

Park J, Rosania G R, Shedden K A, Nguyen M, Lyu N, Saitou KChem. Cent. J., 2009, 34.

[16]

Oldenhof M, Arany A, Moreau Y, Simm JJ. Chem. Inf. Model., 2020, 604506.

[17]

Zhang X-C, Yi J-C, Yang G-P, Wu C-K, Hou T-J, Cao D-SBrief. Bioinform., 2022, 23bbac033.

[18]

LeCun Y, Bottou L, Bengio Y, Haffner PProc. IEEE, 1998, 862278.

[19]

Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby NProceedings of the 9th International Conference on Learning Representations (ICLR), 2021Virtual Event

[20]

Elman J LCogn. Sci., 1990, 14179.

[21]

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, Kaiser T, Polosukhin IProceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), 2017

[22]

Yang Z, Li JJ. East China Univ. Sci. Technol., 2023, 49135.

[23]

Khokhlov I, Krasnov L, Fedorov M V, Sosnin SChem. Methods, 2022, 2e202100069.

[24]

Rajan K, Zielesny A, Steinbeck CJ. Cheminform., 2021, 1361.

[25]

Rajan K, Brinkhaus H O, Agea M I, Zielesny A, Steinbeck CNat. Commun., 2023, 145045.

[26]

Xu Z, Li J, Yang Z, Li S, Li HJ. Cheminform., 2022, 1441.

[27]

Lin F, Li JComplex Intell. Syst., 2024, 101.

[28]

Zhang H., Wu C., Zhang Z., Zhu Y., Lin H., Zhang Z., Sun Y., He T., Mueller J., Manmatha R., Li M., Smola A., arXiv, 2020, arXiv:2004.08955.

[29]

He K, Zhang X, Ren S, Sun JProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

[30]

Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna ZProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

[31]

Tan M, Le Q VProceedings of the 36th International Conference on Machine Learning (ICML), 2019

[32]

Tan M, Le Q VProceedings of the 38th International Conference on Machine Learning (ICML), 2021Virtual Event

[33]

Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo BProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021Virtual Conference

[34]

Lee Y, Kim J, Willette J, Hwang S JProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

[35]

Kim S, Chen J, Cheng T, Gindulyte A, He J, He S, Li Q, Shoemaker B A, Thiessen P A, Yu B, Zaslavsky L, Zhang J, Bolton E ENucleic Acids Res., 2021, 49D1388.

[36]

Willighagen E L, Mayfield J W, Alvarsson J, Berg A, Carlsson L, Jeliazkova N, Kuhn S, Pluskal T, Rojas-Chertó M, Spjuth O, Torrance G, Evelo C T, Guha R, Steinbeck CJ. Cheminform., 2017, 933.

[37]

Landrum GRDKit: A Software Suite for Cheminformatics, Computational Chemistry, and Predictive Modeling, 2013

[38]

Zhang H, Zeng P, Gao L, Lyu X, Song J, Shen H TIEEE Trans. Circuits Syst. Video Technol., 2024, 344829.

[39]

Yi X, Fu Y, Liu R, Zhang H, Hua RIEEE J. Biomed. Health Inform., 2024, 282152.

[40]

Chen Z, Song Y, Chang T-H, Wan XProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020Online

[41]

Wang R, Ji Y, Li Y, Lee S-TJ. Phys. Chem. Lett., 2025, 1612787.

[42]

OpenAI, GPT-4o System Card, OpenAI, 2024-08-08, https://cdn.openai.com/gpt-4o-system-card.pdf, accessed on 2026-07-18.

[43]

OpenAI, Introducing OpenAI o3 and o4-mini, OpenAI, 2025-04-16, https://openai.com/index/introducing-o3-and-o4-mini/, accessed on 2026-05-31.

[44]

OpenAI, GPT-5 System Card, OpenAI, 2025-08-07, https://openai.com/index/gpt-5-system-card/, accessed on 2026-05-31.

[45]

DeepSeek-AI, DeepSeek-V3.1 Release, DeepSeek API Docs, 2025-08-21, https://api-docs.deepseek.com/news/news250821, accessed on 2026-05-31.

[46]

Guo D, Yang D, Zhang H, Song J, Wang P, Zhu Q, Xu R, Zhang R, Ma S, Bi X, Zhang X, Yu X, Wu Y, Wu Z F, Gou Z, Shao Z, Li Z, Gao Z, Liu A, Xue B, Wang B, Wu B, Feng B, Lu C, Zhao C, Deng C, Ruan C, Dai D, Chen D, Ji D, Li E, Lin F, Dai F, Luo F, Hao G, Chen G, Li G, Zhang H, Xu H, Ding H, Gao H, Qu H, Li H, Guo J, Li J, Chen J, Yuan J, Tu J, Qiu J, Li J, Cai J L, Ni J, Liang J, Chen J, Dong K, Hu K, You K, Gao K, Guan K, Huang K, Yu K, Wang L, Zhang L, Zhao L, Wang L, Zhang L, Xu L, Xia L, Zhang M, Zhang M, Tang M, Zhou M, Li M, Wang M, Li M, Tian N, Huang P, Zhang P, Wang Q, Chen Q, Du Q, Ge R, Zhang R, Pan R, Wang R, Chen R J, Jin R L, Chen R, Lu S, Zhou S, Chen S, Ye S, Wang S, Yu S, Zhou S, Pan S, Li S S, Zhou S, Wu S, Yun T, Pei T, Sun T, Wang T, Zeng W, Liu W, Liang W, Gao W, Yu W, Zhang W, Xiao W L, An W, Liu X, Wang X, Chen X, Nie X, Cheng X, Liu X, Xie X, Liu X, Yang X, Li X, Su X, Lin X, Li X Q, Jin X, Shen X, Chen X, Sun X, Wang X, Song X, Zhou X, Wang X, Shan X, Li Y K, Wang Y Q, Wei Y X, Zhang Y, Xu Y, Li Y, Zhao Y, Sun Y, Wang Y, Yu Y, Zhang Y, Shi Y, Xiong Y, He Y, Piao Y, Wang Y, Tan Y, Ma Y, Liu Y, Guo Y, Ou Y, Wang Y, Gong Y, Zou Y, He Y, Xiong Y, Luo Y, You Y, Liu Y, Zhou Y, Zhu Y X, Huang Y, Li Y, Zheng Y, Zhu Y, Ma Y, Tang Y, Zha Y, Yan Y, Ren Z Z, Ren Z, Sha Z, Fu Z, Xu Z, Xie Z, Zhang Z, Hao Z, Ma Z, Yan Z, Wu Z, Gu Z, Zhu Z, Liu Z, Li Z, Xie Z, Song Z, Pan Z, Huang Z, Xu Z, Zhang Z, Zhang ZNature, 2025, 645633.

Rights & permissions

Jilin University, The Editorial Department of Chemical Research in Chinese Universities and Springer-Verlag GmbH

PDF

83

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉