Enhanced multi-tuple extraction for materials: integrating pointer networks and augmented attention

Abstract

Extracting reliable, tuple-level information from materials texts is essential for data-driven materials design, yet multi-tuple sentences remain difficult due to intertwined semantics, syntactic complexity, and sparse supervision in higher-density cases. In this study, we address these challenges by formulating information extraction as an integrated process that couples entity extraction with tuple allocation. The framework combines an entity extraction module based on bidirectional encoder representations from transformers (MatSciBERT) with pointer networks and an allocation module that models inter- and intra-entity attention to enforce tuple coherence. Using the mechanical properties of multi-principal element alloys as a case study, we define the target schema and evaluate exact match tuple accuracy. Our experiments demonstrate F1 scores of 0.96, 0.95, 0.85, and 0.75 on datasets containing one to four tuples per sentence, and 0.85 on a randomly curated set. Ablation studies show that the allocation module is most critical, with inter-entity attention contributing more than intra-entity attention. Error analysis attributes the density-related performance decline mainly to semantic overlap and syntactic complexity, with upstream extraction errors more prominent under sparse supervision and allocation errors concentrated in structurally complex templates. This approach delivers precise, structured outputs suitable for downstream analysis and offers a domain-adaptable alternative to prompt-based large models when strict correctness is required.

Keywords

AI for materials / multi-tuple extraction / MatSciBERT / attention mechanism

References

[1] Gormley AJ.Machine learning in combinatorial polymer chemistry.Nat Rev Mater2021;6:642-4 PMCID:PMC8356908

[2] Wang S,Yuan X.Accelerating polymer discovery with uncertainty-guided PGCNN: explainable AI for predicting properties and mechanistic insights.J Chem Inf Model2024;64:5500-9

[3] Carvalho RP,Brandell D.Artificial intelligence driven in-silico discovery of novel organic lithium-ion battery cathodes.Energy Storage Mater2022;44:313-25

[4] Bhowmik A,Casas‐Cabanas M.Implications of the BATTERY 2030+ AI‐assisted toolkit on future low‐TRL battery discoveries and chemistries.Adv Energy Mater2021;12:2102698

[5] Zhou Z,Liu X.A generative deep learning framework for inverse design of compositionally complex bulk metallic glasses.npj Comput Mater2023;9:15

[6] Basu B,Xiao Y,Leong KW.Biomaterialomics: data science-driven pathways to develop fourth-generation biomaterials.Acta Biomater2022;143:1-25

[7] Singh AV,Ansari MHD.Artificial intelligence and machine learning empower advanced biomedical material design to toxicity prediction.Adv Intell Syst2020;2:2000084

[8] Debnath A,Sun H.Generative deep learning as a tool for inverse design of high entropy refractory alloys.J Mater Inf2021;1:3

[9] Hart GLW,Toher C.Machine learning for alloys.Nat Rev Mater2021;6:730-55

[10] Kononova O,Huo H,Olivetti EA.Opportunities and challenges of text mining in aterials research.iScience2021;24:102155 PMCID:PMC7905448

[11] Sierepeklis O.A thermoelectric materials database auto-generated from the scientific literature using ChemDataExtractor.Sci Data2022;9:648 PMCID:PMC9587980

[12] Kumar P,Cole JM.Auto-generating databases of Yield Strength and Grain Size using ChemDataExtractor.Sci Data2022;9:292

[13] Wang W,Tian S.Automated pipeline for superalloy data by text mining.npj Comput Mater2022;8:9

[14] Swain MC.ChemDataExtractor: a toolkit for automated extraction of chemical information from the scientific literature.J Chem Inf Model2016;56:1894-904

[15] Widiastuti NI.Convolution neural network for text mining and natural language processing.IOP Conf Ser Mater Sci Eng2019;662:052010

[16] Devlin J,Lee K.BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, June 2, 2019; Association for Computational Linguistics; Vol. 1. pp. 4171-86.

[17] Gupta T,Krishnan NMA.MatSciBERT: a materials domain language model for text mining and information extraction.npj Comput Mater2022;8:102

[18] Shetty P,Kuenneth C.A general-purpose material property data extraction pipeline from large polymer corpora using natural language processing.npj Comput Mater2023;9:52

[19] Lee J,Kim S.BioBERT: a pre-trained biomedical language representation model for biomedical text mining.Bioinformatics2020;36:1234-40 PMCID:PMC7703786

[20] Huang S.BatteryBERT: a pretrained language model for battery database enhancement.J Chem Inf Model2022;62:6365-77 PMCID:PMC9795558

[21] Brown TB,Ryder N. Language models are few-shot learners. arXiv 2020, arXiv:2005.14165. Available online: https://doi.org/10.48550/arXiv.2005.14165. (accessed 23 Mar 2026)

[22] OpenAI, Achiam, J.; Adler, S.; et al. GPT-4 technical report. arXiv 2023, arXiv:2303.08774. Available online: https://doi.org/10.48550/arXiv.2303.08774. (accessed 23 Mar 2026)

[23] Touvron H,Izacard G. LLaMA: open and efficient foundation language models. arXiv 2023, arXiv:2302.13971. Available online: https://doi.org/10.48550/arXiv.2302.13971. (accessed 23 Mar 2026)

[24] Chowdhery A,Devlin J. PaLM: scaling language modeling with pathways. arXiv 2022, arXiv:2204.02311. Available online: https://doi.org/10.48550/arXiv.2204.02311. (accessed 23 Mar 2026)

[25] Team G,Borgeaud S. Gemini: a family of highly capable multimodal models. arXiv 2023, arXiv:2312.11805. Available online: https://doi.org/10.48550/arXiv.2312.11805. (accessed 23 Mar 2026)

[26] Wei J,Zhao VY. Finetuned language models are zero-shot learners. arXiv 2021, arXiv:2109.01652. Available online: https://doi.org/10.48550/arXiv.2109.01652. (accessed 23 Mar 2026)

[27] Dagdelen J,Lee S.Structured information extraction from scientific text with large language models.Nat Commun2024;15:1418 PMCID:PMC10869356

[28] Vinyals O,Jaitly N.Pointer networks. In NIPS'15: Proceedings of the 29th International Conference on Neural Information Processing Systems, Montreal, Canada, December 7-12, 2015; MIT Press: Cambridge, Massachusetts, United States, 2015; Vol. 28. pp. 2692-700.

[29] Vaswani A,Parmar N. Attention is all you need. arXiv 2017, arXiv:1706.03762. Available online: https://doi.org/10.48550/arXiv.1706.03762. (accessed 23 Mar 2026)

[30] Sun F,Sun H,Ou W. Multi-source pointer network for product title summarization. arXiv 2018, arXiv:1808.06885. Available online: https://doi.org/10.48550/arXiv.1808.06885. (accessed 23 Mar 2026)

[31] Anthropic. The Claude 3 model family: opus, sonnet, haiku. https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf. (accessed 2026-03-23)

[32] Gemini Team Google. Gemini 1.5: unlocking multimodal understanding across millions of tokens of context. arXiv 2024, arXiv:2403.05530. Available online: https://doi.org/10.48550/arXiv.2403.05530. (accessed 23 Mar 2026)

[33] Grattafiori A,Jauhri A. The Llama 3 herd of models. arXiv 2024, arXiv:2407.21783 Available online: https://doi.org/10.48550/arXiv.2407.21783. (accessed 23 Mar 2026)

[34] Ji Z,Frieske R.Survey of hallucination in natural language generation.ACM Comput Surv2023;55:1-38

[35] Farquhar S,Kuhn L.Detecting hallucinations in large language models using semantic entropy.Nature2024;630:625-30 PMCID:PMC11186750

[36] Hei M,Zhang X.Enhancing Information Extraction from Low-sample Materials Science Literature by Transfer Learning. In 2024 10th International Conference on Big Data and Information Analytics (BigDIA), Chiang Mai, Thailand, Oct 25-28, 2024; IEEE, 2024; pp. 736-41.