A New Machine-Learning Extracting Approach to Construct a Knowledge Base: A Case Study on Global Stromatolites over Geological Time

Abstract

Within any scientific disciplines, a large amount of data are buried within various literature depositories and archives, making it difficult to manually extract useful information from the datum swamps. The machine-learning extraction of data therefore is necessary for the big-data-based studies. Here, we develop a new text-mining technique to reconstruct the global database of the Precambrian to Recent stromatolites, providing better understanding of secular changes of stromatolites though geological time. The step-by-step data extraction process is described as below. First, the PDF documents of stromatolite-containing literatures were collected, and converted into text formation. Second, a glossary and tag-labeling system using NLP (Natural Language Processing) software was employed to search for all possible candidate pairs from each sentence within the papers collected here. Third, each candidate pair and features were represented as a factor graph model using a series of heuristic procedures to score the weights of each pair feature. Occurrence data of stromatolites versus stratigraphical units (abbreviated as Strata), facies types, locations, and age worldwide were extracted from literatures, respectively, and their extraction accuracies are 92%/464, 87%/778, 92%/846, and 93%/405 from 3 750 scientific abstracts, respectively, and are 90%/1 734, 86%/2 869, 90%/2 055 and 91%/857 from 11 932 papers, respectively. A total of 10 072 unique datum items were identified. The newly obtained stromatolite dataset demonstrates that their stratigraphical occurrences reached a pronounced peak during the Proterozoic (2 500–541 Ma), followed by a distinct fall during the Early Phanerozoic, and overall fluctuations through the Phanerozoic (541–0 Ma). Globally, seven stromatolite hotspots were identified from the new dataset, including western United States, eastern United States, western Europe, India, South Africa, northern China, and southern China. The proportional occurrences of inland aquatic stromatolites remain rather low (∼ 20%) in comparison to marine stromatolites from the Precambrian to Jurassic, and then display a significant increase (30%–70%) from the Cretaceous to the present.

Keywords

machine learning / knowledge base construction / stromatolites / Precambrian / knowledge graph

References

[1] Al-Badrashiny, M., Bolton, J., Chaganty, A. T., et al., 2017. Tinkerbell: Cross-Lingual Cold-Start Knowledge Base Construction. The 2017 Text Analysis Conference, TAC 2017, November 13–14, Gaithersburg

[2] Allwood A C, Rosing M T, Flannery D T, . Reassessing Evidence of Life in 3 700-Million-Year-Old Rocks of Greenland. Nature, 2018, 563(7730): 241-244.

[3] Angeli, G., Gupta, S., Jose, M., et al., 2014. Stanford’s 2014 Slot Filling Systems. TAC KBP, 695

[4] Banon, S., 2021. Elasticsearch. https://www.elastic.co/

[5] Carlson, A., Betteridge, J., Kisiel, B., et al., 2010. Toward an Architecture for Never-Ending Language Learning. The Twenty-Fourth AAAI Conference on Artificial Intelligence. July 11–15, 2010, Atlanta, Georgia. ACM, New York, 1306–1313. https://doi.org/10.5555/2898607.2898816

[6] Cermeño P, Falkowski P G, Romero O E, . Continental Erosion and the Cenozoic Rise of Marine Diatoms. Proceedings of the National Academy of Sciences of the United States of America, 2015, 112(14): 4239-4244.

[7] Chen Z Q, Tong J N, Liao Z T, . Structural Changes of Marine Communities over the Permian - Triassic Transition: Ecologically Assessing the End-Permian Mass Extinction and Its Aftermath. Global and Planetary Change, 2010, 73(1/2): 123-140.

[8] Chen Z Q, Tu C Y, Pei Y, . Biosedimentological Features of Major Microbe-Metazoan Transitions (MMTS) from Precambrian to Cenozoic. Earth-Science Reviews, 2019, 189: 21-50.

[9] Cheng Q M. IUGS’ Initiative on Data-Driven Geoscience Discovery. Journal of Earth Science, 2021, 32(2): 468-470.

[10] Cohen, K. M., Finney, S. C., Gibbard, P. L., et al., 2013. The Ics International Chronostratigraphic Chart. Episodes 36, 199–204. Community, A. P., 2021. Apache PDFBox A Java PDF Library. https://pdfbox.apache.org/

[11] Deleger L, Molnar K, Savova G, . Large-Scale Evaluation of Automated Clinical Note De-Identification and Its Impact on Information Extraction. Journal of the American Medical Informatics Association, 2013, 20(1): 84-94.

[12] Fan R, Wang L, Yan J, . Deep Learning-Based Named Entity Recognition and Knowledge Graph Construction for Geological Hazards. International Journal of GeoInformation, 2019, 9: 15 (in Chinese with English Abstract)

[13] Feng X Q, Chen Z Q, Benton M J, . Resilience of Infaunal Ecosystems during the Early Triassic Greenhouse Earth. Science Advances, 2022, 8(26): eabo0597

[14] Foundation, A. S., 2021. Apache Tika-Apache Tika. https://tika.apache.org

[15] Getoor L, Taskar B. Getoor L, Taskar B. Introduction to Statistical Relational Learning. Adaptive Computation and Machine Learning Series, 2007, Boston: The MIT Press

[16] Glyph, C., 2021. Xpdf. http://www.xpdfreader.com/

[17] Gradstein F M, Ogg J G, Schmitz M D, . Geologic Time Scale 2020, 2020, New York: Elsevier

[18] Hochreiter S, Schmidhuber J. Long Short-Term Memory. Neural Computation, 1997, 9 8 1735-1780.

[19] Hoffmann, R., Zhang, C., Weld, D. S., 2010. Learning 5 000 Relational Extractors. The 48th Annual Meeting of the Association for Computational Linguistics, July 13, 2010, Uppsala

[20] Hong Z, Ward L, Chard K, . Challenges and Advances in Information Extraction from Scientific Literature: A Review. JOM, 2021, 73(11): 3383-3400.

[21] Huang Y G, Chen Z Q, Roopnarine P D, . Ecological Dynamics of Terrestrial and Freshwater Ecosystems across Three Mid-Phanerozoic Mass Extinctions from Northwest China. Proceedings Biological Sciences, 2021, 288(1947): 20210148

[22] Karttunen L, Chanod J -P, Grefenstette G, . Regular Expressions for Language Engineering. Natural Language Engineering, 1996, 2 4 305-328.

[23] Knoll A H, Follows M J. A Bottom-up Perspective on Ecosystem Change in Mesozoic Oceans. Proceedings Biological Sciences, 2016, 283(1841): 20161755

[24] Kruiper, R., Vincent, J. F. V., Chen-Burger, J., et al., 2020. In Layman’s Terms: Semi-Open Relation Extraction from Scientific Texts. arXiv: 2005.07751. https://arxiv.org/abs/2005.07751

[25] Ku J P, Hicks J L, Hastie T, . The Mobilize Center: An NIH Big Data to Knowledge Center to Advance Human Movement Research and Improve Mobility. Journal of the American Medical Informatics Association, 2015, 22 6 1120-1125.

[26] Liu, J., Wright, S. J., Ré, C., et al., 2014. An Asynchronous Parallel Stochastic Coordinate Descent Algorithm. The 31st International Conference on International Conference on Machine Learning-Volume 32. June 21–26, 2014, Beijing. https://doi.org/10.5555/3044805.3044945

[27] Lowagie, B., 2021. The Leading Pdf Library for Developers Itext. https://itextpdf.com/en

[28] Ma X G, Wu C L, Carranza E J M, . Development of a Controlled Vocabulary for Semantic Interoperability of Mineral Exploration Geodata for Mining Projects. Computers & Geosciences, 2010, 36(12): 1512-1522.

[29] Mallory E K, Zhang C, Ré C, . Large-Scale Extraction of Gene Interactions from Full-Text Literature Using DeepDive. Bioinformatics, 2016, 32(1): 106-113.

[30] Microsoft, 2021. Microsoft Academic Graph. https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/

[31] Mintz, M., Bills, S., Snow, R., et al., 2009. Distant Supervision for Relation Extraction without Labeled Data. The Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. August 2–7, 2009, Singapor

[32] Mooney R J, Bunescu R. Mining Knowledge from Text Using Information Extraction. ACM SIGKDD Explorations Newsletter, 2005, 7(1): 3-10.

[33] Nasar Z, Jaffry S W, Malik M K. Information Extraction from Scientific Articles: A Survey. Scientometrics, 2018, 117(3): 1931-1990.

[34] Niu, F., Recht, B., Re, C., et al., 2011. HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent. The 24th International Conference on Neural Information Processing Systems. December 12–15, 2011, Granada. https://doi.org/10.5555/2986459.2986537

[35] Niu F, Zhang C, Ré C, . Elementary: Large-Scale Knowledge-Base Construction via Machine Learning and Statistical Inference. International Journal on Semantic Web & Information Systems, 2012, 8(3): 42-73.

[36] Nutman A P, Bennett V C, Friend C R L, . Rapid Emergence of Life Shown by Discovery of 3, 700-Million-Year-Old Microbial Structures. Nature, 2016, 537(7621): 535-538.

[37] Paleobiodb, 2021. The Paleobiology Database. https://paleobiodb.org/

[38] Peters, S. E., Zhang, C., Livny, M., et al., 2014. A Machine-Compiled Macroevolutionary History of Phanerozoic Life. arXiv: 1406.2963. https://arxiv.org/abs/1406.2963

[39] Qiu Q J, Xie Z, Wu L, . Geoscience Keyphrase Extraction Algorithm Using Enhanced Word Embedding. Expert Systems with Applications, 2019, 125 157-169.

[40] Qiu Q J, Xie Z, Wu L, . Automatic Spatiotemporal and Semantic Information Extraction from Unstructured Geoscience Reports Using Text Mining Techniques. Earth Science Informatics, 2020, 13(4): 1393-1410.

[41] Rabosky D L, Sorhannus U. Diversity Dynamics of Marine Planktonic Diatoms across the Cenozoic. Nature, 2009, 457(7226): 183-186.

[42] Raup D. Extinction: Bad Genes or Bad Luck?, 1981, New York: W. W. Norton & Company

[43] Richardson M, Domingos P. Markov Logic Networks. Machine Learning, 2006, 62(1): 107-136.

[44] Riloff, E., Jones, R., 1999. Learning Dictionaries for Information Extraction by Multi-Level Bootstrapping, In: Proceedings of the Sixteenth National Conference on Artificial Intelligence and Eleventh Conference on Innovative Applications of Artificial Intelligence, July 18–22, 1999, Orlando, 474–479

[45] Shinyama, Y., 2021. Pdfminer. https://www.unixuser.org/∼euske/python/pdfminer/

[46] Small S G, Medsker L. Review of Information Extraction Technologies and Applications. Neural Computing and Applications, 2014, 25(3): 533-548.

[47] Soudry D, Weissbrod T. Morphogenesis and Facies Relationships of Thrombolites and Siliciclastic Stromatolites in a Cambrian Tidal Sequence (Elat Area, Southern Israel). Palaeogeography, Palaeoclimatology, Palaeoecology, 1995, 114(2/3/4): 339-355.

[48] Translated, 2021. Mymemory. https://mymemory.translated.net/

[49] Vaswani, A., Shazeer, N., Parmar, N., et al., 2017. Attention is All You Need, In: Guyon, I., Luxburg, U. V., Bengio, S., eds., Advances in Neural Information Processing Systems, Curran Associates, Inc. https://doi.org/10.48550/arxiv.1706.03762

[50] Wainwright M J, Jordan M I. Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, 2007, 1(1–2): 1-305.

[51] Wang C B, Ma X G, Chen J G, . Information Extraction and Knowledge Graph Construction from Geoscience Literature. Computers & Geosciences, 2018, 112: 112-120.

[52] Webber, B., 2009. Discourse—Early Problems, Current Successes, Future Challenges. The Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. August 2–7, 2009, Singapore

[53] Wondershare Technology Group Co. Limited, 2021. Wondershare PDFelement. https://pdf.wondershare.com/

[54] Wu S, Hsiao L, Cheng X, . Fonduer: Knowledge Base Construction from Richly Formatted Data. Proceedings ACM-SIGMOD International Conference on Management of Data, 2018, 2018: 1301-1316.

[55] Yi H, Lin J, Zhou K, . The Origin of Miocene Lacustrine Stromatolites in the Hoh Xil Area and Its Paleoclimatic Implications. Journal of Mineralogy and Petrology, 2008, 28: 106-113. (in Chinese with English Abstract)

[56] Zhang C. DeepDive: A Data Management System for Automatic Knowledge Base Construction: [Dissertation], 2015, Madison: University of Wisconsin-Madison

[57] Zhang, C., Govindaraju, V., Borchardt, J., et al., 2013. GeoDeepDive: Statistical Inference Using Familiar Data-Processing Languages. The 2013 ACM SIGMOD International Conference on Management of Data. June 22–27, 2013, New York. https://doi.org/10.1145/2463676.246 3680

[58] Zhang, C., Ré, C., 2013. Towards High-Throughput Gibbs Sampling at Scale: A Study across Storage Managers. The 2013 ACM SIGMOD International Conference on Management of Data. June 22–27, 2013, New York. https://doi.org/10.1145/2463676.2463702