Identification of Cenozoic Ostracods in the Qaidam Basin Using Convolutional and Transformer-Based Neural Networks

Abstract

Microfossils play a crucial role in biostratigraphy and paleoenvironmental reconstructions, as the first appearance datum (FAD) and last appearance datum (LAD) of specific microfossils enable precise stratigraphic correlations and age determinations. However, traditional identification methods are often time-intensive and heavily dependent on expert knowledge. To overcome these limitations, we propose a dual-path deep learning model, MicroViT, which integrates convolutional neural networks (CNNs) and vision transformers (ViTs) to automate the identification of Cenozoic ostracods (Microlimnocythere, Cyprideis, Qaidamocythere, Hemicyprinotus, Qaibeigouia, Austrocypris, and Candoniella) from the Qaidam Basin. MicroViT achieves an accuracy of 95.34%, demonstrating superior performance across all classification metrics. Furthermore, we utilized Gradient-weighted Class Activation Mapping (Grad-CAM) to visualize the decision-making process of the model, revealing that DL models focus on morphological features such as reticulation and honeycomb-like spots. We also investigated the potential for extending this approach to other microfossil groups, such as charophytes and sporopollen, as well as to diverse ostracod populations. These results highlight the significant potential of deep learning techniques for rapid and accurate microfossil classification, offering promising applications in micropaleontology and stratigraphic studies.

Keywords

Ostracods identification / deep learning / transformer-based neural networks / convolutional neural networks

References

[1] Aladhadh S, Alsanea M, Aloraini M, et al.. An Effective Skin Cancer Classification Mechanism via Medical Vision Transformer. Sensors, 2022, 22(11): 4008.

[2] Alqudah M, Ali Hussein M, van den Boorn S, et al.. Eocene Oil Shales from Jordan: Paleoenvironmental Implications from Reworked Microfossils. Marine and Petroleum Geology, 2014, 52: 93-106.

[3] Athersuch J, Banner F T, Higgins A C, et al.. The Application of Expert Systems to the Identification and Use of Microfossils in the Petroleum Industry. Mathematical Geology, 1994, 26(4): 483-489.

[4] Bazi Y, Bashmal L, Al Rahhal M M, et al.. Vision Transformers for Remote Sensing Image Classification. Remote Sensing, 2021, 13(3): 516.

[5] Beghin J, Storme J Y, Blanpied C, et al.. Microfossils from the Late Mesoproterozoic–Early Neoproterozoic Atar/El Mreïti Group, Taoudeni Basin, Mauritania, Northwestern Africa. Precambrian Research, 2017, 291: 63-82.

[6] Börner N, De Baere B, Yang Q C, et al.. Ostracod Shell Chemistry as Proxy for Paleoenvironmental Change. Quaternary International, 2013, 313: 17-37.

[7] Chakraborty A, Ghosh A K, Dey R, et al.. Record of the Miocene Climate Optimum in the Northeast Indian Ocean: Evidence from the Microfossils. Palaeobiodiversity and Palaeoenvironments, 2019, 99(2): 159-175.

[8] Choi B D, Jia B Y, Huh M, et al.. Taxonomy, Biostratigraphic and Paleoecological Aspects of Non-Marine Ostracod Fauna from the Jinju Formation (Albian) of the Gyeongsang Basin, South Korea. Cretaceous Research, 2021, 127: 104944.

[9] Choi B D, Wang Y Q. Nonmarine Ostracod Fauna from the Lower Cretaceous Shinekhudag Formation (Southwest Mongolia): Taxonomy, Biostratigraphy, and Paleoecology. Journal of Paleontology, 2023, 97(3): 612-630.

[10] Choi B D, Wang Y Q, Hu L, et al.. Ostracod Faunas from the Dalazi and Tongfosi Formations (Yanji Basin, Northeast China): Biostratigraphic, Palaeogeographic and Palaeoecological Implications. Cretaceous Research, 2020, 105: 104018.

[11] Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Jul. 21–26, 2017, 2017. Honolulu, HI, IEEE: 1800-1807.

[12] Cohen P A, Irvine S W, Strauss J V. Vase-Shaped Microfossils from the Tonian Callison Lake Formation of Yukon, Canada: Taxonomy, Taphonomy and Stratigraphic Palaeobiology. Palaeontology, 2017, 60(5): 683-701.

[13] Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al., 2020. An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale. arXiv: 2010.11929. https://arxiv.org/abs/2010.11929

[14] Fan X L, Yu P H, Zeng L, et al.. The Biostratigraphic and Chronological Research of Cenozoic in the Qaidam Basin, Northwest, China. Acta Micropalaeontologica Sinica, 2016, 33(4): 16. (in Chinese with English Abstract).

[15] Feng Z H, Fang W, Wang X, et al.. Microfossils and Molecular Records in Oil Shales of the Songliao Basin and Implications for Paleo-Depositional Environment. Science in China Series D: Earth Sciences, 2009, 52(10): 1559-1571.

[16] Ferreira-Chacua I, Koeshidayatullah A I. ForamViT-GAN: Exploring New Paradigms in Deep Learning for Micropaleontological Image Analysis. IEEE Access, 2023, 11: 67298-67307.

[17] Golubkova E Y, Kuzmenkova O F, Kushim E A, et al.. Distribution of Microfossils in the Vendian Deposits of the Orsha Depression of the East European Platform, Belarus. Stratigraphy and Geological Correlation, 2021, 29(6): 627-640.

[18] Golubkova E Y, Raevskaya E G, Kuznetsov A B. Lower Vendian Microfossil Assemblages of East Siberia: Significance for Solving Regional Stratigraphic Problems. Stratigraphy and Geological Correlation, 2010, 18(4): 353-375.

[19] Haleem A, Javaid M, Singh R P. An Era of ChatGPT as a Significant Futuristic Support Tool: A Study on Features, Abilities, and Challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2022, 2(4): 100089.

[20] Han K, Xiao A, Wu E, et al.. Transformer in Transformer. Advances in Neural Information Processing Systems, 2021, 34: 15908-15919

[21] He K M, Zhang X Y, Ren S Q, et al.. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Jun. 27–30, 2016, 2016. Las Vegas, NV, IEEE: 770-778.

[22] Horne D J. A Revised Ostracod Biostratigraphy for the Purbeck-Wealden of England. Cretaceous Research, 1995, 16(6): 639-663.

[23] Hou C B, Lin X Y, Huang H H, et al.. Fossil Image Identification Using Deep Learning Ensembles of Data Augmented Multiviews. Methods in Ecology and Evolution, 2023, 14(12): 3020-3034.

[24] Hou Y M, Canul-Ku M, Cui X D, et al.. Semantic Segmentation of Vertebrate Microfossils from Computed Tomography Data Using a Deep Learning Approach. Journal of Micropalaeontology, 2021, 40(2): 163-173.

[25] Howard, A. G., Zhu, M. L., Chen, B., et al., 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv: 1704.04861. https://arxiv.org/abs/1704.04861

[26] Hsiang A Y, Brombacher A, Rillo M C, et al.. Endless Forams: >34, 000 Modern Planktonic Foraminiferal Images for Taxonomic Training and Automated Species Recognition Using Convolutional Neural Networks. Paleoceanography and Paleoclimatology, 2019, 34(7): 1157-1177.

[27] Ide H, Kurita T. Improvement of Learning for CNN with ReLU Activation by Sparse Regularization. 2017 International Joint Conference on Neural Networks (IJCNN). May 14 - 19, 2017, 2017. Anchorage, AK, IEEE: 2684-2691.

[28] Jung H K, Choi G S. Improved YOLOv5: Efficient Object Detection Using Drone Images under Various Conditions. Applied Sciences, 2022, 12(14): 7255.

[29] Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks. Communications of the ACM, 2017, 60(6): 84-90.

[30] Lee B S, Cho S H, Choh S J, et al.. Reassessment of the Serratognathus bilobatus Conodont Biozone (Early Ordovician) of the Taebaek Group, Korea. Journal of Earth Science, 2025, 36(2): 373-381.

[31] Li D D, Luo G M, Tang Q, et al.. New Record of the Green Algal Fossil Proterocladus and Coexisting Microfossils from the Meso-Neoproterozoic Diaoyutai Formation in Southern Liaoning, North China. Precambrian Research, 2023, 393: 107104.

[32] Li H, Wang M, Li J Q, et al.. Geochemistry and Zircon U-Pb and Hf Isotopes of Early Devonian Hardawu Granites in the Eastern Segment of the Ultrahigh-Pressure Metamorphic Belt, Northern Qaidam Basin. Journal of Earth Science, 2024, 35(3): 866-877.

[33] Li M, Shao L Y, Lu J, et al.. Sequence Stratigraphy and Paleogeography of the Middle Jurassic Coal Measures in the Yuqia Coalfield, Northern Qaidam Basin, Northwestern China. AAPG Bulletin, 2014, 98(12): 2531-2550.

[34] Li X Z, Liu W G, Zhang L, et al.. Distribution of Recent Ostracod Species in the Lake Qinghai Area in Northwestern China and Its Ecological Significance. Ecological Indicators, 2010, 10(4): 880-890.

[35] Li X. Sr/Ca Ratio Characteristics of Cenozoic Ostracod Fossils in the Northwestern Qaidam Basin and Their Paleoenvironmental Significance, 2020. Lanzhou, Lanzhou University. (in Chinese with English Abstract).

[36] Liu H L, Zou C N, Zhu R K, et al.. Accumulation Mechanism of Organic Matters in Paleogene Qaidam Basin, Northwestern China. Journal of Earth Science, 2025, 36(5): 2117-2137.

[37] Liu X K, Jiang S Y, Wu R, et al.. Automatic Taxonomic Identification Based on the Fossil Image Dataset (>415, 000 Images) and Deep Convolutional Neural Networks. Paleobiology, 2023, 49(1): 1-22.

[38] Liu X K, Song H J. Automatic Identification of Fossils and Abiotic Grains during Carbonate Microfacies Analysis Using Deep Convolutional Neural Networks. Sedimentary Geology, 2020, 410: 105790.

[39] Liu Z, Lin Y T, Cao Y, et al.. Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Oct. 10–17, 2021, 2021. Montreal, QC, IEEE: 9992-10002

[40] Lu Y Z, Chen D, Olaniyi E, et al.. Generative Adversarial Networks (GANs) for Image Augmentation in Agriculture: A Systematic Review. Computers and Electronics in Agriculture, 2022, 200: 107208.

[41] Manzari O N, Ahmadabadi H, Kashiani H, et al.. MedViT: A Robust Vision Transformer for Generalized Medical Image Classification. Computers in Biology and Medicine, 2023, 157: 106791.

[42] Marchant R, Tetard M, Pratiwi A, et al.. Automated Analysis of Foraminifera Fossil Records by Image Classification Using a Convolutional Neural Network. Journal of Micropalaeontology, 2020, 39(2): 183-202.

[43] McFarland A J. Ostracode Paleoecology of the Eocene Green River Formation, Fossil Basin, Wyoming, 2012. Lincoln, Nebraska, University of Nebraska-Lincoln: 85

[44] Miele, V., Dussert, G., Cucchi, T., et al., 2020. Deep Learning for Species Identification of Modern and Fossil Rodent Molars. BioRxiv, 2020–08. https://doi.org/10.1101/2020.08.20.259176

[45] Perez, L., Wang, J., 2017. The Effectiveness of Data Augmentation in Image Classification Using Deep Learning. arXiv: 1712.04621. https://arxiv.org/abs/1712.04621

[46] Peyerl D, Bosetti E P. Technique and Exploration: The Beginning of Micropaleontology in the Brazilian Oil Industry. History, Exploration & Exploitation of Oil and Gas, 2019. Cham, Springer International Publishing: 59-69.

[47] Pires de Lima R, Welch K F, Barrick J E, et al.. Convolutional Neural Networks as an Aid to Biostratigraphy and Micropaleontology: A Test on Late Paleozoic Microfossils. Palaios, 2020, 35(9): 391-402.

[48] Pirkenseer C, Spezzaferri S, Berger J P. Reworked Microfossils as a Paleogeographic Tool. Geology, 2011, 39(9): 843-846.

[49] Poropat S F, Colin J P. Early Cretaceous Ostracod Biostratigraphy of Eastern Brazil and Western Africa: An Overview. Gondwana Research, 2012, 22(3/4): 772-798.

[50] Qin Z H, Xi D P, Choi B D, et al.. Lowermost Occurrence of Ostracod Cypridea Species in East Asia and Implications for the Non-Marine Jurassic/Cretaceous Boundary. Palaeoworld, 2021, 30(1): 148-168.

[51] Qu H Y, Xi D P, Li S, et al.. Late Cretaceous–Early Paleocene Ostracod Biostratigraphy of Scientific Drilling SK1 (N) in the Songliao Basin, Northeast China. Journal of Paleontology, 2014, 88(4): 786-799.

[52] Rahali A, Akhloufi M A. End-to-End Transformer-Based Models in Textual-Based NLP. AI, 2023, 4(1): 54-110.

[53] Rasmussen B, Muhling J R, Fischer W W. Ancient Oil as a Source of Carbonaceous Matter in 1.88-Billion-Year-Old Gunflint Stromatolites and Microfossils. Astrobiology, 2021, 21(6): 655-672.

[54] Rehn E, Rehn A, Possemiers A. Fossil Charcoal Particle Identification and Classification by Two Convolutional Neural Networks. Quaternary Science Reviews, 2019, 226: 106038.

[55] Şafak Ü, Güldürek M, Nurlu N, et al.. Micropaleontological (Ostracoda) Content and Mineralogical Properties of the Neogene Ergene Formation (SW Thrace Region): Implications for the Evolution of Thrace Basin. Arabian Journal of Geosciences, 2022, 15(6): 515.

[56] Schopf J W, Kudryavtsev A B, Czaja A D, et al.. Evidence of Archean Life: Stromatolites and Microfossils. Precambrian Research, 2007, 158(3/4): 141-155.

[57] Sergeev V N. The Importance of Precambrian Microfossils for Modern Biostratigraphy. Paleontological Journal, 2006, 40(S5): S664-S673.

[58] Selvaraju R R, Cogswell M, Das A, et al.. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV). Oct. 22–29, 2017, 2017618-626

[59] Sharma N, Jain V, Mishra A. An Analysis of Convolutional Neural Networks for Image Classification. Procedia Computer Science, 2018, 132: 377-384.

[60] Simonyan, K., Zisserman, A., 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv: 1409.1556. https://arxiv.org/abs/1409.1556

[61] Song B W, Ji J L, Wang C W, et al.. Intensified Aridity in the Qaidam Basin during the Middle Miocene: Constraints from Ostracod, Stable Isotope, and Weathering Records. Canadian Journal of Earth Sciences, 2017, 54(3): 242-256.

[62] Song B W, Zhang K X, Han F, et al.. Reconstruction of the Latest Eocene–Early Oligocene Paleoenvironment in the Hoh Xil Basin (Central Tibet) Based on Palynological and Ostracod Records. Journal of Asian Earth Sciences, 2021, 217: 104860.

[63] Strauss J V, Rooney A D, MacDonald F A, et al.. 740 Ma Vase-Shaped Microfossils from Yukon, Canada: Implications for Neoproterozoic Chronology and Biostratigraphy. Geology, 2014, 42(8): 659-662.

[64] Sun Z C, Feng X J, Li D M, et al.. Cenozoic Ostracoda and Palaeoenvironments of the Northeastern Tarim Basin, Western China. Palaeogeography, Palaeoclimatology, Palaeoecology, 1999, 148(1/2/3): 37-50.

[65] Szegedy C, Liu W, Jia Y Q, et al.. Going Deeper with Convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Jun. 7–12, 2015, 2015. Boston, MA, IEEE: 1-9

[66] Tang P, Yang P L, Nie D, et al.. Unified Medical Image Segmentation by Learning from Uncertainty in an End-to-End Manner. Knowledge-Based Systems, 2022, 241: 108215.

[67] Tang W Q, Yi F, Chen X D, et al.. Abrupt Aridification in the Upper Eocene of the Western Qaidam Basin, Northeastern Tibetan Plateau. Palaeogeography, Palaeoclimatology, Palaeoecology, 2021, 577: 110515.

[68] Tang W Q, Zhang D W, Zhou Y X, et al.. Astronomical Forcing in the Coal-Bearing Middle Jurassic Dameigou Formation, Qaidam Basin, Northwestern China. Ore Geology Reviews, 2023, 161: 105663.

[69] Touvron H, Cord M, Douze M, et al.. Training Data-Efficient Image Transformers & Distillation through Attention. International Conference on Machine Learning (ICML). Jul. 18–24, 2021, 202110347-10357

[70] Vaswani A, Shazeer N, Parmar N, et al.. Attention is all You need. Proceedings of the 31st International Conference on Neural Information Processing Systems, 20176000-6010

[71] Wang F, Jiang M Q, Qian C, et al.. Residual Attention Network for Image Classification. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Jul. 21–26, 2017, 2017. Honolulu, HI, IEEE: 6450-6458.

[72] Wang H Z, Li C F, Zhang Z F, et al.. Fossil Brachiopod Identification Using a New Deep Convolutional Neural Network. Gondwana Research, 2022, 105: 290-298.

[73] Wang W H, Xie E Z, Li X, et al.. Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions. IEEE/CVF International Conference on Computer Vision (ICCV), 2021. Montreal, QC, IEEE: 548-558

[74] Wang W T, Zhang P Z, Duan L, et al.. Cenozoic Stratigraphic Chronology and Sedimentary-Tectonic Evolution of the Qaidam Basin. Chinese Science Bulletin, 2022, 67(28/29): 3452-3475. in Chinese).

[75] Wang Y Q, Olsen P E, Sha J G, et al.. Stratigraphy, Correlation, Depositional Environments, and Cyclicity of the Early Cretaceous Yixian and Jurassic-Cretaceous Tuchengzi Formations in the Sihetun Area (NE China) Based on Three Continuous Cores. Palaeogeography, Palaeoclimatology, Palaeoecology, 2016, 464: 110-133.

[76] Wei, J., Tay, Y., Bommasani, R., et al., 2022. Emergent Abilities of Large Language Models. arXiv preprint:2206.07682

[77] Willman S, Slater B J. Late Ediacaran Organic Microfossils from Finland. Geological Magazine, 2021, 158(12): 2231-2244.

[78] Xie X W, Pan X P, Zhang W D, et al.. A Context Hierarchical Integrated Network for Medical Image Segmentation. Computers and Electrical Engineering, 2022, 101: 108029.

[79] Yu C Y, Qin F B, Watanabe A, et al.. Artificial Intelligence in Paleontology. Earth-Science Reviews, 2024, 252: 104765.

[80] Yuan L, Chen Y P, Wang T, et al.. Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Oct. 10–17, 2021, 2021. Montreal, QC, IEEE: 538-547

[81] Zarei E, Ghasemi-Nejad E. Sequence Stratigraphy of the Gurpi Formation (Campanian–Paleocene) in Southwest of Zagros, Iran, Based on Palynomorphs and Foraminifera. Arabian Journal of Geosciences, 2015, 8(6): 4011-4023.

[82] Zhang W. The High Precise Cenozoic Magnetostratigraphy of the Qaidam Basin and Uplift of the Northern Tibetan Plateau, 2006. Lanzhou, Lanzhou University. (in Chinese with English Abstract).

[83] Zhang Y C, Zhang W B, Yu J Y, et al.. Complete and Accurate Holly Fruits Counting Using YOLOX Object Detection. Computers and Electronics in Agriculture, 2022, 198: 107062.

[84] Zhao D D, Ma H, Yang Z D, et al.. Finger Vein Recognition Based on Lightweight CNN Combining Center Loss and Dynamic Regularization. Infrared Physics & Technology, 2020, 105: 103221.

[85] Zhao Z Y, Lu Y Y, Tong Y J, et al.. PENet: A Phenotype Encoding Network for Automatic Extraction and Representation of Morphological Discriminative Features. Methods in Ecology and Evolution, 2023, 14(12): 3035-3046.

[86] Zheng D Y, Wu S X, Ma C, et al.. Zircon Classification from Cathodoluminescence Images Using Deep Learning. Geoscience Frontiers, 2022, 13(6): 101436.

[87] Zhou N R, Zhang T F, Xie X W, et al.. Hybrid Quantum-Classical Generative Adversarial Networks for Image Generation via Learning Discrete Distribution. Signal Processing: Image Communication, 2023, 110: 116891

RIGHTS & PERMISSIONS

China University of Geosciences (Wuhan) and Springer-Verlag GmbH Germany, Part of Springer Nature