Transformer-based DNA methylation detection on ionic signals from Oxford Nanopore sequencing data

Abstract

Background: Oxford Nanopore long-read sequencing technology addresses current limitations for DNA methylation detection that are inherent in short-read bisulfite sequencing or methylation microarrays. A number of analytical tools, such as Nanopolish, Guppy/Tombo and DeepMod, have been developed to detect DNA methylation on Nanopore data. However, additional improvements can be made in computational efficiency, prediction accuracy, and contextual interpretation on complex genomics regions (such as repetitive regions, low GC density regions).

Method: In the current study, we apply Transformer architecture to detect DNA methylation on ionic signals from Oxford Nanopore sequencing data. Transformer is an algorithm that adopts self-attention architecture in the neural networks and has been widely used in natural language processing.

Results: Compared to traditional deep-learning method such as convolutional neural network (CNN) and recurrent neural network (RNN), Transformer may have specific advantages in DNA methylation detection, because the self-attention mechanism can assist the relationship detection between bases that are far from each other and pay more attention to important bases that carry characteristic methylation-specific signals within a specific sequence context.

Conclusion: We demonstrated the ability of Transformers to detect methylation on ionic signal data.

Graphical abstract

Keywords

Nanopore / long-read sequencing / deep learning / Transformer model / DNA methylation.

References

[1] Bell,J. (2021). Genetic impacts on DNA methylation: research findings and future perspectives. Genome Biol., 22: 127

[2] Kulis,M. (2010). DNA methylation and cancer. Adv. Genet., 70: 27–56

[3] Jin,B. Robertson,K. (2013). DNA methyltransferases, DNA damage repair, and cancer. Adv. Exp. Med. Biol., 754: 3–29

[4] Bernstein,C., Nfonsam,V., Prasad,A. R. (2013). Epigenetic field defects in progression to cancer. World J. Gastrointest. Oncol., 5: 43–49

[5] nez-Iglesias,O., Carrera,I., Carril,J. C., ndez-Novoa,L., Cacabelos,N. (2020). DNA methylation in neurodegenerative and cerebrovascular disorders. Int. J. Mol. Sci., 21: 2220

[6] Jeong,H., Mendizabal,I., Berto,S., Chatterjee,P., Layman,T., Usui,N., Toriumi,K., Douglas,C., Singh,D., Huh,I. . (2021). Evolution of DNA methylation in the human brain. Nat. Commun., 12: 2021

[7] Jobe,E. M. (2017). DNA methylation and adult neurogenesis. Brain Plast., 3: 5–26

[8] Tognini,P., Napoli,D. (2015). Dynamic DNA methylation in the brain: a new epigenetic mark for experience-dependent plasticity. Front. Cell. Neurosci., 9: 331

[9] McCoy,C. R., Glover,M. E., Flynn,L. T., Simmons,R. K., Cohen,J. L., Ptacek,T., Lefkowitz,E. J., Jackson,N. L., Akil,H., Wu,X. . (2019). Altered DNA methylation in the developing brains of rats genetically prone to high versus low anxiety. J. Neurosci., 39: 3144–3158

[10] Jones,P. A., Issa,J. P. (2016). Targeting the cancer epigenome for therapy. Nat. Rev. Genet., 17: 630–641

[11] Mani,S. (2010). DNA demethylating agents and epigenetic therapy of cancer. Adv. Genet., 70: 327–340

[12] Issa,J. P., Garcia-Manero,G., Giles,F. J., Mannari,R., Thomas,D., Faderl,S., Bayar,E., Lyons,J., Rosenfeld,C. S., Cortes,J. . (2004). Phase 1 study of low-dose prolonged exposure schedules of the hypomethylating agent 5-aza-2′-deoxycytidine (decitabine) in hematopoietic malignancies. Blood, 103: 1635–1640

[13] Ding,X. L., Yang,X., Liang,G. (2016). Isoform switching and exon skipping induced by the DNA methylation inhibitor 5-Aza-2′-deoxycytidine. Sci. Rep., 6: 24545

[14] Ovenden,E. S., McGregor,N. W., Emsley,R. A. (2018). DNA methylation and antipsychotic treatment mechanisms in schizophrenia: progress and future directions. Prog. Neuropsychopharmacol. Biol. Psychiatry, 81: 38–49

[15] Clark,T. A., Lu,X., Luong,K., Dai,Q., Boitano,M., Turner,S. W., He,C. (2013). Enhanced 5-methylcytosine detection in single-molecule, real-time sequencing via Tet1 oxidation. BMC Biol., 11: 4

[16] Beaulaurier,J., Zhang,X. Zhu,S., Sebra,R., Rosenbluh,C., Deikus,G., Shen,N., Munera,D., Waldor,M. K., Chess,A. . (2015). Single molecule-level detection and long read-based phasing of epigenetic variations in bacterial methylomes. Nat. Commun., 6: 7438

[17] Liu,Q., Georgieva,D. C., Egli,D. (2019). NanoMod: a computational tool to detect DNA modifications using Nanopore long-read sequencing data. BMC Genomics, 20: 78

[18] Simpson,J. T., Workman,R. E., Zuzarte,P. C., David,M., Dursi,L. J. (2017). Detecting DNA cytosine methylation using Nanopore sequencing. Nat. Methods, 14: 407–410

[19] Pimiento,C., Ehret,D. J., Macfadden,B. J. (2010). Ancient nursery area for the extinct giant shark megalodon from the Miocene of Panama. PLoS One, 5: e10552

[20] Ni,P., Huang,N., Zhang,Z., Wang,D. Liang,F., Miao,Y., Xiao,C. Luo,F. (2019). DeepSignal: detecting DNA methylation state from Nanopore sequencing reads using deep-learning. Bioinformatics, 35: 4586–4595

[21] Weirather,J. L., de Cesare,M., Wang,Y., Piazza,P., Sebastiano,V., Wang,X. Buck,D. Au,K. (2017). Comprehensive comparison of Pacific Biosciences and Oxford Nanopore Technologies and their applications to transcriptome analysis. F1000 Res., 6: 100

[22] Yuen,Z. W. Srivastava,A., Daniel,R., McNevin,D., Jack,C. (2021). Systematic benchmarking of tools for CpG methylation detection from Nanopore sequencing. Nat. Commun., 12: 3438

[23] Liu,Q., Fang,L., Yu,G., Wang,D., Xiao,C. (2019). Detection of DNA base modifications by deep recurrent neural network on Oxford Nanopore sequencing data. Nat. Commun., 10: 2449

[24] Liu,Y., Rosikiewicz,W., Pan,Z., Jillette,N., Wang,P., Taghbalout,A., Foox,J., Mason,C., Carroll,M., Cheng,A. . (2021). DNA methylation-calling tools for Oxford Nanopore sequencing: a survey and human epigenome-wide evaluation. Genome Biol., 22: 295

[25] ZhangY.Yamaguchi K.,HatakeyamaS.,FurukawaY.,MiyanoS., YamaguchiR.. (2021) On the application of bert models for Nanopore methylation detection. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 320–327

[26] Jiao,L., Zhang,F., Liu,F., Yang,S., Li,L., Feng,Z. (2019). A survey of deep learning-based object detection. IEEE Access, 7: 128837–128868

[27] Amarasinghe,S. L., Su,S., Dong,X., Zappia,L., Ritchie,M. E. (2020). Opportunities and challenges in long-read sequencing data analysis. Genome Biol., 21: 30

[28] DevlinJ.,Chang M.LeeK.. (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv, 181004805

[29] VaswaniA.,Shazeer N. M.,ParmarN.,UszkoreitJ.,JonesL., GomezA. N.,Kaiser L.. (2017) Attention is all you need. arXiv, 1706.03762

RIGHTS & PERMISSIONS

The Author(s). Published by Higher Education Press.