Model reduction of parametric ordinary differential equations via autoencoders: representation properties and convergence analysis

Abstract

We propose a reduced-order modeling approach for nonlinear, parameter-dependent ordinary differential equations (ODE). Dimensionality reduction is achieved using nonlinear maps represented by autoencoders. The resulting low-dimensional ODE is then solved using standard integration in time schemes, and the high-dimensional solution is reconstructed from the low- dimensional one. We investigate the architecture of neural networks for constructing effective autoencoders that hold necessary properties to reconstruct the input manifold with exact representation capabilities. We study the convergence of the reduced-order model to the high-fidelity one. Numerical experiments show the robustness and accuracy of our approach in different scenarios, highlighting its effectiveness in highly complex and nonlinear settings without sacrificing accuracy. Moreover, we examine how the reduction influences the stability properties of the reconstructed high-dimensional solution.

Keywords

reduced order modeling / parametric differential equations / autoencoders / nonlinear dimensionality reduction / latent dynamics / manifold learning / architecture analysis / convergence analysis / stability analysis / scientific machine learning

References

[1] J. Ansel, E. Yang, H. He, N. Gimelshein, A. Jain, M. Voznesensky, et al., PyTorch 2: faster machine learning through dynamic Python bytecode transformation and graph compilation, 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, 2 (2024), 929-947. https://doi.org/10.1145/3620665.3640366

[2] E. Ballini, A. Cominelli, L. Dovera, A. Forello, L. Formaggia, A. Fumagalli, et al., Enhancing computational efficiency of numerical simulation for subsurface fluid-induced deformation using deep learning reduced order models, SPE Reservoir Simulation Conference, 2025. https://doi.org/10.2118/223897-MS

[3] E. Ballini, A. Cominelli, L. Dovera, A. Forello, L. Formaggia, A. Fumagalli, et al., Enhancing computational efficiency of numerical simulation for subsurface fluid-induced deformation using deep learning reduced order models, Comput. Geosci., 30 (2026), 14. https://doi.org/10.1007/s10596-026-10409-6

[4] E. Ballini, L. Formaggia, A. Fumagalli, A. Scotti, P. Zunino, Application of deep learning reducedorder modeling for single-phase flow in faulted porous media, Comput. Geosci., 28 (2024), 1279-1303. https://doi.org/10.1007/s10596-024-10320-y

[5] J. Barnett, C. Farhat, Y. Maday, Neural-network-augmented projection-based model order reduction for mitigating the Kolmogorov barrier to reducibility, J. Comput. Phys., 492 (2023), 112420. https://doi.org/10.1016/j.jcp.2023.112420

[6] J. C. Butcher, Numerical methods for ordinary differential equations, John Wiley & Sons, Ltd., 2016. https://doi.org/10.1002/9781119121534

[7] Y. Cai, Achieve the minimum width of neural networks for universal approximation, arXiv, 2022. https://doi.org/10.48550/arXiv.2209.11395

[8] P. Y. Chen, J. Xiang, D. H. Cho, Y. Chang, G. A. Pershing, H. T. Maia, et al., CROM: Continuous reduced-order modeling of PDEs using implicit neural representations, International Conference on Learning Representations (ICLR), 2023, 1-41.

[9] R. T. Q. Chen, B. Amos, M. Nickel, Learning neural event functions for ordinary differential equations, arXiv, 2020. https://doi.org/10.48550/arXiv.2011.03902

[10] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, D. K. Duvenaud, Neural ordinary differential equations, In: S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett, Advances in neural information processing systems, 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Vol. 31, 2018.

[11] P. Conti, M. Guo, A. Manzoni, A. Frangi, S. L. Brunton, J. Nathan Kutz, Multi-fidelity reducedorder surrogate modelling, Proc. A, 480 (2283), 2024. https://doi.org/10.1098/rspa.2023.0655

[12] Q. Du, Y. Gu, H. Yang, C. Zhou, The discovery of dynamics via linear multistep methods and deep learning: error estimation, SIAM J. Numer. Anal., 60 (2022), 2014-2045. https://doi.org/10.1137/21M140691X

[13] R. D’Ambrosio, S. D. Giovacchino, Mean-square contractivity of stochastic theta-methods, Commun. Nonlinear Sci. Numer. Simul., 96 (2021), 105671. https://doi.org/10.1016/j.cnsns.2020.105671

[14] N. Farenga, S. Fresca, S. Brivio, A. Manzoni, On latent dynamics learning in nonlinear reduced order modeling, Neural Networks, 185 (2025), 107146. https://doi.org/10.1016/j.neunet.2025.107146

[15] D. Floryan, M. D. Graham, Data-driven discovery of intrinsic dynamics, Nat. Mach. Intell., 4 (2022), 1113-1120. https://doi.org/10.1038/s42256-022-00575-4

[16] N. R. Franco, S. Fresca, F. Tombari, A. Manzoni, Deep learning-based surrogate models for parametrized PDEs: Handling geometric variability through graph neural networks, Chaos, 33 (2023), 123121. https://doi.org/10.1063/5.0170101

[17] N. R. Franco, A. Manzoni, P. Zunino, A deep learning approach to reduced order modelling of parameter dependent partial differential equations, Math. Comput., 92 (2022), 483-524.

[18] N. R. Franco, A. Manzoni, P. Zunino, Mesh-informed neural networks for operator learning in finite element spaces, J. Sci. Comput., 97 (2023), 35. https://doi.org/10.1007/s10915-023-02331-1

[19] S. Fresca, L. Dede’, A. Manzoni, A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs, J. Sci. Comput., 87 (2021), 61. https://doi.org/10.1007/s10915-021-01462-7

[20] S. Fresca, F. Fatone, A. Manzoni, Long-time prediction of nonlinear parametrized dynamical systems by deep learning-based reduced order models, Math. Eng., 5 (2023), 1-36. https://doi.org/10.3934/mine.2023096

[21] S. Fresca, A. Manzoni, POD-DL-ROM: Enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition, Comput. Methods Appl. Mech. Eng., 388 (2022), 114181. https://doi.org/10.1016/j.cma.2021.114181

[22] R. Fu, D. Xiao, I. Navon, F. Fang, L. Yang, C. Wang, et al., A non-linear non-intrusive reduced order model of fluid flow by auto-encoder and self-attention deep learning methods, Int. J. Numer. Methods Eng., 124 (2023), 3087-3111. https://doi.org/10.1002/nme.7240

[23] A. Fumagalli, A. Scotti, A mathematical model for thermal single-phase flow and reactive transport in fractured porous media, J. Comput. Phys., 434 (2021), 110205. https://doi.org/10.1016/j.jcp.2021.110205

[24] F. J. Gonzalez, M. Balajewicz, Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systems, arXiv, 2018. https://doi.org/10.48550/arXiv.1808.01346

[25] I. Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT Press, 2017.

[26] R. Gupta, R. Jaiman, Three-dimensional deep learning-based reduced order model for unsteady flow dynamics with variable Reynolds number, Phys. Fluids, 34 (2022), 033612. https://doi.org/10.1063/5.0082741

[27] I. Gühring, M. Raslan, Approximation rates for neural networks with encodable weights in smoothness spaces, Neural Networks, 134 (2021), 107-130. https://doi.org/10.1016/j.neunet.2020.11.010

[28] E. Hairer, G. Wanner, Solving ordinary differential equations II: stiff and differential-algebraic problems, Springer Series in Computational Mathematics, Vol. 14, Springer Berlin, Heidelberg, 1996. https://doi.org/10.1007/978-3-642-05221-7

[29] B. Hanin, M. Sellke, Approximating continuous functions by ReLU nets of minimal width, arXiv, 2017. https://doi.org/10.48550/arXiv.1710.11278

[30] K. Hasegawa, K. Fukami, T. Murata, K. Fukagata, Machine-learning-based reduced-order modeling for unsteady flows around bluff bodies of various shapes, Theor. Comput. Fluid Dyn., 34 (2020), 367-383. https://doi.org/10.1016/0378-4754(92)90035-F

[31] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90

[32] J. S. Hesthaven, G. Rozza, B. Stamm, Certified reduced basis methods for parametrized partial differential equations, Springer Cham, 2016. https://doi.org/10.1007/978-3-319-22470-1

[33] T. Kadeethum, F. Ballarin, Y. Choi, D. O’Malley, H. Yoon, N. Bouklas, Non-intrusive reduced order modeling of natural convection in porous media using convolutional autoencoders: comparison with linear subspace techniques, Adv. Water Resour., 160 (2022), 104098. https://doi.org/10.1016/j.advwatres.2021.104098

[34] P. Kidger, On neural differential equations, Ph.D. Thesis, University of Oxford, 2021.

[35] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv, 2014. https://doi.org/10.48550/arXiv.1412.6980

[36] A. Kumar, R. Hu, S. D. C. Walsh, Development of reduced order hydro-mechanical models of fractured media, Rock Mech. Rock Eng., 55 (2021), 235-248. https://doi.org/10.1007/s00603-021- 02668-9

[37] J. D. Lambert, Numerical methods for ordinary differential systems, Wiley, 1991.

[38] J. M. Lee, Introduction to topological manifolds, New York: Springer, 2000.

[39] K. Lee, K. T. Carlberg, Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders, J. Comput. Phys., 404 (2020), 108973. https://doi.org/10.1016/j.jcp.2019.108973

[40] C. Legaard, T. Schranz, G. Schweiger, J. Drgoˇna, B. Falay, C. Gomes, et al., Constructing neural network based models for simulating dynamical systems, ACM Comput. Surv., 55 (2023), 1-34. https://doi.org/10.1145/3567591

[41] F. Li, Y. Zhang, C. Xiao, Surrogate-based hydraulic fracture propagation prediction using deep neural network proxy, 58th U.S. Rock Mechanics/Geomechanics Symposium, Golden, Colorado, 2024. https://doi.org/10.56952/ARMA-2024-0056

[42] L. Li, Y. Duan, G. Ji, Y. Cai, Minimum width of leaky-ReLU neural networks for uniform universal approximation, arXiv, 2023. https://doi.org/10.48550/arXiv.2305.18460

[43] Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, et al., Fourier neural operator for parametric partial differential equations, arXiv, 2020. https://doi.org/10.48550/arXiv.2010.08895

[44] Z. Li, S. Patil, F. Ogoke, D. Shu, W. Zhen, M. Schneier, et al., Latent neural PDE solver: a reducedorder modeling framework for partial differential equations, J. Comput. Phys., 524 (2025), 113705. https://doi.org/10.1016/j.jcp.2024.113705

[45] J. Lu, Z. Shen, H. Yang, S. Zhang, Deep network approximation for smooth functions, SIAM J. Math. Anal., 53 (2021), 5465-5506. https://doi.org/10.1137/20M134695X

[46] L. Lu, P. Jin, G. Pang, Z. Zhang, G. E. Karniadakis, Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators, Nat. Mach. Intell., 3 (2021), 218-229. https://doi.org/10.1038/s42256-021-00302-5

[47] Y. Lu, A. Zhong, Q. Li, B. Dong, Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations, In: J. Dy, A. Krause, Proceedings of the 35th International Conference on Machine Learning, Volume 80: International Conference on Machine Learning, 2018, 3276-3285.

[48] J. E. Marsden, T. J. R. Hughes, Mathematical foundations of elasticity, Dover Civil and Mechanical Engineering, Dover Publications, New York, 1994.

[49] J. R. Munkres, Topology, Prentice Hall, Inc., 2000.

[50] T. Murata, K. Fukami, K. Fukagata, Nonlinear mode decomposition with convolutional neural networks for fluid dynamics, J. Fluid Mech., 882 (2019), A13. https://doi.org/10.1017/jfm.2019.822

[51] J. M. Nordbotten, M. A. Celia, Geological storage of CO₂ modeling approaches for large-scale simulation , Wiley, 2011.

[52] M. Ohlberger, S. Rave, Reduced basis methods: success, limitations and future challenges, Proceedings of the Conference Algoritmy, 2016, 1-12.

[53] P. Pant, R. Doshi, P. Bahl, A. Barati Farimani, Deep learning for reduced order modelling and efficient temporal evolution of fluid simulations, Phys. Fluids, 33 (2021), 107101. https://doi.org/10.1063/5.0062546

[54] S. Park, C. Yun, J. Lee, J. Shin, Minimum width for universal approximation, arXiv, 2020. https://doi.org/10.48550/arXiv.2006.08859

[55] M. Pintore, B. Despr´es, Computable lipschitz bounds for deep neural networks, arXiv, 2024. https://doi.org/10.48550/arXiv.2410.21053

[56] A. Quarteroni, A. Manzoni, F. Negri, Reduced basis methods for partial differential equations: an introduction, Vol. 92, Springer Cham, 2016. https://doi.org/10.1007/978-3-319-15431-2

[57] A. Quarteroni, R. Sacco, F. Saleri, Numerical mathematics, Vol. 37, Springer Berlin, Heidelberg, 2007. https://doi.org/10.1007/b98885

[58] M. Raissi, P. Perdikaris, G. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys., 378 (2019), 686-707. https://doi.org/10.1016/j.jcp.2018.10.045

[59] F. Regazzoni, S. Pagani, M. Salvador, L. Dede’, A. Quarteroni, Learning the intrinsic dynamics of spatio-temporal processes through latent dynamics networks, Nat. Commun., 15 (2024), 1834 . https://doi.org/10.1038/s41467-024-45323-x

[60] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, In: N. Navab, J. Hornegger, W. Wells, A. Frangi, Medical image computing and computer-assisted intervention - MICCAI 2015, Springer, Cham, 9351 (2015), 234-241. https://doi.org/10.1007/978-3-319-24574-4 28

[61] Y. Rubanova, R. T. Q. Chen, D. Duvenaud, Latent ODEs for irregularly-sampled time series, Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019, 5320-5330.

[62] A. Sabzi Shahrebabaki, S. Fouladi, E. Holtar, L. Vynnytska, Autoencoder-based generation of subsurface models, Second EAGE Digitalization Conference and Exhibition, European Association of Geoscientists & Engineers, 2022 (2022), 1-5. https://doi.org/10.3997/2214-4609.202239082

[63] C. I. Steefel, A. C. Lasaga, A coupled model for transport of multiple chemical species and kinetic precipitation/dissolution reactions with application to reactive flow in single phase hydrothermal systems, Amer. J. Sci., 294 (1994), 529-592.

[64] P. A. Thompson, Compressible-fluid dynamics, McGraw-Hill, Inc., 1972.

[65] T. Tripura, S. Chakraborty, Wavelet neural operator for solving parametric partial differential equations in computational mechanics problems, Comput. Methods Appl. Mech. Eng., 404 (2023), 115783. https://doi.org/10.1016/j.cma.2022.115783

[66] W. I. T. Uy, D. Hartmann, B. Peherstorfer, Operator inference with roll outs for learning reduced models from scarce and low-quality data, Comput. Math. Appl., 145 (2023), 224-239. https://doi.org/10.1016/j.camwa.2023.06.012

[67] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, et al., SciPy 1.0: Fundamental algorithms for scientific computing in Python, Nat. Methods, 17 (2020), 261-272. https://doi.org/10.1038/s41592-019-0686-2

[68] G. Ziarelli, N. Parolini, M. Verani, Learning epidemic trajectories through kernel operator learning: from modelling to optimal control, Numer. Math., 18 (2025), 286-324.