Experimental design and model reduction in systems biology

Abstract

Background: In systems biology, the dynamics of biological networks are often modeled with ordinary differential equations (ODEs) that encode interacting components in the systems, resulting in highly complex models. In contrast, the amount of experimentally available data is almost always limited, and insufficient to constrain the parameters. In this situation, parameter estimation is a very challenging problem. To address this challenge, two intuitive approaches are to perform experimental design to generate more data, and to perform model reduction to simplify the model. Experimental design and model reduction have been traditionally viewed as two distinct areas, and an extensive literature and excellent reviews exist on each of the two areas. Intriguingly, however, the intrinsic connections between the two areas have not been recognized.

Results: Experimental design and model reduction are deeply related, and can be considered as one unified framework. There are two recent methods that can tackle both areas, one based on model manifold and the other based on profile likelihood. We use a simple sum-of-two-exponentials example to discuss the concepts and algorithmic details of both methods, and provide Matlab-based code and implementation which are useful resources for the dissemination and adoption of experimental design and model reduction in the biology community.

Conclusions: From a geometric perspective, we consider the experimental data as a point in a high-dimensional data space and the mathematical model as a manifold living in this space. Parameter estimation can be viewed as a projection of the data point onto the manifold. By examining the singularity around the projected point on the manifold, we can perform both experimental design and model reduction. Experimental design identifies new experiments that expand the manifold and remove the singularity, whereas model reduction identifies the nearest boundary, which is the nearest singularity that suggests an appropriate form of a reduced model. This geometric interpretation represents one step toward the convergence of experimental design and model reduction as a unified framework.

Author summary

In systems biology, a common challenge is that models are often highly complex while data is almost always insufficient. Two intuitive strategies to address this challenge are experimental design (obtain more data to improve parameter estimation) and model reduction (simplify the model to reveal key mechanism). In the literature, those two have been viewed as distinct areas. We present a geometric framework to connect the two areas. We consider a model as a manifold, and explore its geometry to perform experimental design and model reduction. This framework is interesting because of both its mathematical beauty (unifying two seemingly distinct areas) and its potential impact to biology (helping biologists to design experiments and find important mechanisms).

Graphical abstract

Keywords

experimental design / model reduction / model manifold / profile likelihood

References

[1] Lander, A. D. (2004) A calculus of purpose. PLoS Biol., 2, e164

[2] Sobie, E. A., Lee, Y. S., Jenkins, S. L. and Iyengar, R. (2011) Systems biology‒biomedical modeling. Sci. Signal., 4, tr2

[3] Fages, F., Gay, S. and Soliman, S. (2015) Inferring reaction systems from ordinary differential equations. Theor. Comput. Sci., 599, 64–78

[4] Jha, S. K. and Langmead, C. J. (2012) Exploring behaviors of stochastic differential equation models of biological systems using change of measures. BMC Bioinformatics, 13, S8

[5] Kauffman, S. A. (1969) Metabolic stability and epigenesis in randomly constructed genetic nets. J. Theor. Biol., 22, 437–467

[6] Sachs, K., Gifford, D., Jaakkola, T., Sorger, P. and Lauffenburger, D. A. (2002) Bayesian network approach to cell signaling pathway modeling. Sci. STKE, 2002, pe38

[7] Koch, I. (2015) Petri nets in systems biology. Soft. Syst. Model., 14, 703–710

[8] Materi, W. and Wishart, D. S. (2007) Computational systems biology in drug discovery and development: methods and applications. Drug Discov. Today, 12, 295–303

[9] Machado, D., Costa, R. S., Rocha, M., Ferreira, E. C., Tidor, B. and Rocha, I. (2011) Modeling formalisms in systems biology. AMB Express, 1, 45

[10] Bartocci, E. and Lió, P. (2016) Computational modeling, formal analysis, and tools for systems biology. PLoS Comput. Biol., 12, e1004591

[11] Kitano, H. (2002) Computational systems biology. Nature, 420, 206–210

[12] Aldridge, B. B., Burke, J. M., Lauffenburger, D. A. and Sorger, P. K. (2006) Physicochemical modelling of cell signalling pathways. Nat. Cell Biol., 8, 1195–1203

[13] Anderson, J., Chang, Y. C. and Papachristodoulou, A. (2011) Model decomposition and reduction tools for large-scale networks in systems biology. Automatica, 47, 1165–1174

[14] Quaiser, T., Dittrich, A., Schaper, F. and Mönnigmann, M. (2011) A simple work flow for biologically inspired model reduction--application to early JAK-STAT signaling. BMC Syst. Biol., 5, 30

[15] Villaverde, A. F., Henriques, D., Smallbone, K., Bongard, S., Schmid, J., Cicin-Sain, D., Crombach, A., Saez-Rodriguez, J., Mauch, K., Balsa-Canto, E., (2015) BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology. BMC Syst. Biol., 9, 8

[16] Machta, B. B., Chachra, R., Transtrum, M. K. and Sethna, J. P. (2013) Parameter space compression underlies emergent theories and predictive models. Science, 342, 604–607

[17] Boyd, S. and Vandenberghe, L. (2004) Convex Optimization. New York: Cambridge University Press

[18] Moles, C. G., Mendes, P. and Banga, J. R. (2003) Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res., 13, 2467–2474

[19] Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007) Parameter estimation for differential equations: a generalized smoothing approach. J. R. Stat. Soc. Series B Stat. Methodol., 69, 741–796.

[20] Zenker, S., Rubin, J. and Clermont, G. (2007) From inverse problems in mathematical physiology to quantitative differential diagnoses. PLoS Comput. Biol., 3, e204

[21] Campbell, D. A. and Chkrebtii, O. (2013) Maximum profile likelihood estimation of differential equation parameters through model based smoothing state estimates. Math. Biosci., 246, 283–292

[22] Banga, J. R. and Balsa-Canto, E. (2008) Parameter estimation and optimal experimental design. Essays Biochem., 45, 195–210

[23] Kreutz, C. and Timmer, J. (2009) Systems biology: experimental design. FEBS J., 276, 923–942

[24] Meyer, P., Cokelaer, T., Chandran, D., Kim, K. H., Loh, P. R., Tucker, G., Lipson, M., Berger, B., Kreutz, C., Raue, A. (2014) Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach. BMC Syst. Biol., 8, 13

[25] Apri, M., de Gee, M. and Molenaar, J. (2012) Complexity reduction preserving dynamical behavior of biochemical networks. J. Theor. Biol., 304, 16–26

[26] Danø, S., Madsen, M. F., Schmidt, H. and Cedersund, G. (2006) Reduction of a biochemical model with preservation of its basic dynamic properties. FEBS J., 273, 4862–4877

[27] Kourdis, P. D., Palasantza, A. G. and Goussis, D. A. (2013) Algorithmic asymptotic analysis of the NF- kB signaling system. Comput. Math. Appl., 65, 1516–1534

[28] Radulescu, O., Gorban, A. N., Zinovyev, A. and Noel, V. (2012) Reduction of dynamical biochemical reactions networks in computational biology. Front. Genet., 3, 131

[29] Vanlier, J., Tiemann, C. A., Hilbers, P. A. J. and van Riel, N. A. W. (2012) An integrated strategy for prediction uncertainty analysis. Bioinformatics, 28, 1130–1135

[30] Vanlier, J., Tiemann, C. A., Hilbers, P. A. J. and van Riel, N. A. W. (2012) A Bayesian approach to targeted experiment design. Bioinformatics, 28, 1136–1142

[31] Huan, X. and Marzouk, Y. M. (2013) Simulation-based optimal Bayesian experimental design for nonlinear systems. J. Comput. Phys., 232, 288–317

[32] Pauwels, E., Lajaunie, C. and Vert, J. P. (2014) A Bayesian active learning strategy for sequential experimental design in systems biology. BMC Syst. Biol., 8, 102

[33] Liepe, J., Filippi, S., Komorowski, M. and Stumpf, M. P. H. (2013) Maximizing the information content of experiments in systems biology. PLoS Comput. Biol., 9, e1002888

[34] Busetto, A. G., Hauser, A., Krummenacher, G., Sunnåker, M., Dimopoulos, S., Ong, C. S., Stelling, J. and Buhmann, J. M. (2013) Near-optimal experimental design for model selection in systems biology. Bioinformatics, 29, 2625–2632

[35] Faller, D., Klingmüller, U. and Timmer, J. (2003) Simulation methods for optimal experimental design in systems biology. Simulation, 79, 717–725

[36] Casey, F. P., Baird, D., Feng, Q., Gutenkunst, R. N., Waterfall, J. J., Myers, C. R., Brown, K. S., Cerione, R. A. and Sethna, J. P. (2007) Optimal experimental design in an epidermal growth factor receptor signalling and down-regulation model. IET Syst. Biol., 1, 190–202

[37] Krüger, R. and Heinrich, R. (2004) Model reduction and analysis of robustness for the Wnt/-Catenin signal transduction pathway. Genome Inform., 15, 138–148

[38] Gerdtzen, Z. P., Daoutidis, P. and Hu, W. S. (2004) Non-linear reduction for kinetic models of metabolic reaction networks. Metab. Eng., 6, 140–154

[39] Vora, N. and Daoutidis, P. (2001) Nonlinear model reduction of chemical reaction systems. AIChE J., 47, 2320–2332

[40] Lam, S. H. (2013) Model reductions with special CSP data. Combust. Flame, 160, 2707–2711

[41] Kuo, J. C. W. and Wei, J. (1969) Lumping analysis in monomolecular reaction systems. analysis of approximately lumpable system. Ind. Eng. Chem. Fundam., 8, 124–133

[42] Liao, J. C. and Lightfoot, E. N. Jr. (1988) Lumping analysis of biochemical reaction systems with time scale separation. Biotechnol. Bioeng., 31, 869–879

[43] Brochot, C., Tóth, J. and Bois, F. Y. (2005) Lumping in pharmacokinetics. J. Pharmacokinet. Pharmacodyn., 32, 719–736

[44] Dokoumetzidis A, Aarons L (2009) Proper lumping in systems biology models. IET Syst. Biol., 3, 40–51

[45] Seigneur, C., Stephanopoulos, G. and Carr Jr., R. W. (1982) Dynamic sensitivity analysis of chemical reaction systems: a variational method. Chem. Eng. Sci., 37, 845–853

[46] Turányi, T., Bérces, T. and Vajda, S. (1989) Reaction rate analysis of complex kinetic systems. Int. J. Chem. Kinet., 21, 83–99

[47] Petzold, L. and Zhu, W. (1999) Model reduction for chemical kinetics: an optimization approach. AIChE J., 45, 869–886

[48] Liu, G., Swihart, M. T. and Neelamegham, S. (2005) Sensitivity, principal component and flux analysis applied to signal transduction: the case of epidermal growth factor mediated signaling. Bioinformatics, 21, 1194–1202

[49] Schmidt, H., Madsen, M. F., Danø, S. and Cedersund, G. (2008) Complexity reduction of biochemical rate expressions. Bioinformatics, 24, 848–854

[50] Steiert, B., Raue, A., Timmer, J. and Kreutz, C. (2012) Experimental design for parameter estimation of gene regulatory networks. PLoS One, 7, e40052

[51] Maiwald, T., Hass, H., Steiert, B., Vanlier, J., Engesser, R., Raue, A., Kipkeew, F., Bock, H. H., Kaschek, D., Kreutz, C., (2016) Driving the model to its limit: profile likelihood based model reduction. PLoS One, 11, e0162366

[52] Transtrum, M. K. and Qiu, P. (2012) Optimal experiment selection for parameter estimation in biological differential equation models. BMC Bioinformatics, 13, 181

[53] Transtrum, M. K. and Qiu, P. (2014) Model reduction by manifold boundaries. Phys. Rev. Lett., 113, 098701

[54] Transtrum, M. K. and Qiu, P. (2016) Bridging mechanistic and phenomenological models of complex biological systems. PLoS Comput. Biol., 12, e1004915

[55] Kutalik, Z., Cho, K. H. and Wolkenhauer, O. (2004) Optimal sampling time selection for parameter estimation in dynamic pathway modeling. Biosystems, 75, 43–55

[56] Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G. and Meyer, T. (2009) Optimal experimental design for parameter estimation of a cell signaling model. PLoS Comput. Biol., 5, e1000558

[57] Hagen, D. R., White, J. K. and Tidor, B. (2013) Convergence in parameters and predictions using computational experimental design. Interface Focus, 3, 20130008

[58] Toni, T., Welch, D., Strelkowa, N., Ipsen, A. and Stumpf, M. P. (2009) Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface, 6, 187–202

[59] Frieden BR (2000) Physics from fisher information: a unification. Am. J. Phys., 68, 1064–1065

[60] Transtrum, M. K., Machta, B. B. and Sethna, J. P. (2011) Geometry of nonlinear least squares with applications to sloppy models and optimization. Phys. Rev. E Stat. Nonlin. Soft Matter Phys., 83, 036701

[61] Leis, J. R. and Kramer, M. A. (1988) The simultaneous solution and sensitivity analysis of systems described by ordinary differential equations. ACM Trans. Math. Softw., 14, 45–60

[62] Kumar, A., Christofides, P. D. and Daoutidis, P. (1998) Singular perturbation modeling of nonlinear processes with nonexplicit time-scale multiplicity. Chem. Eng. Sci., 53, 1491–1504

[63] Snowden, T. J., van der Graaf, P. H. and Tindall, M. J. (2017) Methods of model reduction for large-scale biological systems: a survey of current methods and trends. Bull. Math. Biol., 79, 1449–1486

[64] Heinrich, R. and Schuster, S. (1996) The Regulation of Cellular Systems. Springer: New York

[65] Voit, E. (2012) A First Course in Systems Biology. 1st ed., Garland Science: New York

[66] Okino, M. S. and Mavrovouniotis, M. L. (1998) Simplification of mathematical models of chemical reaction systems. Chem. Rev., 98, 391–408

[67] Wolf, J. and Heinrich, R. (2000) Effect of cellular interaction on glycolytic oscillations in yeast: a theoretical investigation. Biochem. J., 345, 321–334

[68] Sauter, T., Gilles, E. D., Allgöwer, F., Saez-Rodriguez, J., Conzelmann, H. and Bullinger, E. (2004) Reduction of mathematical models of signal transduction networks: simulation-based approach applied to EGF receptor signalling. Syst. Biol. (Stevenage), 1, 159–169

[69] Liebermeister, W., Baur, U. and Klipp, E. (2005) Biochemical network models simplified by balanced truncation. FEBS J., 272, 4034–4043

[70] Maertens, J., Donckels, B., Lequeux, G. and Vanrolleghem, P. (2005) Metabolic model reduction by metabolite pooling on the basis of dynamic phase planes and metabolite correlation analysis. In Proceedings of the Conference on Modeling and Simulation in Biology, Medicine and Biomedical Engineering. Linkping , Sweden.

ACKNOWLEDGMENTS

This work is supported by funding from the National Science Foundation (CCF1552784). Peng Qiu is an ISAC Marylou Ingram Scholar.

COMPLIANCE WITH ETHICS GUIDELINES

The authors Jenny E. Jeong, Qinwei Zhuang, Mark K. Transtrum, Enlu Zhou and Peng Qiu declare that they have no conflict of interests.

This article is a review article and does not contain any studies with human or animal subjects performed by any of the authors.

RIGHTS & PERMISSIONS

2018 Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature