Please wait a minute...

Frontiers of Earth Science

Front. Earth Sci.
REVIEW |
Ensembles vs. information theory: supporting science under uncertainty
Grey S. NEARING1, Hoshin V. GUPTA2()
1. University of Alabama, Department of Geological Sciences, Tuscaloosa, AL 35294-1152, USA
2. University of Arizona, Department of Hydrology and Atmospheric Sciences, Tucson, AZ 85721-0158, USA
Download: PDF(168 KB)   HTML
Export: BibTeX | EndNote | Reference Manager | ProCite | RefWorks
Abstract

Multi-model ensembles are one of the most common ways to deal with epistemic uncertainty in hydrology. This is a problem because there is no known way to sample models such that the resulting ensemble admits a measure that has any systematic (i.e., asymptotic, bounded, or consistent) relationship with uncertainty. Multi-model ensembles are effectively sensitivity analyses and cannot – even partially – quantify uncertainty. One consequence of this is that multi-model approaches cannot support a consistent scientific method – in particular, multi-model approaches yield unbounded errors in inference. In contrast, information theory supports a coherent hypothesis test that is robust to (i.e., bounded under) arbitrary epistemic uncertainty. This paper may be understood as advocating a procedure for hypothesis testing that does not require quantifying uncertainty, but is coherent and reliable (i.e., bounded) in the presence of arbitrary (unknown and unknowable) uncertainty. We conclude by offering some suggestions about how this proposed philosophy of science suggests new ways to conceptualize and construct simulation models of complex, dynamical systems.

Keywords information theory      multi-model ensembles      Bayesian methods      uncertainty quantification      hypothesis testing     
Corresponding Authors: Hoshin V. GUPTA   
Just Accepted Date: 08 April 2018   Online First Date: 09 May 2018   
 Cite this article:   
Grey S. NEARING,Hoshin V. GUPTA. Ensembles vs. information theory: supporting science under uncertainty[J]. Front. Earth Sci., 09 May 2018. [Epub ahead of print] doi: 10.1007/s11707-018-0709-9.
 URL:  
http://journal.hep.com.cn/fesci/EN/10.1007/s11707-018-0709-9
http://journal.hep.com.cn/fesci/EN/Y/V/I/0
Service
E-mail this article
E-mail Alert
RSS
Articles by authors
Grey S. NEARING
Hoshin V. GUPTA
Fig.1  Diagram of a simple experiment with a single input and single output. Even in the simplest case, we require at least one process hypothesis p, and at least two measurement models u and y. Aleatory uncertainty is defined as the (unknown) distribution &(zy |zu), which would only be knowable if we had access to both a perfect process model and also perfect measurement models. A full model is the conjunction = {p, u , y}.
1 Albrecht A, Phillips D (2014). Origin of probabilities and their application to the multiverse. Phys Rev D Part Fields Gravit Cosmol, 90(12): 123514
https://doi.org/10.1103/PhysRevD.90.123514
2 Beven K, Freer J (2001). Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. J Hydrol (Amst), 249(1–4): 11–29
https://doi.org/10.1016/S0022-1694(01)00421-8
3 Beven K J (2006). Searching for the Holy Grail of scientific hydrology: Qt = (SR)A as closure. Hydrol Earth Syst Sci, 10(5): 609–618
https://doi.org/10.5194/hess-10-609-2006
4 Beven K J (2016). Facets of uncertainty: epistemic uncertainty, non-stationarity, likelihood, hypothesis testing, and communication. Hydrol Sci J, 61(9): 1652–1665
https://doi.org/10.1080/02626667.2015.1031761
5 Beven K J, Smith P J, Freer J E (2008). So just why would a modeller choose to be incoherent? J Hydrol (Amst), 354(1): 15–32
https://doi.org/10.1016/j.jhydrol.2008.02.007
6 Clark M P, Kavetski D, Fenicia F (2011). Pursuing the method of multiple working hypotheses for hydrological modeling. Water Resour Res, 47(9): https://doi.org/10.1029/2010WR009827
7 Clark M P, Nijssen B, Lundquist J D, Kavetski D, Rupp D E, Woods R A, Freer J E, Gutmann E D, Wood A W, Brekke L D, Arnold J R, Gochis D J, Rasmussen R M (2015). A unified approach for process-based hydrologic modeling: 1. Modeling concept. Water Resour Res, 51(4): 2498–2514
https://doi.org/10.1002/2015WR017198
8 Eyring V, Bony S, Meehl G A, Senior C A, Stevens B, Stouffer R J, Taylor K E (2016). Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci Model Dev, 9(5): 1937–1958
https://doi.org/10.5194/gmd-9-1937-2016
9 Gelman A, Shalizi C R (2013). Philosophy and the practice of Bayesian statistics. Br J Math Stat Psychol, 66(1): 8–38
https://doi.org/10.1111/j.2044-8317.2011.02037.x
10 Gong W, Gupta H V, Yang D, Sricharan K, Hero A O III (2013). Estimating epistemic and aleatory uncertainties during hydrologic modeling: an information theoretic approach. Water Resour Res, 49(4): 2253–2273
https://doi.org/10.1002/wrcr.20161
11 Grünwald P, Langford J (2007). Suboptimal behavior of Bayes and MDL in classification under misspecification. Mach Learn, 66(2–3): 119–149
https://doi.org/10.1007/s10994-007-0716-7
12 Hornik K (1991). Approximation capabilities of multilayer feedforward networks. Neural Netw, 4(2): 251–257
https://doi.org/10.1016/0893-6080(91)90009-T
13 Kinney J B, Atwal G S (2014). Equitability, mutual information, and the maximal information coefficient. Proc Natl Acad Sci USA, 111(9): 3354–3359
https://doi.org/10.1073/pnas.1309933111
14 Metropolis N (1987). The beginning of the Monte Carlo method. Los Alamos Sci, 15(584): 125–130
15 Montanari A (2007). What do we mean by ‘uncertainty’? The need for a consistent wording about uncertainty assessment in hydrology. Hydrol Processes, 21(6): 841–845
https://doi.org/10.1002/hyp.6623
16 Nearing G S, Gupta H V (2015). The quantity and quality of information in hydrologic models. Water Resour Res, 51(1): 524–538
https://doi.org/10.1002/2014WR015895
17 Nearing G S, Mocko D M, Peters-Lidard C D, Kumar S V, Xia Y (2016a). Benchmarking NLDAS-2 soil moisture and evapotranspiration to separate uncertainty contributions. J Hydrometeorol, 17(3): 745–759
https://doi.org/10.1175/JHM-D-15-0063.1
18 Nearing G S, Tian Y, Gupta H V, Clark M P, Harrison K W, Weijs S V (2016b). A philosophical basis for hydrologic uncertainty. Hydrol Sci J, 61(9): 1666–1678
https://doi.org/10.1080/02626667.2016.1183009
19 Popper K R (1959). The Logic of Scientific Discovery. London: Hutchinson & Co.
20 Rasmussen C, Williams C (2006). Gaussian Processes for Machine Learning. Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press
21 Renard B, Kavetski D, Kuczera G, Thyer M, Franks S W (2010). Understanding predictive uncertainty in hydrologic modeling: the challenge of identifying input and structural errors. Water Resour Res, 46(5): https://doi.org/10.1029/2009WR008328
22 Shannon C E (1948). A mathematical theory of communication. Bell Syst Tech J, 27(3): 379–423
https://doi.org/10.1002/j.1538-7305.1948.tb01338.x
23 Stanford K (2016). Underdetermination of Scientific Theory. In: Zalta N, ed. The Stanford Encyclopedia of Philosophy
24 Taleb N N (2010). The Black Swan: the Impact of the Highly Improbable Fragility. New York: Random House Group
25 Van Horn K S (2003). Constructing a logic of plausible inference: a guide to Cox’s theorem. Int J Approx Reason, 34(1): 3–24
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed