The quest for conditional independence in prospectivity modeling: weights-of-evidence, boost weights-of-evidence, and logistic regression

Abstract

The objective of prospectivity modeling is prediction of the conditional probability of the presence \mathsf{T}=\mathrm{1} or absence \mathsf{T}=\mathrm{0} of a target \mathsf{T} given favorable or prohibitive predictors \mathsf{B}, or construction of a two classes \left\{\mathrm{0},\mathrm{1}\right\} classification of \mathsf{T}. A special case of logistic regression called weights-of-evidence (WofE) is geologists’ favorite method of prospectivity modeling due to its apparent simplicity. However, the numerical simplicity is deceiving as it is implied by the severe mathematical modeling assumption of joint conditional independence of all predictors given the target. General weights of evidence are explicitly introduced which are as simple to estimate as conventional weights, i.e., by counting, but do not require conditional independence. Complementary to the regression view is the classification view on prospectivity modeling. Boosting is the construction of a strong classifier from a set of weak classifiers. From the regression point of view it is closely related to logistic regression. Boost weights-of-evidence (BoostWofE) was introduced into prospectivity modeling to counterbalance violations of the assumption of conditional independence even though relaxation of modeling assumptions with respect to weak classifiers was not the (initial) purpose of boosting. In the original publication of BoostWofE a fabricated dataset was used to “validate” this approach. Using the same fabricated dataset it is shown that BoostWofE cannot generally compensate lacking conditional independence whatever the consecutively processing order of predictors. Thus the alleged features of BoostWofE are disproved by way of counterexamples, while theoretical findings are confirmed that logistic regression including interaction terms can exactly compensate violations of joint conditional independence if the predictors are indicators.

Keywords

general weights of evidence / joint conditional independence / naïve Bayes model / Hammersley–Clifford theorem / interaction terms / statistical significance

Introduction

The objective of prospectivity modeling is to identify locations (pixels, voxels) x\subset D in some domain of definition D for which the conditional probability P(\mathsf{T}(x)=\mathrm{1}|\mathsf{B}(x)) of the presence, \mathsf{T}(x)=\mathrm{1} , of a well defined target type of ore mineralization, given favorable or prohibitive factors \mathsf{B}(x) is a relative maximum. Of course, the major prerequisite for such predictions is a proper conceptual model of the specified ore mineralization. A proper conceptual model may be turned into a regression-type model using the factors as spatially referenced predictors. Generally, a model considers the predictor \mathsf{B}(x)={({\mathsf{B}}_{\mathrm{0}}(x),{\mathsf{B}}_{\mathrm{1}}(x),\dots ,{\mathsf{B}}_m(x))}^{\mathsf{T}} , with {\mathsf{B}}_{\mathrm{0}}(x)\equiv \mathrm{1} for all x\subset D , and assigns a parameter \mathit{\theta }={({\theta }_{\mathrm{0}},\dots ,{\theta }_m)}^{\mathsf{T}} , which quantifies by means of a link function \mathfrak{L} the extent of dependence of the conditional probability P(\mathsf{T}(x)=\mathrm{1}|\mathsf{B}(x)) on the predictors, i.e.,

(1) \mathfrak{L}P(\mathsf{T}(x)=\mathrm{1}|\mathsf{B}(x))=\mathsf{B}{(x)}^{\mathsf{T}}\mathit{\theta }.

The target \mathsf{T}(x) as well as the predictor \mathsf{B}(x) refer to locations x\subset D with areal or volumetric extent, pixels or voxels, which are assumed to provide their physical support. Once computed, predicted conditional probabilities and the associated prediction errors, respectively, will be assigned to them as additional properties.

Recent surveys of mathematical models and numerical methods for prospectivity modeling have been compiled in two special issues, Mineral prospectivity analysis and quantitative resource estimation of Ore Geology Reviews 38(3), 121–304, guest edited by Kreuzer and Porwal (2010), and GIS-based mineral potential modelling and geological data analyses for mineral exploration of Ore Geology Reviews 71, 477–881, guest edited by Porwal and Carranza (2015). Agterberg (2014) devotes a major chapter to this topic. ArcGIS by ESRI features ArcSDM providing tools for prospectivity modeling. Despite its ubiquity in geological prospection, the mathematical assumptions to authorize an approach do not seem to be well communicated. Of special concern is the role of conditional independence which is not yet another version of stochastic independence but a concept on its own.

Moreover, a preference to publish case studies of prospectivity modeling applying “novel” procedures or “novel” variants can be observed. Whatever new method of prediction or classification has been developed in statistics, fuzzy logic or machine learning, it is being applied to case studies of prospectivity modeling, cf. the references of ( Kreuzer and Porwal, 2010; Porwal and Carranza, 2015). Subsequently, these case studies give rise to plenty empirical comparisons, for instance ( Harris and Pan, 1999; Harris et al., 2003; Porwal et al., 2010; Rodriguez-Galiano et al., 2015; Ford et al., 2016) which often conclude superior or inferior performance in one way or another. Clarification of the origins and considerations of the mutual relationships of various methods may render some differences in the results of their practical applications less surprising than others. Eventually, mathematical models and corresponding methods cannot be validated, their properties cannot be derived by way of case studies.

As a result, the practitioner of prospectivity modeling often seems lost in a vast variety of procedures, especially when exposing the essentials of his/her method of choice. The threefold objective of this communication is (i) to clarify the mathematical relationship of logistic regression, weights-of-evidence, and boost weights-of-evidence, (ii) to disprove the major claims put forward by Cheng (2012, 2015) with respect to his boost weights-of-evidence by virtue of counterexamples including the fabricated training dataset provided by Cheng (2015), and (iii) to exemplify theoretical findings that interaction terms of logistic regression models compensate lack of conditional independence.

Boosting is the affirmative answer to the mathematical problem whether it is possible to construct a strong classifier (with superior properties) from a set of weak classifiers (with poor properties). Boosting was not meant to relax modeling assumptions possibly required by classifiers. Boost weights-of-evidence (BoostWofE) ( Cheng, 2015) is the most recent attempt to relax the mathematical modeling assumption of weights-of-evidence ( Good, 1950, 1960, 1968; Minsky and Selfridge, 1961; Agterberg et al., 1990; Bonham-Carter, 1994; Schaeben, 2014c) which is joint conditional independence of all predictors given the target. Applying weights-of-evidence despite lacking conditional independence corrupts not only the predicted conditional probabilities but also their rank transforms, and thus the spatial pattern of prospectivity ( Schaeben, 2014a). Boosted weights of evidence differ from ordinary weights by additive terms ( Cheng, 2015, Eqs. (26) and (27), p. 602–603), introduced as approximations of conditional weights ( Cheng, 2015, Eqs. (11) and (12), p. 597) taken from the \nu -model ( Polyakova and Journel, 2007) without citing it. For reasons of completeness the \nu -model is recalled in the Appendix A.

Weights-of-evidence is an application of Bayes theorem for several variables, and the special case of logistic regression if the predictors \mathsf{B} are nominal (categorical) and jointly conditional independent given the target \mathsf{T} ( Schaeben, 2014b). In turn, logistic regression ( Reed and Berkson, 1929; Berkson, 1944; Hosmer et al., 2013) is the canonical generalization of Bayesian weights-of-evidence allowing for deviations from conditional independence of a restricted form. Applying Hammersley–Clifford theorem, it was proven that logistic regression including interaction terms corresponding to violations of conditional independence compensates this lack exactly and is optimum, i.e., recovers the true conditional probability, if the joint distribution of predictors and target is of log-linear form ( Schaeben, 2014c).

Weights-of-evidence, novel boost weights-of-evidence, and classical logistic regression are discussed in basic mathematical terms, and then applied to the fabricated training dataset used by Cheng (2015) for the purpose of empirical comparison. To gain additional insight, the methods are also applied to the fabricated training datasets RANKIT used in earlier communications, e.g. ( Schaeben, 2014a, b).

Fundamentals: stochastic independence, conditional independence

Definition

For a set \{\mathrm{0},\dots ,m\} of indexes, the \otimes -product denotes both the product of random variables {\mathsf{Z}}_{\ell } defined as {\otimes }_{\ell =\mathrm{0}}^m{\mathsf{Z}}_{\ell }=({\mathsf{Z}}_{\mathrm{0}},\mathrm{...},{\mathsf{Z}}_m) , and the product of their probability measures P_{{\mathsf{Z}}_{\ell }} . If the random variables {\mathsf{Z}}_{\ell },\ell =\mathrm{0},\dots ,m , are independent, then the joint probability of any subset of random variables {\mathsf{Z}}_{\ell } can be factorized into the product of the individual probabilities, i.e.,

(2) P_{{\otimes }_{\ell \in M}{\mathsf{Z}}_{\ell }}=\underset{\ell \in M}{\otimes }{P}_{{\mathsf{Z}}_{\ell }},

where M denotes any non-empty subset of the set \{\mathrm{0},\dots ,m\} . In particular

P_{\mathsf{Z}}=P_{{\otimes }_{\ell =\mathrm{0}}^m{\mathsf{Z}}_{\ell }}=\underset{\ell =\mathrm{0}}{\overset{m}{\otimes }}{P}_{{\mathsf{Z}}_{\ell }}.

If the random variables {\mathsf{Z}}_{\ell },\ell =\mathrm{1},\dots ,m , are conditionally independent given {\mathsf{Z}}_{\mathrm{0}} , then the joint conditional probability of any subset of random variables {\mathsf{Z}}_{\ell } given {\mathsf{Z}}_{\mathrm{0}} can be factorized into the product of the individual conditional probabilities, i.e.,

(3) P_{{\otimes }_{\ell \in M}{\mathsf{Z}}_{\ell }|{\mathsf{Z}}_{\mathrm{0}}}=\underset{\ell \in M}{\otimes }{P}_{{\mathsf{Z}}_{\ell }|{\mathsf{Z}}_{\mathrm{0}}},

and in particular

P_{{\otimes }_{\ell =\mathrm{1}}^m{\mathsf{Z}}_{\ell }|{\mathsf{Z}}_{\mathrm{0}}}=\underset{\ell =\mathrm{1}}{\overset{m}{\otimes }}{P}_{{\mathsf{Z}}_{\ell }|{\mathsf{Z}}_{\mathrm{0}}}.

Given {\mathsf{Z}}_{\mathrm{0}} , the observable variables {\mathsf{Z}}_{\ell },\ell \in M , become independent. In practice, often the interpretation of {\mathsf{Z}}_{\mathrm{0}} as common cause for {\mathsf{Z}}_{\ell },\ell \in M , applies. Thus, conditional independence is a probabilistic approach to causality ( Suppes, 1970; Dawid, 1979, 2004, 2007; Pearl, 2009; Chalak and White, 2012) while correlation, i.e., linear dependence, is not. Correlated random variables may be conditionally independent or not, conditionally independent random variables may be (significantly) correlated or not. Independence does not imply conditional independence and vice versa; pairwise conditional independence does not imply joint conditional independence.

Weak conditional independence was introduced by Wong and Butz (1999), and elaborated on by Butz and Sanscartier (2002). The definition of weak conditional independence by Cheng (2015) is irrelevant at this time as a statistical significance test is not provided.

Testing conditional independence

If the predictor variables {\mathsf{B}}_{\ell },\ell =\mathrm{1},\dots ,m , and the target variable \mathsf{T} are indicator variables, i.e., binary, then the joint probability is of log-linear form. If the predictor variables are jointly conditionally independent given the target variable, then by virtue of the Hammersley–Clifford theorem the log–linear model factorized into terms corresponding to the target variable \mathsf{T} , the individual predictor variables {\mathsf{B}}_{\ell } , and the individual products \mathsf{T}\otimes {\mathsf{B}}_{\ell },\ell =\mathrm{1},\dots ,m , is sufficiently large to represent the joint probability ( Schaeben, 2014c). The sufficient size of such a factorized form of an appropriate log–linear model can be turned into the null-hypothesis of a statistical significance test. Thus, if the likelihood ratio test of this null-hypothesis leads to its possible rejection, then the assumption of joint conditional independence can be rejected, too.

Logistic regression, weights-of-evidence, boost weights-of-evidence

To fit a model, i.e., to estimate the model parameters \theta of model ansatz Eq. (1), data within a training region are required. However, in contrast to geostatistics (Chilès and Delfiner, 2012), and to their major detriment, the methods of prospectivity modeling considered here do not consider spatially induced dependencies between the target and the predictors nor between predictor variables themselves. Alternatives have been suggested by van den Boogaart and Schaeben (2012) and Tolosana-Delgado et al. (2014).

A proper definition of the notion of spatial association does not exist in the realm of potential mapping or prospectivity modeling. In fact, the classical assumption of independently identically distributed random variables applies, distributions do not depend on location. Therefore, any spatial reference can be dropped, and models of the form

(4) \mathfrak{L}P(\mathsf{T}=\mathrm{1}|\mathsf{B})={\mathsf{B}}^{\mathsf{T}}\mathit{\theta },

are considered, only. Instead of an illusive spatial association, the ordinary correlation matrix may provide some instructive information how to choose the predictors of a proper regression model.

Logistic regression

Conditional expectation of a binary random target variable \mathsf{T} given a (m+\mathrm{1}) –variate random predictor variable \mathsf{B}={({\mathsf{B}}_{\mathrm{0}},{\mathsf{B}}_{\mathrm{1}},\dots ,{\mathsf{B}}_m)}^{\mathsf{T}} with {\mathsf{B}}_{\mathrm{0}}\equiv \mathrm{1} is equal to a conditional probability, i.e.,

\mathrm{E(}\mathsf{T}|\mathsf{B})=P(\mathsf{T}=\mathrm{1}|\mathsf{B}).

Neglecting the error term as often the ordinary logistic regression model (without interaction terms) of a logit-transformed conditional probability in terms of a linear combination of predictors reads

(5) \mathrm{logit}P(\mathsf{T}=\mathrm{1}|\mathsf{B})={\beta }_{\mathrm{0}}+\sum _{\ell }{\beta }_{\ell }{\mathsf{B}}_{\ell },

which can be rewritten in terms of a conditional probability as

(6) P(\mathsf{T}=\mathrm{1}|\mathsf{B})=\mathrm{\Lambda }\left({\beta }_{\mathrm{0}}+\sum _{\ell }{\beta }_{\ell }{\mathsf{B}}_{\ell }\right),

as the logistic function denoted \mathrm{\Lambda } is the inverse of the logit-transform.

If the joint probability of the indicator target variable and the predictor variables is of log–linear form and all predictor variables are conditionally independent given the target variable, then the conditional probability of the target variable given the predictors is of the form of Eq. (6) of the ordinary logistic regression model. Thus, in this case the ordinary logistic regression model is optimum. In particular, it is optimum if the predictor variables are categorical or discrete and jointly conditionally independent given the target variable ( Schaeben, 2014a).

The logistic regression model with interaction terms reads in terms of a logit

\mathrm{logit}P(\mathsf{T}=\mathrm{1}|\mathsf{B})={\beta }_{\mathrm{0}}+\sum _{\ell }{\beta }_{\ell }{\mathsf{B}}_{\ell }+\sum _{{\ell }_i,\dots ,{\ell }_j}{\beta }_{{\ell }_i,\dots ,{\ell }_j}{\mathsf{B}}_{{\ell }_i}\dots {\mathsf{B}}_{{\ell }_j})

and in terms of a probability

(7) P(\mathsf{T}=\mathrm{1}|\mathsf{B})=\mathrm{\Lambda }\left({\beta }_{\mathrm{0}}+\sum _{\ell }{\beta }_{\ell }{\mathsf{B}}_{\ell }+\sum _{{\ell }_i,\dots ,{\ell }_j}{\beta }_{{\ell }_i,\dots ,{\ell }_j}{\mathsf{B}}_{{\ell }_i}\dots {\mathsf{B}}_{{\ell }_j})\right).

If the joint probability of the indicator target variable and the predictor variables is of log–linear form including interaction terms corresponding to lacking conditionally independence given the target variable, then the conditional probability of the target variable given the predictors is of the form of Eq. (7) of the logistic regression model enlarged by interaction terms. Thus, in this case the augmented logistic regression model is optimum. In particular, for categorical or discrete predictor variables, interaction terms can compensate any lack of conditional independence exactly, i.e., logistic regression with interaction terms is optimum in case of lacking conditional independence ( Schaeben, 2014a).

Given the sample b_{\ell ,i},{\mathrm{T}}_i,i=\mathrm{1},\dots ,n,\ell =\mathrm{1},\dots ,m , the parameters of the logistic regression model are estimated with well established, well understood methods based on probability, and encoded in any major statistical software package applying (i) the method of maximum likelihood estimation numerically (ii) realized with Fisher scoring algorithm (a form of Newton-Raphson, a special case of iteratively reweighted least squares algorithm) ensuring nice statistical properties of the estimates like consistency, asymptotic normality, and efficiency of the estimates.

The properties of a fitted logistic regression model are assessed by the significance of the estimated regression parameters, and by Akaike information criterion (AIC)

\mathrm{AIC}=-\ln (L)+\mathrm{2}k,

where L is the maximized value of the likelihood function, and k be the total number of estimated parameters of the model. AIC provides a measure of the relative quality of a statistical model for a given training dataset. More measures to assess the fit of a logistic model including the receiver operating characteristic curve are discussed in (Hosmer et al., 2013). Despite a small AIC, a logistic regression model is discarded if at least one of its fitted parameters is not significant to prevent overfitting to the given training dataset and poor predicting performance for any other application dataset.

A fitted regression model is mathematically authorized for prediction if the model assumptions are satisfied, if it resembles the true relationship of the target and the predictors, and if their fitted regression parameters are significant. Large errors of predictions render a fitted model inappropriate for prediction in any case.

Weights-of-evidence

Weights-of-evidence is an application of Bayes theorem. Bayes theorem for several variables {\mathsf{B}}_{\mathrm{0}},{\mathsf{B}}_{\mathrm{1}},\dots ,{\mathsf{B}}_m , {\mathsf{B}}_{\mathrm{0}}\equiv \mathrm{1} , reads in terms of odds

\begin{array}{l}\mathrm{O(}\mathsf{T}=\mathrm{1}|\mathsf{B}=\mathsf{B})\\ =\mathrm{O}(\mathsf{T}=\mathrm{1}){\displaystyle \frac{{{\displaystyle \prod }}_{\ell =\mathrm{1}}^{m}P({\mathsf{B}}_{\ell }|{\otimes }_{j=\mathrm{0}}^{\ell -\mathrm{1}}{\mathsf{B}}_j=(\mathrm{1},b_{\mathrm{1}},\dots ,b_{\ell -\mathrm{1}})\wedge \mathsf{T}=\mathrm{1})}{{{\displaystyle \prod }}_{\ell =\mathrm{1}}^{m}P({\mathsf{B}}_{\ell }|{\otimes }_{j=\mathrm{0}}^{\ell -\mathrm{1}}{\mathsf{B}}_j=(\mathrm{1},b_{\mathrm{1}},\dots ,b_{\ell -\mathrm{1}})\wedge \mathsf{T}=\mathrm{0})}}\\ =\mathrm{O}(\mathsf{T}=\mathrm{1}){\displaystyle \prod }_{\ell =\mathrm{1}}^m{F}_{\ell }\end{array}

with Bayes factors ( Good, 1968)

(8) F_{\ell }={\displaystyle \frac{P({\mathsf{B}}_{\ell }|{\otimes }_{j=\mathrm{0}}^{\ell -\mathrm{1}}{\mathsf{B}}_j=(\mathrm{1},b_{\mathrm{1}},\dots ,b_{\ell -\mathrm{1}})\wedge \mathsf{T}=\mathrm{1})}{P({\mathsf{B}}_{\ell }|{\otimes }_{j=\mathrm{0}}^{\ell -\mathrm{1}}{\mathsf{B}}_j=(\mathrm{1},b_{\mathrm{1}},\dots ,b_{\ell -\mathrm{1}})\wedge \mathsf{T}=\mathrm{0})}}, \ell =\mathrm{1},\dots ,m.

Applying the logit transform yields

(9) \mathrm{logit}P(\mathsf{T}=\mathrm{1}|\mathsf{B})=\mathrm{logit}P(\mathsf{T}=\mathrm{1})+{\displaystyle \sum }_{\ell =\mathrm{1}}^m\ln F_{\ell }.

General weights of evidence

Let V_{\ell },\ell =\mathrm{1},\dots ,m , denote the set of \ell -variations of the set \{\mathrm{0},\mathrm{1}\} , and v_{\ell }={\mathrm{2}}^{\ell } its total number of elements. Let {\mathsf{Z}}_{\ell }={\otimes }_{j=\mathrm{1}}^{\ell }{\mathsf{B}}_j,\ell =\mathrm{1},\dots ,m . Given a realization {\mathbf{b}}_{k,\ell }\in V_{\ell },k=\mathrm{1},\dots ,v_{\ell } of {\mathsf{Z}}_{\ell },\ell =\mathrm{1},\dots ,m , F_{\ell } is rewritten for \ell =\mathrm{2},\dots ,m more detailed as

F_{\ell }=F({\mathit{b}}_{k,\ell })={\displaystyle \frac{P({\mathsf{Z}}_{\ell }={\mathbf{b}}_{k,\ell }\wedge \mathsf{T}=\mathrm{1})}{P({\mathsf{Z}}_{\ell }={\mathbf{b}}_{k,\ell }\wedge \mathsf{T}=\mathrm{0})}}{\displaystyle \frac{P({\mathsf{Z}}_{\ell -\mathrm{1}}={\mathbf{b}}_{k,\ell -\mathrm{1}}\wedge \mathsf{T}=\mathrm{0})}{P({\mathsf{Z}}_{\ell -\mathrm{1}}={\mathbf{b}}_{k,\ell -\mathrm{1}}\wedge \mathsf{T}=\mathrm{1})}},k=\mathrm{1},\dots ,v_{\ell },

where {\mathbf{b}}_{k,\ell -\mathrm{1}} agrees with {\mathsf{b}}_{k,\ell } in the first (\ell -\mathrm{1}) entries. Then

F({\mathsf{Z}}_{\ell })=\sum _{k=\mathrm{1}}^{v_{\ell }}{F}_{\ell }({\mathbf{b}}_{k,\ell }){\mathbb{I}}_{\{{\mathrm{B}}_{k,\ell }\}}({\mathsf{Z}}_{\ell }),

where {\mathbb{I}}_{\{{\mathbf{b}}_{k,\ell }\}} denotes the indicator function with respect to the set \{{\mathbf{b}}_{k,\ell }\} containing the vector {\mathbf{b}}_{k,\ell } . Then

(10) \mathrm{O}(\mathsf{T}=\mathrm{1}|\mathsf{B})=\mathrm{O}(\mathsf{T}=\mathrm{1}){\displaystyle \prod }_{\ell =\mathrm{1}}^m{\displaystyle \sum }_{k=\mathrm{1}}^{v_{\ell }}F({\mathsf{b}}_{k,\ell }){\mathbb{I}}_{\{{\mathrm{B}}_{k,\ell }\}}({\mathsf{Z}}_{\ell }),

and

(11) \mathrm{logit}P(\mathsf{T}=\mathrm{1}|\mathsf{B})=\mathrm{logit}P(\mathsf{T}=\mathrm{1})+\sum _{\ell =\mathrm{1}}^m{\displaystyle \sum }_{k=\mathrm{1}}^{v_{\ell }}\ln F(b_{k,\ell }){\mathbb{I}}_{\{{\mathbf{b}}_{k,\ell }\}}({\mathsf{Z}}_{\ell }),

where \ln F({\mathbf{b}}_{k,\ell }) of Eq. (11) could be referred to as general weights of evidence, emphasizing that conditional independence is not required. Since the tacit assumption as usually is that the Bayes factors and their logarithms exist, i.e., that none of the involved probabilities is 0 or 1, they can be estimated elementarily by corresponding frequencies, i.e., by counting occurrences within a given training region.

Weights of evidence assuming conditional independence

Assuming joint conditional independence of all predictors {\mathsf{B}}_{\mathrm{1}},\dots ,{\mathsf{B}}_m given the target \mathsf{T} simplifies F_{\ell } to

F_{\ell }^{\mathrm{CI}}={\displaystyle \frac{P({\mathsf{B}}_{\ell }|\mathsf{T}=\mathrm{1})}{P({\mathsf{B}}_{\ell }|\mathsf{T}=\mathrm{0})}},\ell =\mathrm{1},\dots ,m,

and results in

(12) \begin{array}{l}\mathrm{logit}P(\mathsf{T}=\mathrm{1}|\mathsf{B})=\mathrm{logit}P(\mathsf{T}=\mathrm{1})+\sum _{\ell =\mathrm{1}}^m\ln {\displaystyle \frac{P({\mathsf{B}}_{\ell }| \mathsf{T}=\mathrm{1})}{P({\mathsf{B}}_{\ell } | \mathsf{T}=\mathrm{0})}},\\ =\mathrm{logit}P(\mathsf{T}=\mathrm{1})+{\displaystyle \sum }_{\ell =\mathrm{1}}^m{W}_{\ell }\end{array}

with

(13) W_{\ell }=\ln {\displaystyle \frac{P({\mathsf{B}}_{\ell }|\mathsf{T}=\mathrm{1})}{P({\mathsf{B}}_{\ell }|\mathsf{T}=\mathrm{0})}},

provided that neither numerator nor denominator in the definition of W_{\ell } , Eq. (13), vanish. Eq. (12) tells us how to update and improve the unconditional \mathrm{logit}P(\mathsf{T}=\mathrm{1}) considering information provided by {\mathsf{B}}_{\mathrm{1}},\dots ,{\mathsf{B}}_m assuming their joint conditional independence given the target. Introducing

(14) W_{\ell }^{(\mathrm{1})}=\ln {\displaystyle \frac{P({\mathsf{B}}_{\ell }=\mathrm{1}|\mathsf{T}=\mathrm{1})}{P({\mathsf{B}}_{\ell }=\mathrm{1}|\mathsf{T}=\mathrm{0})}},\hspace{1em}{W}_{\ell }^{(\mathrm{0})}=\ln {\displaystyle \frac{P({\mathsf{B}}_{\ell }=\mathrm{0}|\mathsf{T}=\mathrm{1})}{P({\mathsf{B}}_{\ell }=\mathrm{0}|\mathsf{T}=\mathrm{0})}}.

Eq. (12) can be rewritten in terms of predictor variables as

(15) \begin{array}{l}\mathrm{logit}P(\mathsf{T}=\mathrm{1}|\mathsf{B})=\mathrm{logit}P(\mathsf{T}=\mathrm{1})+{\displaystyle \sum }_{\ell =\mathrm{1}}^m(W_{\ell }^{(\mathrm{1})}{\mathsf{B}}_{\ell }+W_{\ell }^{(\mathrm{0})}(\mathrm{1}-{\mathsf{B}}_{\ell }))\\ =\mathrm{logit}P(\mathsf{T}=\mathrm{1})+W^{(\mathrm{0})}+{\displaystyle \sum }_{\ell =\mathrm{1}}^m{C}_{\ell }{\mathsf{B}}_{\ell },\end{array}

with contrasts

C_{\ell }=W_{\ell }^{(\mathrm{1})}-W_{\ell }^{(\mathrm{0})},\ell =\mathrm{1},\dots ,m,

and W^{(\mathrm{0})}={\displaystyle \sum }_{\ell =\mathrm{1}}^m{W}_{\ell }^{(\mathrm{0})} . Besides less involved weights, the major difference between the two models, conventional weights-of-evidence assuming joint conditional independence, Eq. (12), and general weights-of-evidence, Eq. (11), is the presence of interaction terms, i.e., product terms of predictors, in the latter. Moreover, comparing Eq. (5) and Eq. (15) reveals that in case of joint conditional independence of all predictors given the target variable the regression coefficients simplify to

{\beta }_{\mathrm{0}}=\mathrm{logit}P(\mathsf{T}=\mathrm{1})+W^{(\mathrm{0})},\hspace{1em}{\beta }_{\ell }=C_{\ell },\ell =\mathrm{1},\dots ,m,

( Schaeben, 2014a, b). Obviously the model parameters become independent of one another, and can be estimated by mere counting. This special case of a logistic regression model is usually referred to as the method of weights of evidence. Its practical application is restricted by the modeling assumption of joint conditional independence of all predictors given the target. The other way round, logistic regression is the canonical generalization of weights of evidence.

Boost weights-of-evidence (BoostWofE)

As with respect to weights-of-evidence, numerous attempts aim at relaxing its modeling assumption of joint conditional independence and subsequent corrections of the weights, e.g., the multiplicative \tau - and the additive \nu -correction of weights ( Journel, 2002; Polyakova and Journel, 2007; Krishnan, 2008).

BoostWofE has recently been introduced by Cheng (2015) as combining elements of weights-of-evidence and AdaBoost ( Freund and Schapire, 1997; Freund and Schapire, 1999; Hastie et al., 2009) to simplify estimation of the general weights of evidence, Eq. (11), and the additive \nu correction term of Polyakova and Journel (2007), respectively ( Cheng, 2015, p. 597).

In general, boosting turns several weak learners into a single strong learner. For the two-class problem, boosting can be viewed as an approximation to additive modeling on the logistic scale using maximum Bernoulli likelihood as a criterion ( Friedman et al., 2000) used in logistic regression. AdaBoost is a linear classifier with all its desirable properties, its output converges to the logarithm of the likelihood ratio ( Šochman and Matas, 2004). AdaBoost fits an additive logistic regression model, using a criterion similar to the binomial log-likelihood. LogitBoost directly optimizes the binomial log-likelihood ( Friedman et al., 2000).

Weights-of-evidence is the special case of logistic regression characterized by the additional assumption of jointly conditionally independent indicator predictors. Boosting weights of evidence appears to aim at improving especially weak learners depending on joint conditional independence while the first canonical improvement would be to proceed from weights-of-evidence to logistic regression.

Boost weights-of-evidence ( Cheng, 2015) processes indicator predictors to update a prior unconditional probability sequentially with the conventional weight of evidence assigned to the first predictor, and successively boosted weights assigned to subsequent predictors ( Cheng, 2015, Eq. (26), p. 602) resulting in

(16) { }^{\mathrm{b}\mathrm{oost}}{W}_{\ell }^{(i)}=W_{\ell }^{(i)}+Q_{\ell }^{(i)},i=\mathrm{0},\mathrm{1},\ell =\mathrm{1},\dots ,m,

with Q_{\mathrm{1}}^{(i)}=\mathrm{0} , and Q_{\ell }^{(i)} presumably approximating \ln {\nu }_{\ell }^{(i)} , the correction term provided by the \nu -model (Polyakova and Journel, 2007), Eq. (A2) of the Appendix A, by a weighted mean of corresponding conditional probabilities ( Cheng, 2015, p. 597). The boost terms Q_{\ell } are explicitly given as sum of (\ell -\mathrm{1}) terms, where each term is given as logarithm of a ratio of sums of two ratios of conditional probabilities each ( Cheng, 2015, Eqs. (26) and (27), pp. 602–603). For more details the reader is referred to Appendix B.

In a similar way as the additive \nu modifications of conventional weights of evidence (Appendix A), the additive modifications by successive boosting ( Cheng, 2015) of weights of evidence may allow for some small deviations from joint conditional independence in a very restricted form. They cannot emulate the effect of multiplicative interaction terms of predictors included in logistic regression models. Since Q_{\mathrm{1}}=\mathrm{0} , there are as many different boost weights-of-evidence models as permutations of m predictors {\mathsf{B}}_{\ell },\ell =\mathrm{1},\dots ,m , i.e., m! different boost models. Whether the procedure presented by Cheng (2015) to estimate the boost weights of evidence permutes correspondingly was not addressed; there is no obvious reason to assume that the order of predictors is irrelevant.

From an application with fabricated training data and a case study with observed training data, ( Cheng, 2015, p. 618) concludes that his novel boost weights-of-evidence method can significantly reduce the effect of conditional dependence in a simple and intuitive way as derived by Cheng (2015, pp. 596–607), in fact in a more generic way than other approaches ( Cheng, 2015, p. 620).

Empirical comparison

Since case studies cannot generally provide insight in the method applied, instructive examples with fabricated or simulated data are used to exemplify, expose and confirm theoretical findings. Here we primarily use the same dataset Q as Cheng (2015) for the particular purposes (i) to criticize and refute validation by case studies in general, and (ii) to disprove the major claims put forward by Cheng (2012, 2015) that BoostWofE significantly reduces the effect of lack of joint conditional independence ( Cheng, 2015, p. 618). Since the assumption of conditional independence is only mildly violated for the dataset Q, we demonstrate the effect of its serious violation using once more the dataset RANKIT ( Schaeben, 2014a, b).

Chengs’s (2015) fabricated training dataset Q

The training dataset Q is taken from the publication ( Cheng, 2015). The digital map images of spatial distributions of three indicator predictor variables {\mathsf{B}}_{\ell },\ell =\mathrm{1},\mathrm{2},\mathrm{3} , and the indicator target variable \mathrm{T} given their realizations b_{\ell ,i},t_i,i=\mathrm{1},\dots ,\mathrm{100},\ell =\mathrm{1},\dots ,\mathrm{3} , are displayed in Fig. 1.

For three indicator predictor variables there are eight different combinations of their joint possible realizations. Then there are eight corresponding conditional frequencies h(\mathsf{T}=\mathrm{1}|({\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}})=(b_{\mathrm{1}},b_{\mathrm{2}},b_{\mathrm{3}})),b_{\mathrm{1}},b_{\mathrm{2}},b_{\mathrm{3}}=\mathrm{0},\mathrm{1} , referred to as ground truth of the training dataset Q because these conditional frequencies determined by counting are unbiased estimates of the corresponding conditional probabilities P(\mathsf{T}=\mathrm{1}|({\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}})=(B_{\mathrm{1}},b_{\mathrm{2}},B_{\mathrm{3}})) . The spatial distribution of the conditional frequencies is displayed in Fig. 2, the numbers are compiled in Table 15.

Pearson and Kendall correlation

A measure of spatial association like the variogram of geostatistics is unknown in all conventional methods of prospectivity modeling. Since the general assumption of identically independent distributed random variables applies to all conventional methods of prospectivity modeling, inspection of the correlation matrices is reasonable. In case of indicator predictors and data, respectively, it is sufficient to check any of the three conventional correlations, e.g. Table 1, as Pearson, Kendall, and Spearman correlation coefficients agree.

Statistical tests of Kendall correlation coefficients are summarized in Table 2 and Table 3.

Thus, {\mathsf{B}}_{\mathrm{1}} and {\mathsf{B}}_{\mathrm{2}} are inferred to be significantly correlated with \mathsf{T} , while {\mathsf{B}}_{\mathrm{3}} seems to be rather uncorrelated with \mathsf{T} .

The tests suggest that {\mathsf{B}}_{\mathrm{1}} is neither correlated with {\mathsf{B}}_{\mathrm{2}} nor with {\mathsf{B}}_{\mathrm{3}} , and that {\mathsf{B}}_{\mathrm{2}} and {\mathsf{B}}_{\mathrm{3}} are significantly correlated for every level of significance α>3.261456e‒06.

Conditional independence

Checking for instance for ({\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}})=(\mathrm{1},\mathrm{1},\mathrm{1}) whether the observed joint conditional frequencies interpreted as elementary estimates of the corresponding probabilities can be factorized given \mathsf{T}=\mathrm{1}

\widehat{P}(({\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}})=(\mathrm{1},\mathrm{1},\mathrm{1})|\mathsf{T}=\mathrm{1})=\mathrm{0.300},

\widehat{P}({\mathsf{B}}_{\mathrm{1}}=\mathrm{1}|\mathsf{T}=\mathrm{1})*\widehat{P}({\mathsf{B}}_{\mathrm{2}}=\mathrm{1}|\mathsf{T}=\mathrm{1})*\widehat{P}({\mathsf{B}}_{\mathrm{3}}=\mathrm{1}|\mathsf{T}=\mathrm{1})=\mathrm{0.196},

or \mathsf{T}=\mathrm{0}

\widehat{P}(({\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}})=(\mathrm{1},\mathrm{1},\mathrm{1})|\mathsf{T}=\mathrm{0})=\mathrm{0.011},

\widehat{P}({\mathsf{B}}_{\mathrm{1}}=\mathrm{1}|\mathsf{T}=\mathrm{0})*\widehat{P}({\mathsf{B}}_{\mathrm{2}}=\mathrm{1}|\mathsf{T}=\mathrm{0})*\widehat{P}({\mathsf{B}}_{\mathrm{3}}=\mathrm{1}|\mathsf{T}=\mathrm{0})=\mathrm{0.010},

suggests that the mathematical modeling assumption of joint conditional independence is violated to a minor extent only. Whether these minor deviations are statistically significant or not is the objective of a corresponding statistical test.

In terms of random variables joint conditional independence of indicator predictor variables given the indicator target variable can be tested with reference to a corresponding log–linear model. The full log–linear model is of course sufficiently large as all variables are indicator variables, and therefore the joint probability, i.e., all contingency tables, can be represented as log–linear model.

The statistical test of joint conditional independence referring to the three-terms log–linear model of Table 4 leads to infer that the null–hypothesis of joint conditional independence given \mathsf{T} can reasonably be rejected for all levels of significance α>0.008 with respect to the likelihood ratio or α>0.003 with respect to Pearson statistic, i.e., the modeling assumption of joint conditional independence is significantly violated. However, the statistical tests of pairwise conditional independence referring to the corresponding two-terms log–linear models of Table 5, Table 6, and Table 7, respectively, reveal that only {\mathsf{B}}_{\mathrm{2}} and {\mathsf{B}}_{\mathrm{3}} are significantly not conditional independent given \mathrm{T} , while conditional independence of {\mathsf{B}}_{\mathrm{1}} and {\mathsf{B}}_{\mathrm{2}} , and {\mathsf{B}}_{\mathrm{1}} and {\mathsf{B}}_{\mathrm{3}} , respectively, given \mathsf{T} is violated to a statistically insignificant extent only.

In ( Cheng, 2015) it is merely concluded that joint conditional independence does not apply as pairwise conditional independence of {\mathsf{B}}_{\mathrm{2}} and {\mathsf{B}}_{\mathrm{3}} given \mathsf{T} does not apply.

Application of general weights-of-evidence

Applying the general weights of evidence, Eq. (11), estimated by counting as conventional weights of evidence results in

\begin{array}{l}\mathrm{O}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}})=\mathrm{0.111}*[\mathrm{2.739}{\mathsf{B}}_{\mathrm{1}}+\mathrm{0.402}(\mathrm{1}-{\mathsf{B}}_{\mathrm{1}})]\\ *[\mathrm{5.476}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.481}{\mathsf{B}}_{\mathrm{2}}(\mathrm{1}-{\mathsf{B}}_{\mathrm{1}})+\mathrm{0.328}(\mathrm{1}-{\mathsf{B}}_{\mathrm{2}})*{\mathsf{B}}_{\mathrm{1}}+\mathrm{0.455}(\mathrm{1}-{\mathsf{B}}_{\mathrm{2}})*(\mathrm{1}-{\mathsf{B}}_{\mathrm{1}})]\\ *[\mathrm{1.800}{\mathsf{B}}_{\mathrm{3}}{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}+\mathrm{1.000}{\mathsf{B}}_{\mathrm{3}}(\mathrm{1}-{\mathsf{B}}_{\mathrm{1}}){\mathsf{B}}_{\mathrm{2}}+\mathrm{0.600}(\mathrm{1}-{\mathsf{B}}_{\mathrm{3}}){\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}+\mathrm{1.111}(\mathrm{1}-{\mathsf{B}}_{\mathrm{3}}){\mathsf{B}}_{\mathrm{1}}(\mathrm{1}-{\mathsf{B}}_{\mathrm{2}})\\ +\mathrm{1.000}(\mathrm{1}-{\mathsf{B}}_{\mathrm{3}})(\mathrm{1}-{\mathsf{B}}_{\mathrm{1}}){\mathsf{B}}_{\mathrm{2}}+\mathrm{1.088}(\mathrm{1}-{\mathsf{B}}_{\mathrm{3}})(\mathrm{1}-{\mathsf{B}}_{\mathrm{1}})(\mathrm{1}-{\mathsf{B}}_{\mathrm{2}})]\end{array}

and perfectly recovers the ground truth, of course, cf. Fig. 2. It should be noted that the interaction terms {\mathsf{B}}_{\mathrm{3}}{\mathsf{B}}_{\mathrm{1}}(\mathrm{1}-{\mathsf{B}}_{\mathrm{2}}) and {\mathsf{B}}_{\mathrm{3}}(\mathrm{1}-{\mathsf{B}}_{\mathrm{1}})(\mathrm{1}-{\mathsf{B}}_{\mathrm{2}}) are not included as their weights vanish. The general weights of evidence could be used for the purpose of prediction in the same way as conventional weights. However, as with conventional weights of evidence, a measure for the predictive power or the reliability of the prediction is missing. Caution seems to be appropriate, as the general weights of evidence may result in overfitting.

Application of conventional weights-of-evidence assuming conditional independence

Despite the obvious violation of the required mathematical modeling assumption of joint conditional independence of all predictors given the target, weights-of-evidence is applied to the training dataset Q and results in weights and contrasts

The figures 0.898 and ‒0.477 for the conventional weights of {\mathsf{B}}_{\mathrm{2}} given with the first and second equation of Eq. (49) by Cheng (2015, p. 612) are wrong, probably typos. Consequently, the figures for the conditional probabilities estimated with conventional weights of evidence given in the right column of Eq. (52) by Cheng (2015, p. 613) are erroneous, too.

as compiled in Table 8.

With

P(\mathsf{T}=\mathrm{1})=\mathrm{0.100},\mathrm{O}(\mathsf{T}=\mathrm{1})=\mathrm{0.111},\ln (\mathrm{O}(\mathsf{T}=\mathrm{1}))=-\mathrm{2.197}

and

W^{(\mathrm{0})}=-\mathrm{2.162},\ln (\mathrm{O}(\mathsf{T}=\mathrm{1}))+W^{(\mathrm{0})}=-\mathrm{4.359}

the weights-of-evidence model reads explicitly

(17) {\widehat{P}}_{\mathrm{Wof}\mathrm{3}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.359}+\mathrm{1.917}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.037}{\mathsf{B}}_{\mathrm{2}}+\mathrm{1.126}{\mathsf{B}}_{\mathrm{3}}).

Comparing the total number of occurrences of the target event \mathsf{T}=\mathrm{1} given by the sum of the realizations t_i,i=\mathrm{1},\dots ,\mathrm{100} , with the sum of all estimated conditional probabilities (interpreted as estimated total number of occurrences of the target event \mathsf{T}=\mathrm{1} ( Cheng, 2015))

\sum t_i=\mathrm{10},\sum {\widehat{P}}_{\mathrm{Wof}\mathrm{3}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{10.257},

(18) \sum {\widehat{P}}_{\mathrm{Wof}\mathrm{3}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})-\sum t_i=\mathrm{0.257},

which is the test statistic of the so-called “new omnibus test” of conditional independence ( Agterberg and Cheng, 2002, p. 252), confirms that joint conditional independence is disturbed to a small extent only.

For the two-terms weights-of-evidence model

(19) {\widehat{P}}_{\mathrm{Wof}\mathrm{2}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}})=\mathrm{\Lambda }(-\mathrm{4.359}+\mathrm{1.917}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.037}{\mathsf{B}}_{\mathrm{2}}),

the difference

(20) \sum {\widehat{P}}_{\mathsf{Wof}\mathrm{2}\mathsf{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}})-\sum t_i=-\mathrm{0.611},

and reveals in particular a change of sign of the error compared to the three-terms weights-of-evidence model of Eq. (17).

Comparing digital map images of Fig. 3 displaying the ground truth estimated elementarily with conditional frequencies, and the conditional probabilities {\widehat{P}}_{\mathrm{Wof}\mathrm{3}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}}) and {\widehat{P}}_{\mathrm{Wof}\mathrm{2}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}) , respectively, estimated with weights of evidence reveals that weights-of-evidence with three predictors yields a corrupted pattern of predicted conditional probabilities due to the violation of the modeling assumption. Weights-of-evidence with the two predictors {\mathsf{B}}_{\mathrm{1}} and {\mathsf{B}}_{\mathrm{2}} , for which the null-hypothesis of conditional independence given \mathsf{T} could not reasonably be rejected, cf. Table 5, results in a simplified pattern reflecting merely the spatial distribution of the two predictors used for prediction.

Application of BoostWofE

Processing the predictor variables in the same order {\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}} as ( Cheng, 2015) the weights and the contrast with respect to the predictor {\mathsf{B}}_{\mathrm{1}} remain unchanged, while the weights and the contrasts with respect to predictors {\mathsf{B}}_{\mathrm{2}} and {\mathsf{B}}_{\mathrm{3}} are affected by boosting

The figures 0.018 and ‒0.016 for the boosted weights of the third predictor {\mathsf{B}}_{\mathrm{3}} given with Eq. (48) by Cheng (2015, p. 611) are likely to be typos. However their difference 0.034 is about right such that the computed predicted probabilities are not much affected.

, cf. Table 9.

With

{ }^{\mathrm{boost}\mathrm{123}}{W}^{(\mathrm{0})}=-\mathrm{1.876},\ln (\mathrm{O}(\mathrm{T}=\mathrm{1})){+}^{\mathrm{boost}\mathrm{123}}{W}^{(\mathrm{0})}=-\mathrm{4.073},

the Boost123WofE model by Cheng (2015) reads explicitly

(21) { }^{\mathrm{boost}\mathrm{123}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.073}+\mathrm{1.917}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.192}{\mathsf{B}}_{\mathrm{2}}+\mathrm{0.034}{\mathsf{B}}_{\mathrm{3}}),

and results in

\sum { }^{\mathrm{boost}\mathrm{123}}{\widehat{P}}_{\mathsf{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{9.919},

(22) \sum { }^{\mathrm{boost}\mathrm{123}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})-\sum t_i=-\mathrm{0.081.}

Comparing WofE Eq. (17) and Boost123WofE Eq. (21), in this example boosting as introduced by Cheng (2015) puts less weight on {\mathsf{B}}_{\mathrm{3}} and more weight on {\mathsf{B}}_{\mathrm{2}} resulting in a largely improved error Eq.(22) of the same sign as the error Eq.(20) of the two-terms weights-of-evidence model Eq.(19) .

Comparing the map images of Fig. 4 leads to recognize that the result of Boost123WofE largely agrees with the result of Wof2E, the application of weights-of-evidence considering {\mathsf{B}}_{\mathrm{1}} and {\mathsf{B}}_{\mathrm{2}} only. The map images just confirm the comparison of the two models given explicitly with Eq. (19) and Eq. (21), respectively.

Next, the order of boosting is changed from ({\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{3}}) to ({\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{3}}) and ({\mathsf{B}}_{\mathrm{3}},{\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}}) , respectively.

For ({\mathsf{B}}_{\mathrm{2}},{\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{3}}) Table 10 gives the corresponding figures.

With

{ }^{\mathrm{boost}\mathrm{213}}{W}^{(\mathrm{0})}=-\mathrm{1.903},\ln (O(\mathsf{T}=\mathrm{1})){+}^{\mathrm{boost}\mathrm{213}}{W}^{(\mathrm{0})}=-\mathrm{4.100}

the Boost213WofE model now reads explicitly

(23) { }^{\mathrm{boost}\mathrm{213}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.100}+\mathrm{2.088}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.036}{\mathsf{B}}_{\mathrm{2}}+\mathrm{0.140}{\mathsf{B}}_{\mathrm{3}}),

and results in

\sum { }^{\mathrm{boost}\mathrm{213}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{10.045},

(24) \sum { }^{\mathrm{boost}\mathrm{213}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})-\sum t_i=\mathrm{0.045.}

For ({\mathsf{B}}_{\mathrm{3}},{\mathsf{B}}_{\mathrm{1}},{\mathsf{B}}_{\mathrm{2}}) Table 11 gives the corresponding figures.

With

{ }^{\mathrm{boost}\mathrm{312}}{W}^{(\mathrm{0})}=-\mathrm{2.247},\ln (O(\mathsf{T}=\mathrm{1})){+}^{\mathrm{boost}\mathrm{312}}{W}^{(\mathrm{0})}=-\mathrm{4.444},

the Boost312WofE model now reads explicitly

(25) { }^{\mathrm{boost}\mathrm{312}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.444}+\mathrm{2.101}{\mathsf{B}}_{\mathrm{1}}+\mathrm{1.798}{\mathsf{B}}_{\mathrm{2}}+\mathrm{1.125}{\mathsf{B}}_{\mathrm{3}}),

and results in

\sum { }^{\mathrm{boost}\mathrm{312}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{9.377}

(26) \sum { }^{\mathrm{boost}\mathrm{312}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})-\sum t_i=-\mathrm{0.623.}

The error Eq. (24) of the Boost213WofE model Eq. (23) is small and of the opposite sign as the error Eq. (22) of the Boost123WofE model Eq. (21), while the error Eq. (26) of the Boost312WofE model Eq. (25) is about the same as of the two-terms weights-of-evidence model, Eq. (20).

Visual inspection of the map images of Fig. 5 clearly indicates that the BoostWofE model ( Cheng, 2015) depends on the choice of the first predictor variable to initiate the boost algorithm, and on the order of the subsequent predictors. Checking all permutations, cf. Fig. 5, the dataset designed by Cheng (2015) to “prove” properties of BoostWofE indicates that the relative order of predictors {\mathsf{B}}_{\mathrm{2}} and {\mathsf{B}}_{\mathrm{3}} , which were inferred to be not conditionally independent given the target \mathsf{T} , is particularly sensitive. When the predictor {\mathsf{B}}_{\mathrm{2}} precedes the predictor {\mathsf{B}}_{\mathrm{3}} , the patterns of the predictors {\mathsf{B}}_{\mathrm{1}} and {\mathsf{B}}_{\mathrm{2}} are visible in the pattern of predicted conditional probability while the pattern of {\mathsf{B}}_{\mathrm{3}} is not. Contrary, when the predictor {\mathsf{B}}_{\mathrm{3}} precedes the predictor {\mathsf{B}}_{\mathrm{2}} , the pattern of the predictor {\mathsf{B}}_{\mathrm{3}} becomes visible in the pattern of the predicted conditional probability and apparently contributes spatial resolution to the the pattern provided by {\mathsf{B}}_{\mathrm{2}} . Unfortunately, Cheng (2015) does neither discuss how to choose the first predictor variable, nor the influence of the order of the subsequently one by one included predictors.

Application of logistic regression

Three logistic regression models are fitted to the dataset Q. The first uses {\mathsf{B}}_{\mathrm{1}} and {\mathsf{B}}_{\mathrm{2}} only, Table 12, the second adds {\mathsf{B}}_{\mathrm{3}} , Table 13, and the third adds the interaction term {\mathsf{B}}_{\mathrm{2}}:{\mathsf{B}}_{\mathrm{3}} to compensate for the lack of conditional independence, Table 14.

For all logistic regression models \sum {\widehat{P}}_{\mathrm{lrM}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\sum t_i as this is a constitutive equation.

Obviously, the full logistic regression model recovers the ground truth perfectly, cf. Fig. 6. However, its fitted parameters are not significant except the so-called intercept, and therefore it represents an instance of overfitting. Thus, for the training dataset Q the full logistic model is not at all appropriate for prediction.

Generally, for indicator predictors the general weights-of-evidence approach and the full logistic regression model will always recover the ground truth. As for the training dataset Q the full logistic regression model with \sum _{\ell =\mathrm{1}}^m\left(\begin{array}{l}m\\ \ell \end{array}\right)+\mathrm{1}={\mathrm{2}}^m=\mathrm{8} terms might be rendered parsimonious compared to the general weights-of-evidence model with {\displaystyle \sum }_{\ell =\mathrm{1}}^m{\mathrm{2}}^{\ell }+\mathrm{1}={\mathrm{2}}^{m+\mathrm{1}}-\mathrm{1}=\mathrm{15} terms. Moreover, only logistic regression provides statistical significance of its fitted parameters to judge whether a model is appropriate for prediction or not.

The only significant logistic regression model with predictors {\mathrm{B}}_{\mathrm{1}} and {\mathrm{B}}_{\mathrm{2}} results in a simplified pattern quite similar to the pattern of Wof2E, cf. Fig. 3, and Boost123WofE, cf. Fig. 4. However, the notion of significance does neither exist in WofE nor in BoostWofE by Cheng (2015).

If there was any doubt in the inappropriateness of the full logistic regression model for prospectivity modeling, the large standard errors as depicted in Fig. 7 confirm that the predicted conditional probabilities cannot be considered reasonably reliable.

Summary of training dataset Q

All fitted models are explicitly compiled in Eqs. (27) to (34).

(27) {\widehat{P}}_{\mathrm{Wof}\mathrm{3}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.359}+\mathrm{1.917}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.037}{\mathsf{B}}_{\mathrm{2}}+\mathrm{1.126}{\mathsf{B}}_{\mathrm{3}}),

(28) {\widehat{P}}_{\mathrm{Wof}\mathrm{2}\mathrm{E}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.044}+\mathrm{1.917}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.037}{\mathsf{B}}_{\mathrm{2}}),

(29) { }^{\mathrm{boost}\mathrm{123}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.073}+\mathrm{1.917}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.192}{\mathsf{B}}_{\mathrm{2}}+\mathrm{0.034}{\mathsf{B}}_{\mathrm{3}}),

(30) { }^{\mathrm{boost}\mathrm{213}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.100}+\mathrm{2.088}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.036}{\mathsf{B}}_{\mathrm{2}}+\mathrm{0.140}{\mathsf{B}}_{\mathrm{3}}),

(31) { }^{\mathrm{boost}\mathrm{312}}{\widehat{P}}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.444}+\mathrm{2.101}{\mathsf{B}}_{\mathrm{1}}+\mathrm{1.798}{\mathsf{B}}_{\mathrm{2}}+\mathrm{1.125}{\mathsf{B}}_{\mathrm{3}}),

(32) {\widehat{P}}_{\mathrm{fit}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.402}+\mathrm{2.299}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.407}{\mathsf{B}}_{\mathrm{2}}),

(33) {\widehat{P}}_{\mathrm{3}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{4.421}+\mathrm{2.301}{\mathsf{B}}_{\mathrm{1}}+\mathrm{2.330}{\mathsf{B}}_{\mathrm{2}}+\mathrm{0.187}{\mathsf{B}}_{\mathrm{3}}),

(34) \begin{array}{l}{\widehat{P}}_{\mathrm{full}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})=\mathrm{\Lambda }(-\mathrm{3.806}+\mathrm{1.609}{\mathsf{B}}_{\mathrm{1}}+\mathrm{1.609}{\mathsf{B}}_{\mathrm{2}}-\mathrm{14.759}{\mathsf{B}}_{\mathrm{3}}.\\ + \mathrm{0.587}{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}-\mathrm{1.609}{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{3}}+\mathrm{14.759}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}}+\mathrm{2.708}{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}}{\mathsf{B}}_{\mathrm{3}})\mathrm{.}\end{array}

Conditional probabilities \widehat{P}(\mathsf{T}=\mathrm{1}|\mathsf{B}) predicted with several fitted models are compiled in Table 15.

Fabricated training dataset RANKIT

The fabricated training dataset RANKIT, Fig. 8, was already used earlier to demonstrate the effects of a serious violation of the mathematical modeling assumption of conditional independence ( Schaeben, 2014a, b). Here, applying boost weights-of-evidence as suggested by Cheng (2015) results in the fitted models explicitly given by

(35) { }^{\mathrm{boost}\mathrm{12}}{P}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}})=\mathrm{\Lambda }(-\mathrm{3.074}+\mathrm{1.725}{\mathsf{B}}_{\mathrm{1}}+\mathrm{1.157}{\mathsf{B}}_{\mathrm{2}}),

(36) { }^{\mathrm{boost}\mathrm{21}}{P}_{\mathrm{WofE}}(\mathsf{T}=\mathrm{1}|{\mathsf{B}}_{\mathrm{1}}{\mathsf{B}}_{\mathrm{2}})=\mathrm{\Lambda }(-\mathrm{2.997}+\mathrm{1.372}{\mathsf{B}}_{\mathrm{1}}+\mathrm{1.349}{\mathsf{B}}_{\mathrm{2}}).

First, Eqs. (35) and (36) confirm again that the fitted models depend on the order of processing the two predictors. However, here they are quite similar, but both of them do not reduce the effect of lacking conditional independence. In fact, Figure 9 clearly reveals that whatever the order boosting does not generally provide a means to compensate the lack of conditional independence but yields similar patterns of predicted conditional probabilities as conventional weights-of-evidence or logistic regression without the interaction term {\mathrm{B}}_{\mathrm{1}}:{\mathrm{B}}_{\mathrm{2}} . Including the interaction term results in the full logistic regression model which is significant ( Schaeben, 2014b) and recovers the ground truth given in terms of conditional frequencies by counting.

Discussion of results

General weights of evidence are not “very difficult if not impossible” to estimate “given a limited number of training data” ( Cheng, 2015, p. 597). In fact, if conditional probabilities being 0 or 1 can reasonably be excluded, they are estimated by counting frequencies of occurrences of several events like conventional weights. Since the target event \mathsf{T}=\mathrm{1} is usually a very rare event in prospectivity modeling, chances are fairly small that the denominators P(({\mathsf{Z}}_{\ell },\mathsf{T})=(b_{j,\ell },\mathrm{0})) involved in the Bayes factors, Eq. (8), referring to \mathsf{T}=\mathrm{0} vanish. When applying general weights-of-evidence to the limited number of training data as provided by the dataset q ( Cheng, 2015) problems were not encountered. Thus, difficulties to be resolved by boost weights of evidence do not seem to exist.

Given the ten pages it takes ( Cheng, 2015, pp. 596–607) to derive the procedure of boost weights-of-evidence relying on an ad hoc approximation which is not justified in any way, it cannot be confirmed that BoostWofE is simple and intuitively appealing as compared with logistic regression, Eq. (5), for instance. Like conventional weights-of-evidence and opposed to logistic regression, boost weights-of-evidence lacks the notion of significance of fitted model parameters.

As for the fabricated training data set Qused by Cheng (2015), the statistical test of the null–hypothesis of joint conditional independence of all three predictors {\mathsf{B}}_{\ell },\ell =\mathrm{1},\mathrm{2},\mathrm{3} , given the target \mathsf{T} leads to infer to reasonably reject it. However, it can be rejected because {\mathsf{B}}_{\mathrm{2}} and {\mathsf{B}}_{\mathrm{3}} are not conditional independent given \mathsf{T} , while the null hypothesis cannot reasonably be rejected for the other pairs of predictors.

Then, the fabricated training data set Q obviously exemplifies that the results of boost weights-of-evidence largely depend on the boosting sequence of predictors. It has to be noted that criteria for the user decision how to choose the sequence are not provided in ( Cheng, 2015). Whatever the boosting sequence, boost weights-of-evidence does not reduce the effect of violated conditional independence but leads either to corrupted patterns of predicted conditional probabilities as conventional weights-of-evidence does, or to simplified patterns similar to those accomplished by omitting {\mathsf{B}}_{\mathrm{3}} from the prediction.

As with respect to the training dataset RANKIT which is significantly lacking joint conditional independence, application of boost weights-of-evidence results in the same corrupted pattern of predicted conditional probabilities as conventional weights-of-evidence.

In fact, boost weights-of-evidence method as introduced by Cheng (2015) does not generally reduce the effect of lacking conditional independence, not to mention the simple, intuitive and more generic way as claimed by Cheng (2015, p. 620). Non of the claims by Cheng (2015) could be verified.

It is commonplace that examples cannot substitute mathematical proofs, while one counterexample is sufficient to disprove a mathematical statement. In the same way, it is not possible to derive properties of an ad hoc procedure nor to validate it by way of an example with fabricated or observed training data. It is impossible to judge the performance of an ad hoc procedure by way of its application to observed training data. It takes mathematical-statistical analysis to derive a method, to turn it into an algorithm, and to encode it in software.

Conclusions

Boost weights-of-evidence ( Cheng, 2015) is another improper attempt to relax the mathematical modeling assumption of joint conditional independence of all predictors given the target hampering reliable predictions of prospectivity. Boost weights-of-evidence does not generally reduce not to mention significantly reduce the effect of lacking joint conditional independence. Its application yields corrupted predicted conditional probabilities and corrupted spatial patterns of prospectivity as weights-of-evidence does. In particular, its results depend on the sequential processing order of predictors. General weights of evidence do not require the modeling assumption of joint conditional independence of all predictors given the target, and can be estimated by mere counting like conventional weights. Logistic regression provides a relative measure of fit, and can distinguish significantly fitted models appropriate for predictions.

References

[1] Agterberg F P (2014). Geomathematics: Theoretical Foundations, Applications and Future Developments. Cham, Heidelberg, New York, Dordrecht, London: Springer

[2] Agterberg F P, Bonham-Carter G F, Wright D F (1990). Statistical pattern integration for mineral exploration. In: Gaál G, Merriam D F, eds. Computer Applications in Resource Estimation Prediction and Assessment for Metals and Petroleum. Oxford, New York: Pergamon Press, 1–21

[3] Agterberg F P, Cheng Q (2002). Conditional independence test for weights-of-evidence modeling. Nat Resour Res, 11(4): 249–255

[4] Berkson J (1944). Application of the logistic function to bio-assay. J Am Stat Assoc, 39(227): 357–365

[5] Bonham-Carter G (1994). Geographic Information Systems for Geoscientists: Modeling with GIS. New York: Pergamon, Elsevier Science

[6] Butz C J, Sanscartier M J (2002). Properties of weak conditional independence. In: Alpigini J J, Peters J F, Skowron A, Zhong N, eds. Rough Sets and Current Trends in Computing, Lecture Notes in Computer Science (Volume 2475). Berlin, Heidelberg: Springer, 349–356

[7] Chalak K, White H (2012). Causality, conditional independence, and graphical separation in settable systems. Neural Comput, 24(7): 1611–1668

[8] Cheng Q (2012). Application of a newly developed boost weights of evidence model (BoostWofE) for mineral resources quantitative assessments. Journal of Jilin University, Earth Sci Ed, 42(6): 1976–1985

[9] Cheng Q (2015). BoostWofE: a new sequential weights of evidence model reducing the effect of conditional dependency. Math Geosci, 47(5): 591–621

[10] Chilès J P, Delfiner P (2012). Geostatistics- Modeling Spatial Uncertainty (2nd ed). New York, Chichester, Weinheim, Brisbane, Singapore, Toronto: John Wiley & Sons

[11] Dawid A P (1979). Conditional independence in statistical theory. J R Stat Soc, B, 41(1): 1–31

[12] Dawid A P (2004). Probability, causality and the empirical world: a Bayes-de Finetti-Popper-Borel synthesis. Stat Sci, 19(1): 44–57

[13] Dawid A P (2007). Fundamentals of Statistical Causality. Research Report 279, Department of Statistical Science, University College London ESRI, ArcGIS.

[14] Ford A, Miller J M, Mol A G (2016). A comparative analysis of weights of evidence, evidential belief functions, and fuzzy logic for mineral potential mapping using incomplete data at the scale of investigation. Nat Resour Res, 25(1): 19–33

[15] Freund Y, Schapire R E (1997). A decision theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci, 55(1): 119–139

[16] Freund Y, Schapire R E (1999). A short introduction to boosting. Jinko Chino Gakkaishi, 14(5): 771–780

[17] Friedman J, Hastie T, Tibshirani R (2000). Additive logistic regression: a statistical view of boosting. Ann Stat, 28(2): 337–407

[18] Good I J (1950). Probability and the Weighing of Evidence. London: Griffin

[19] Good I J (1960). Weight of evidence, corroboration, explanatory power, information and the utility of experiments. J R Stat Soc, B, 22(2): 319–331

[20] Good I J (1968). The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Research Monograph No. 30, The MIT Press, Cambridge, MA, 109

[21] Harris D P, Pan G C (1999). Mineral favorability mapping: a comparison of artificial neural networks, logistic regression and discriminant analysis. Nat Resour Res, 8(2): 93–109

[22] Harris D P, Zurcher L, Stanley M, Marlow J, Pan G (2003). A comparative analysis of favorability mappings by weights of evidence, probabilistic neural networks, discriminant analysis, and logistic regression. Nat Resour Res, 12(4): 241–255

[23] Hastie T, Tibshirani R, Friedman J (2009). The Elements of Statistical Learning (2nd ed). New York: Springer

[24] Hosmer D W, Lemeshow S, Sturdivant R X (2013). Applied Logistic Regression (3rd ed). Hoboken, NJ: Wiley & Sons

[25] Journel A G (2002). Combining knowledge from diverse sources: an alternative to traditional data independence hypotheses. Math Geol, 34(5): 573–596

[26] Kreuzer O, Porwal A, eds. (2010). Special Issue “Mineral Prospectivity Analysis and Quantitative Resource Estimation”. Ore Geol Rev, 38(3): 121–304

[27] Krishnan S (2008). The τ-model for data redundancy and information combination in Earth sciences: theory and application. Math Geol, 40(6): 705–727

[28] Minsky M, Selfridge O G (1961). Learning in random nets. In: Cherry C, ed. 4th London Symposium on Information Theory. London: Butterworths, 335–347

[29] Pearl J (2009). Causality: Models, Reasoning, and Inference. 2nd ed.New York: Cambridge University Press

[30] Polyakova E I, Journel A G (2007). The ν. Math Geol, 39(8): 715–733

[31] Porwal A, Carranza E J M (2015). Introduction to the Special Issue: GIS-based mineral potential modelling and geological data analyses for mineral exploration. Ore Geol Rev, 71: 477–483

[32] Porwal A, González-Álvarez I, Markwitz V, McCuaig T C, Mamuse A (2010). Weights of evidence and logistic regression modeling of magmatic nickel sulfide prospectivity in the Yilgarn Craton, Western Australia. Ore Geol Rev, 38(3): 184–196

[33] Reed L J, Berkson J (1929). The application of the logistic function to experimental data. J Phys Chem, 33(5): 760–779

[34] Rodriguez-Galiano V, Sanchez-Castillo M, Chica-Olmo M, Chica-Rivas M (2015). Machine learning predictive models for mineral prospectivity: an evaluation of neural networks, random forest, regression trees and support vector machines. Ore Geol Rev, 71: 804–818

[35] Schaeben H (2014a). Targeting: logistic regression, special cases and extensions. ISPRS Int J Geoinf, 3(4): 1387–1411

[36] Schaeben H (2014b). Potential modeling: conditional independence matters. GEM-International Journal on Geomathematics, 5(1): 99–116

[37] Schaeben H (2014c). A mathematical view of weights-of-evidence, conditional independence, and logistic regression in terms of Markov random fields. Math Geosci, 46(6): 691–709

[38] Šochman J, Matas J (2004). Adaboost with totally corrective updates for fast face detection. In: Proc. 6th IEEE International Conference on Automatic Face and Gesture Recognition, Seoul, South Korea, 445–450

[39] Suppes P (1970). A Probabilistic Theory of Causality. Amsterdam: North-Holland

[40] Tolosana-Delgado R, van den Boogaart K G, Schaeben H (2014). Potential mapping from geochemical surveys using a Cox process. 10th Conference on Geostatistics for Environmental Applications, Paris, July 9–11, 2014

[41] van den Boogaart K G, Schaeben H (2012). Mineral potential mapping using Cox–type regression for marked point processes. 34th IGC Brisbane, Australia

[42] Wong M S K M , Butz C J (1999). Contextual weak independence in Bayesian networks. In: Proc. 15th Conference on Uncertainty in Artificial Intelligence, Stockholm, Sweden, 670–679

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg