Adaptive regularized scheme for remote sensing image fusion

Abstract

We propose an adaptive regularized algorithm for remote sensing image fusion based on variational methods. In the algorithm, we integrate the inputs using a “grey world” assumption to achieve visual uniformity. We propose a fusion operator that can automatically select the total variation (TV)–L₁ term for edges and L₂-terms for non-edges. To implement our algorithm, we use the steepest descent method to solve the corresponding Euler–Lagrange equation. Experimental results show that the proposed algorithm achieves remarkable results.

Keywords

remote sensing image fusion / adaptive regulariser / variational method / steepest descent method

Introduction

Image fusion (Schowengerdt, 2006) is an effective method to show various types of image information in a single image by combining information acquired through various sources; this process is important in many areas such as the synthesis of multi-focused images (Wang et al., 2008; Chai et al., 2012), medical imaging (Yang et al., 2008; James and Dasarathy, 2014), and remote sensing (Wang et al., 2005; Fang et al., 2013).

In particular, given two or more properly aligned imaging data from different sources, image fusion can integrate all inherent complementary information into a composite, i.e., the fused image carries significantly more effective information (Wang et al., 2005; Schowengerdt, 2006) and provides a more accurate description for human visual perception or computer processing tasks than individual images. The fused image contains more valuable information and less useless information through the elimination of noise, decreased uncertainty, and improved reliability (Cvejic et al., 2009).

Many pixel-wise image fusion methods (Stathaki, 2008), such as the wavelet- or shearlet-based approaches (Li et al., 2010; Rahman et al., 2010; Kim et al., 2011; Liu and Shen, 2012; Ellmauthaler et al., 2013; Gao et al., 2013; Liu et al., 2014; Wang et al., 2014), support vector machines (Pouteau et al., 2010), neural network (Agrawal and Singhai, 2010; Kong et al., 2011; El-taweel and Helmy, 2013), and sparse methods (Yang and Li, 2010; Liu and Shen, 2012; Yang and Yang, 2013) have been proposed. Variational methods have recently been developed to be a standard approach to handle many practical topics in image processing effectively. Successful applications include compressive sensing reconstruction (Lustig et al., 2007; Tafti et al., 2011), de-noising (Tran et al., 2012), and inpainting (Zhang and Chan, 2010). By solving the minimum of an energy functional corresponding to a problem, the prospective result can be obtained. Compared with other schemes, variational methods have remarkable advantages in both theory and implementation (Sapiro, 2001; Chan et al., 2003; Aubert and Kornprobst, 2009). Owing to the outstanding performances of such an approach, researchers have introduced variational methods into the image fusion domain and proposed several image fusion methods (Socolinsky and Wolff, 2002; Chao and Ye, 2007; Wang et al., 2008; Kumar and Dass, 2009; Piella, 2009; Kluckner et al., 2010; Yuan et al., 2012; Fang et al., 2013).

Among these variational fusion methods, Socolinsky and Wolff (2002) defined the contrast of a multi-band image and then proposed a variational paradigm for image fusion. To improve visual performance, Wang et al. (2008) proposed a fusion scheme based on perceptual contrast enhancement. Further, Piella (2009) proposed a gradient-based enhancement fusion model, in which the tensor of a target structure was first obtained on the basis of geometric combination. A variational approach that combined geometric merging, intensity correction, and perceptual local contrast enhancement was then used. This method can achieve excellent results. In addition, Fang et al. (2013) proposed an improved variational method based on Piella’s method. First, Fang removed the perceptual contrast enhancement term used by Piella. Second, Fang used total variation (TV)-L₁ (L₁-norm) as the target gradient regularize to build an energy functional, which can be easily implemented on some existing fast algorithms. Furthermore, Yuan et al. (2012) studied the convex optimization model based on the standard technique of TV-L₂ image approximation and then extended this approach to the TV-L₁ model under the perceptive of primal and dual, which works well in preserving edges.

All these variational methods use only a single regularizer. The L₂-norm may lead to an over-smoothing on edges, and the quadratic data term is not robust against strong outliers in the observed data (Pock et al., 2011). The TV-L₁ regularizer (Yuan et al., 2012; Fang et al., 2013) has desirable properties to preserve sharp discontinuities and is thus effective in removing strong outliers (Nikolova, 2004). However, the staircase effect produced by TV-L₁ regularization becomes apparent when images contain not only flat, but also slanted regions. Remote sensing imaging objects are usually complicated, such that an alternative regularizer can reasonably be adopted according to different image contents.

Based on the above understanding, considerable effort has been exerted to improve the performance of TV regularizers (Lysaker et al., 2003; Hinterberger and Scherzer, 2006; Pock et al., 2011; Papafitsoros and Schönlieb, 2014). However, efforts on image fusion are relatively limited. In this study, we propose an adaptive regularized fusion method, which adopts the TV-L₁ or L₂-norm prior as a regularizer according to flat areas or edges, respectively. The proposed approach integrates the target gradient technique (Piella, 2009) and the “grey world” assumption (Buchsbaum, 1980). In addition, we propose the combination of the input images with the average quadratic local dispersion measure (Bertalmio et al., 2007) to make the fused image highly uniform and natural visually. Experimental results on large-scale remote sensing images demonstrate the favorable properties of the proposed method.

Related work

We start by reviewing some image de-noising models which serve as the bases of image fusion. In their original formulations, some models consider only one observation in the data term. By contrast, we will consider the case with multiple observations in fusion processing.

Quadratic model

The quadratic model (Tikhonov, 1943; Socolinsky, 2000) is the simplest regularization method, which is defined as

(1)\underset{u}{\min }\{\alpha {\displaystyle \underset{\mathrm{\Omega }}{\int }{\left|\nabla u\right|}^{\mathrm{2}}dx+{\displaystyle \sum _{l=\mathrm{1}}^k{\displaystyle \underset{\mathrm{\Omega }}{\int }{\left|u-f_l\right|}^{\mathrm{2}}dx}}}\},

where \mathrm{\Omega }\in R^{\mathrm{2}} is the image domain, k is the number of observed images, and f_l:\mathrm{\Omega }\to R denotes a single observation. The free parameter \alpha \ge \mathrm{0} is used to control the amount of smoothing in u. The first term is the regularization term, which reflects the smoothness assumption, whereas the second term measures the distance between the solutions to the observed data.

Rudin–Osher–Fatemi (ROF) model

L₁ estimation procedures have been found to be effective for many problems. The first L₁ estimation method was the ROF model for image de-noising (Rudin et al., 1992), the unconstrained variational form of which is

(2)\underset{u}{\min }\{\alpha {\displaystyle \underset{\mathrm{\Omega }}{\int }\left|\nabla u\right|dx+{\displaystyle \sum _{l=\mathrm{1}}^k{\displaystyle \underset{\mathrm{\Omega }}{\int }{\left|u-f_l\right|}^{\mathrm{2}}dx}}}\}.

The first term is the so-called TV semi-norm of u.

TV-L₁ model

The TV-L₁ model (Chan and Esedoglu, 2005; Yuan et al., 2012) is obtained from the ROF model by replacing the L₂ norm in the data term with the L₁ norm. The basic form is

(3)\underset{u}{\min }\{\alpha {\displaystyle \underset{\mathrm{\Omega }}{\int }\left|\nabla u\right|dx+{\displaystyle \sum _{l=\mathrm{1}}^k{\displaystyle \underset{\mathrm{\Omega }}{\int }\left|u-f_l\right|dx}}}\}.

The L₁ norm makes the TV-L₁ model more effective than the ROF model for removing strong outliers (Nikolova, 2004).

Proposed fusion method

We now present our fusion scheme. Let u:\mathrm{\Omega }\to [\mathrm{0},\mathrm{1}] be a greyscale image, where Ω represents the image domain. For a given pointx=(x_{\mathrm{1}},x_{\mathrm{2}})\in \mathrm{\Omega }, u(x) represents the intensity value at x. The intensity change information is usually captured by the magnitude of the gradient field \nabla u={\left({\displaystyle \frac{\partial u}{x_{\mathrm{1}}}},{\displaystyle \frac{\partial u}{x_{\mathrm{2}}}}\right)}^T. Thus, \left|\nabla u\right| indicates the change in size, whereas {\displaystyle \frac{\nabla u}{\left|\nabla u\right|}} indicates the change in direction. Generally, a larger \left|\nabla u\right| implies clearer details.

For M original input images u_n:\mathrm{\Omega }\to [\mathrm{0},\mathrm{1}],n=1,\cdots ,M, we aim to produce a composite image u:\mathrm{\Omega }\to [\mathrm{0},\mathrm{1}], which not only possesses the local salient information from all inputs, but also reduces the staircase effect while remaining perceptually uniform and natural. For this purpose, we propose our variational model as

(4)\underset{u}{\min }E(u)=\left\{{\displaystyle {\int }_{\mathrm{\Omega }}{\left|\nabla u-V_w\right|}^{\mathrm{2}}dx+\eta {\displaystyle {\int }_{\mathrm{\Omega }}{(u-u_{\mathrm{0}})}^{\mathrm{2}}dx+\lambda {\displaystyle {\int }_{\mathrm{\Omega }}{\left(u-{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}\right)}^{\mathrm{2}}dx+\beta {\displaystyle {\int }_{\mathrm{\Omega }}{\left|\nabla u\right|}_t^{t}dx}}}}\right\},

where \eta \ge \mathrm{0},\lambda \ge \mathrm{0},\beta \ge \mathrm{0} are free parameters to control the influences of image enhancement on the fusion result. u_{\mathrm{0}} is the initial image, and V_w is the target gradient. In this work, the target gradient technique is introduced by Piella (2009). By combining the gradients of inputs into a target gradient V_w, image fusion can be regarded as the task of finding a fused image with gradient \nabla u that is similar to V_w.

The first term of Eq. (4) accounts for the deviation of the salient features of the fused image from the target gradient, and its minimum ensures that the gradient of the fused image is approximate to the target gradient. The second term penalizes the deviation from u_{\mathrm{0}}, which is the weighted sum of input images to be discussed later. The function of this term is to reduce the difference between the fused image and the original source images. The third term penalizes the deviation with respect to the presumptive theoretical grey mean 1/2 (Buchsbaum, 1980), thereby guaranteeing that the result is perceptually uniform and natural. The last term is an adaptive regularizer according to image content. Primarily, the TV-L₁ prior and the TV-L₂ norm prior are arranged toward edges and non-edges, respectively. That is, the last term is defined as

{\displaystyle {\int }_{\mathrm{\Omega }}{\left|\nabla u\right|}_t^{t}dx},\mathrm{where}t=\{\begin{array}{l}\mathrm{1},\mathrm{if}x\in \mathrm{edges}\hfill \\ \mathrm{2},\mathrm{if}x\in non-edges\hfill \end{array}.

Several methods for obtaining u_{\mathrm{0}} are available. Piella adopted the weighted function

(5)w_n={\displaystyle \frac{\left|\nabla u_n(x)\right|}{{({{\displaystyle {\sum }_{m=\mathrm{1}}^M\left|\nabla u_m(x)\right|}}^{\mathrm{2}})}^{\raisebox{1ex}{$\mathrm{1}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{2}$}\right.}}}

to construct u₀ and V_w. Fang et al. (2013) used

(6)w_n={\displaystyle \frac{\left|\nabla u_n(x)\right|}{{\displaystyle {\sum }_{m=\mathrm{1}}^M\left|\nabla u_m(x)\right|}}}

to compute the weight coefficient for each image and then obtain u₀ and V_w. Moreover, Fang used TV-L₁ as regularizer for implemented by a fast algorithm.

Given that a larger V_w corresponds to a visually clearer image, we use

(7)w_n={\displaystyle \frac{\exp (\left|\nabla u_n(x)\right|)}{{({{\displaystyle {\sum }_{m=\mathrm{1}}^M(\exp (\left|\nabla u_m(x)\right|))}}^{\mathrm{2}})}^{\raisebox{1ex}{$\mathrm{1}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{2}$}\right.}}}

to obtain the initial image u₀ and then construct the target gradient V_w as V_w={\displaystyle {\sum }_{n=\mathrm{1}}^M{w}_n\times \nabla u_n}. In other words, we use the magnification of the exponential function to enlarge the target gradient appropriately for a clearer fusing result.

The usual way to distinguish edges and non-edges is based on the edge extraction technique. Commonly used methods include Sobel, Prewitt, Roberts, Canny, and Kirsch operators. Given that each operator has its advantages and disadvantages, we propose a fusion of multiple operators to extract image edges as much as possible. The basic idea is that we first use the five aforementioned techniques to extract image edges and then adopt the maximum rule to obtain the final edge image. Figure 1 exemplifies the edge-extraction, which reveals that the fused method (Fig. 1(g)) extracts more detailed edges.

After obtaining the edge and non-edge information, we define a label function L as

(8)L=\{\begin{array}{ll}\mathrm{1},\hfill & \mathrm{edges}\hfill \\ \mathrm{0},\hfill & \mathrm{otherwise}\hfill \end{array}.

Then Eq. (4) can then be rewritten as:

(9)\underset{u}{\min }E(u)=\left\{{\displaystyle {\int }_{\mathrm{\Omega }}{\left|\nabla u-V_w\right|}^{\mathrm{2}}dx+\eta {\displaystyle {\int }_{\mathrm{\Omega }}{(u-u_{\mathrm{0}})}^{\mathrm{2}}dx+\lambda {\displaystyle {\int }_{\mathrm{\Omega }}{\left(u-{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}\right)}^{\mathrm{2}}dx+\beta {\displaystyle {\int }_{\mathrm{\Omega }}(L•{\left|\nabla u\right|}_{\mathrm{1}}^{\mathrm{1}}+(\mathrm{1}-L)•{\left|\nabla u\right|}_{\mathrm{2}}^{\mathrm{2}})dx}}}}\right\},

where • is the dot product.

Numerical algorithm for the proposed method

We now discuss the implementation of Eq. (9). Firstly, we rewrite this equation as

(10)\underset{u}{\min }E(u)={\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}{\Vert \nabla u-V_w\Vert }^{\mathrm{2}}+{\displaystyle \frac{\eta }{\mathrm{2}}}{\Vert u-{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}\Vert }^{\mathrm{2}}+\beta {\Vert L•\nabla u\Vert }_{\mathrm{1}}^{\mathrm{1}}+\beta {\Vert (\mathrm{1}-L)•\nabla u\Vert }_{\mathrm{2}}^{\mathrm{2}}.

The Euler–Lagrange equation is

(11)\delta E(u)=(div{V}_w-\mathrm{\Delta }u)+\eta (u-u_{\mathrm{0}})+\lambda \left(u-{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}\right)-\beta \left(L•div\left({\displaystyle \frac{\nabla u}{\left|\nabla u\right|}}\right)+\mathrm{2}(\mathrm{1}-L)•\mathrm{\Delta }u\right).

Given that our equation involves both linear and non-linear terms, we intend to choose the classic steepest descent method. In addition, the steepest descent solution of the Laplacian operator is a Gaussian smoothing operation with increasing variance of the initial condition. Based on the steepest decent method, the above problem is transformed to solve

(12){\displaystyle \frac{\partial u}{\partial t}}=-\delta E(u).

When discretised with respect to parameter t, Eq. (12) becomes

(13){\displaystyle \frac{u^{k+\mathrm{1}}-u^k}{\mathrm{\Delta }t}}=-\left((div{V}_w-\mathrm{\Delta }{u}^k)+\eta (u^k-u_{\mathrm{0}})+\lambda \left(u^k-{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}\right)-\beta \left(L•div\left({\displaystyle \frac{\nabla u^k}{\left|\nabla u^k\right|+\xi }}\right)+\mathrm{2}(\mathrm{1}-L)•\mathrm{\Delta }{u}^k\right)\right),

where \mathrm{\Delta }t\ge \mathrm{0} is the setting constant as the iteration step size, and ξ is a small constant that avoids division by zero. Hence, the iteration equation can be expressed as

(14)u^{k+\mathrm{1}}=u^k-\mathrm{\Delta }t\left((div{V}_w-\mathrm{\Delta }{u}^k)+\eta (u^k-u_{\mathrm{0}})+\lambda \left(u^k-{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}\right)-\beta \left(L•div\left({\displaystyle \frac{\nabla u^k}{\left|\nabla u^k\right|+\xi }}\right)+\mathrm{2}(\mathrm{1}-L)•\mathrm{\Delta }{u}^k\right)\right).

We approximate the divergence of V_w by the backward difference and realize the Laplacian operator by

\mathrm{\Delta }{u}^k(x_{\mathrm{1}},x_{\mathrm{2}})=u^k(x_{\mathrm{1}}+\mathrm{1},x_{\mathrm{2}})+u^k(x_{\mathrm{1}}-\mathrm{1},x_{\mathrm{2}})+u^k(x_{\mathrm{1}},x_{\mathrm{2}}+\mathrm{1})+u^k(x_{\mathrm{1}},x_{\mathrm{2}}-\mathrm{1})-\mathrm{4}{u}^k(x,x).

Similar to many tasks in image processing, we extend the images symmetrically for the boundary region.

With the use of the above solver, the overall procedure of the proposed method can be shown in Algorithm 1.

Algorithm1: Adaptive regularize for image fusion

• Input: The original images u_n:n=\mathrm{1},\cdots ,M

• Compute w_n:w_n={\displaystyle \frac{\exp (\left|\nabla u_n\right|)}{{({{\displaystyle {\sum }_{n=\mathrm{1}}^M(\exp (\left|\nabla u_n\right|))}}^{\mathrm{2}})}^{\raisebox{1ex}{$\mathrm{1}$}\!\left/ \!\raisebox{-1ex}{$\mathrm{2}$}\right.}}}

• Initialise: u_{\mathrm{0}}={\displaystyle {\sum }_{n=\mathrm{1}}^M{w}_n\times u_n}

• Construct V_w: V_w={\displaystyle {\sum }_{n=\mathrm{1}}^M{w}_n\times \nabla u_n}

• Fixed : \eta ,\lambda ,\beta ,\xi ,tol

• while {\displaystyle \frac{\Vert u^{k+\mathrm{1}}-u^k\Vert }{\Vert u^k\Vert }}>tol

u^{k+\mathrm{1}}=u^k-\mathrm{\Delta }t\left((div{V}_w-\mathrm{\Delta }{u}^k)+\eta (u^k-u_{\mathrm{0}})+\lambda \left(u^k-{\displaystyle \frac{\mathrm{1}}{\mathrm{2}}}\right)-\beta \left(L•div\left({\displaystyle \frac{\nabla u^k}{\left|\nabla u^k\right|+\xi }}\right)+\mathrm{2}(\mathrm{1}-L)•\mathrm{\Delta }{u}^k\right)\right).

k=k+\mathrm{1}

• End while

• Output: the fused image u.

Experimental results

To evaluate our algorithm, we tested the proposed method on Petrovic’s image database, which includes three types of images: 1) urban, industrial, and natural scenes collected from the USA Airborne Multi-Sensor Pod System program (AMPS Programme, 1998)<FootNote>

AMPS Programme, http://info.amps.gov:2080, 1998.

</FootNote>; 2) hyper-spectral images of natural scenarios acquired by Bristol University for the UK Defence Research Agency (Brelstaff et al., 1995); and 3) multi-focus and extreme exposure image groups. A more detailed description is given in Petrović(2004) and Zheng et al. (2007). We only use the remote sensing part of this database, which contains 120 images.

All experiments are implemented in Matlab 7.12 and run on an Intel(R) 2.33 GHz machine with 2 GB RAM. All images are at a size of 256\times256. We set parameters as η=0.1, λ=0.02, β=0.01, ξ=10^‒7, tol=10^‒5and a kernel w=3 of Gaussian shape with standard deviation σ=0.1 to smooth noise. The algorithm terminates after 2000 iterations if the stopping criteria were not satisfied.

To perform quantitative analysis, we use seven evaluation metrics: 1) objective quality fusion measure Q_w , 2) objective quality fusion measure Q^f, 3) entropy E , 4) average gradient (AG), 5) mutual information (MI), 6) spatial frequency (SF), and 7) visual information fidelity for fusion (VIFF). For detailed definitions, please see the supplementary material.

Some results on the 120 pairs of remotely sensed images are shown in Figs. 2‒4. The comparison schemes include the proposed method, as well as the methods of Piella, Yuan, and Fang. In each figure, (a) and (b) are source images, whereas (c)–(f) represent the resulting images produced by Piella, Yuan, Fang and the proposed method, respectively.

Tables 1 to 3, which are related to Figs. 2 to 4, show the values of Q_w, Q^f, E, AG, MI, SF, and VIFF for the image fusion schemes. Bold values in each column indicate the best result among all methods.

Figure 2 illustrates the fusion of two images, as well as the fused results. The first image is useful for soil–vegetation differentiation and for distinguishing forest types. In Fig. 2(a), buildings, roads, and trees are clearly discernible. The second image is more convenient for highlighting green vegetation and for detecting tree-road interfaces. By integrating Fig. 2(a) into (b), the fused results contain most features of both input images. By examining the roads, building roofs, and middle part of the right region of each fused image carefully, Fig. 2(c) produces some over-smoothing effects, Fig. 2(d) preserves sharp discontinuities, Fig. 2(e) increases the contrast of Fig. 2(d) and Fig. 2(f) strongly resembles band 2 in Fig .2(b). All comparative methods, i.e., Piella, Yuan, and Fang’s methods, produce block-like effects in slanted regions, i.e., the regions on the building roofs, thus producing staircase effects on roads. The proposed method, i.e., Fig. 2(f), circumvents these problems and produces a significantly clearer and natural fusion image than those produced by other approaches.

Table 1 shows that the proposed method achieves the best performance in terms of Q_w, Q^f, AG, SF and VIFF, whereas Yuan’s method is the best in terms of MI and E.

Similar conclusions can be drawn from Fig. 3. By comparing the left, bottom left, and bottom right areas of each fused image, the proposed method reveals more details, preserves all salient features, and produces a more natural perspective than other methods. Our method [Fig. 3(f)] clearly outperforms the other methods visually. The value in Table 2 quantitatively shows the superiority of the proposed method.

Another example is shown in Fig. 4. By comparing the upper left, middle, and bottom areas of each fused image, we again observe that the proposed method (Fig. 4(f)) obtains a more visually uniform and natural overall perspective than the other methods.

In Table 3, the proposed fusion method achieves the best performance in terms of Q_w, Q^f, MI, E and VIFF, whereas Yuan’s method achieves the best AG and SF. This finding is consistent with the subjective visual comparisons.

From these figures and tables, we can draw a conclusion that the proposed method generally outperforms the other methods described in this work.

Conclusions

We introduced an adaptive regularized scheme for remotely sensed image fusion. The proposed method 1) extracts the gradient features from the original images, 2) integrates these features into a new target gradient, 3) adaptively arranges the TV-L₁ and TV-L₂ regularizer according to edges and non-edges respectively, 4) integrates the inputs using the GW assumption, and 5) obtains the fused image by using a variational method.

The steepest descent method is adopted for implementation. Remote sensing images are used to validate the proposed method, and the performances are evaluated both subjectively and objectively. Compared with the methods by Piella, Yuan, and Fang, the proposed method has favorable properties, as verified by the experimental results. Therefore, we can conclude that the proposed method is generally better than the compared schemes with respect to the relative evaluation criteria.

References

[1] Agrawal D, Singhai J (2010). Multi-focus image fusion using modified pulse coupled neural network for improved image quality. IET Image Processing, 4(6): 443–451

[2] Aubert G, Kornprobst P (2009). Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. Applied Mathematical Sciences. Berlin: Springer-Verlag

[3] Bertalmio M, Caselles V, Provenzi E, Rizzi A (2007). Perceptual color correction through variational techniques. IEEE Trans Image Process, 16(4): 1058–1072

[4] Brelstaff G J, Párraga A, Troscianko T, Carr D (1995). Hyper spectral camera system: acquisition and analysis. In: Satellite Remote Sensing II. International Society for Optics and Photonics: 150–159

[5] Buchsbaum G (1980). A spatial processor model for object color perception. J Franklin Inst, 310(1): 1–26

[6] Chai Y, Li H, Zhang X (2012). Multi-focus image fusion based on features contrast of multi-scale products in non-subsampled contourlet transform domain. Optik-International Journal for Light and Electron Optics, 123(7): 569–581

[7] Chan T F, Esedoglu S (2005). Aspects of total variation regularized l1function approximation. SIAM J Appl Math, 65(5): 1817–1837

[8] Chan T, Shen J, Vese L (2003). Variational PDE models in image processing. Not Am Math Soc, 50(1): 14–26

[9] Cvejic N, Seppanen T, Godsill S (2009). A no reference image fusion metric based on the regional importance measure. IEEE Journal Of Selected Topics In Signal Processing, 3(2): 212–221

[10] Ellmauthaler A, Pagliari C L, Da Silva E A (2013). Multi-scale image fusion using the undecimated wavelet transform with spectral factorization and nonorthogonal filter banks. IEEE Trans Image Process, 22(3): 1005–1017

[11] El-taweel G S, Helmy A K (2013). Image fusion scheme based on modified dual pulse coupled neural network. IET Image Processing, 7(5): 407–414

[12] Fang F, Li F, Zhang G, Shen C (2013). A variational method for multisource remote-sensing image fusion. Int J Remote Sens, 34(7): 2470–2486

[13] Gao G R, Lu P X, Feng D Z (2013). Multi-focus image fusion based on non-sub-sampled shear-let transform. IET Image Processing, 7(6): 633–639

[14] Hinterberger W, Scherzer O (2006). Variational methods on the space of functions of bounded hessian for convexification and de-noising. Computing, 76(1–2): 109–133

[15] James A, Dasarathy B (2014). Medical image fusion: a survey of the state of the art. Inf Fusion, 19: 4–19

[16] Kim Y, Lee C, Han D, Kim Y, Kim Y (2011). Improved additive-wavelet image fusion. IEEE Transactions on Geoscience and Remote Sensing Letters, 8(2): 263–267

[17] Kluckner S, Pock T, Bischof H (2010). Exploiting redundancy for aerial image fusion using convex optimization. Proceedings of the 32nd DAGM conference on Pattern Recognition, Darmstadt, Germany

[18] Kong W, Lei Y, Lei Y, Lu S (2011). Image fusion technique based on non-sub-sampled contour-let transform and adaptive unit-fast-linking pulse-coupled neural network. IET Image Processing, 5(2): 113– 121

[19] Kumar M, Dass S (2009). A total variation-based algorithm for pixel-level image fusion. IEEE Trans Image Process, 18(9): 2137–2143

[20] Li X, He M, Roux M (2010). Multi-focus image fusion based on redundant wavelet transform. IET Image Processing, 4(4): 283–293

[21] Liu G, Shen Y (2012). Ultrasonic image fusion using compressed sensing. Electron Lett, 48(19): 1182–1184

[22] Liu X, Zhou Y, Wang J (2014). Image fusion based on shearlet transform and regional features. AEU, Int J Electron Commun, 68(6): 471– 477

[23] Lustig M, Donoho D, Pauly J M (2007). Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn Reson Med, 58(6): 1182–1195

[24] Lysaker M, Lundervold A, Tai X C (2003). Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans Image Process, 12(12): 1579–1590

[25] Mahbubur Rahman S M, Omair Ahmad M, Swamy M N S (2010). Contrast-based fusion of noisy images using discrete wavelet transform. IET Image Processing, 4(5): 374–384

[26] Nikolova M (2004). A variational approach to remove outliers and impulse noise. J Math Imaging Vis, 20(1–2): 99–120

[27] Papafitsoros K, Schönlieb C B (2014). A combined first and second order variational approach for image reconstruction. J Math Imaging Vis, 48(2): 308–338

[28] Petrovic V (2004). Subjective image fusion evaluation data. Imaging Science Biomedical Engineering. University of Manchester:1–9

[29] Piella G (2009). Image fusion for enhanced visualization: a variational approach. Int J Comput Vis, 83(1): 1–11

[30] Pock T, Zebedin L, Bischof H (2011). TGV-fusion. In: Calude C S, Rozenberg A G, Salomaa A, eds. Maurer Festschrift. LNCS, 6570: 245–258

[31] Pouteau R, Stoll B, Chabrier S (2010). Multi-source SVM fusion for environmental monitoring in marquesas archipelago. In: Geoscience and Remote Sensing Symposium (IGARSS), 2010 IEEE International, 2719–2722

[32] Rudin L I, Osher S, Fatemi E (1992). Non-linear total variation based noise removal algorithms. Physica D, 60(1): 259–268

[33] Sapiro G (2001). Geometric Partial Differential Equations and Image Analysis. Cambridge University Press

[34] Schowengerdt R A (2006). Remote sensing: Models and Methods for Image Processing (3rd Edition). Academic Press

[35] Socolinsky D A (2000). A variational Approach to Image Fusion. Dissertation for PhD degree. The Johns Hopkins University, April 2000

[36] Socolinsky D A, Wolff L B (2002). Multispectral image visualization through first-order fusion. IEEE Trans Image Process, 11(8): 923–931

[37] Stathaki (2008). Image Fusion: Algorithms and Applications. Amsterdam: Elsevier

[38] Tafti P D, Stalder A F, Delgado-Gonzalo R, Unser M (2011). Variational enhancement and de-noising of flow field images. 8th IEEE International symposium on biomedical imaging: from Nano to Macro: 1061–1064

[39] Tikhonov A N (1943). On the stability of inverse problems. Dokl Akad Nauk SSSR, 39(5): 195–198

[40] Tran M P, Peteri R, Bergounioux M (2012). De-noising 3D medical images using a second order variational model and wavelet shrinkage. In: Image Analysis and Recognition. Berlin: Springer, 138–145

[41] Wang C, Ye Z F (2007). Perceptual contrast-based image fusion: a variational approach. Acta Automatica Sinica, 33(2): 132–137

[42] Wang L, Li B, Tian L (2014). Multi-modal medical volumetric data fusion using 3D discrete shearlet transform and global-to-local rule. IEEE Trans Biomed Eng, 61(1): 197–206

[43] Wang W W, Shui P L, Feng X C (2008). Variational models for fusion and de-noising of multi-focus images. IEEE Signal Process Lett, 15: 65–68

[44] Wang Z, Ziou D, Armenakis C, Li D, Li Q (2005). A comparative analysis of image fusion methods. IEEE Trans Geosci Rem Sens, 43(6): 1391–1402

[45] Yang B, Li S (2010). Multi-focus image fusion and restoration with sparse representation. IEEE Trans Instrum Meas, 59(4): 884–892

[46] Yang L, Guo B, Ni W (2008). Multimodality medical image fusion based on multi-scale geometric analysis of contourlet transform. Neurocomputing, 72(1–3): 203–211

[47] Yang Z Z, Yang Z (2013). Novel multi-focus image fusion and reconstruction framework based on compressed sensing. IET Image Processing, 7(9): 837–847

[48] Yuan J, Miles B, Shi J, Garvin G, Tai X C and A. Fenster A (2012). Efficient convex optimization approaches to variational image fusion. UCLA Tech. Report CAM-11-86

[49] Zhang X, Chan T F (2010). Wavelet inpainting by nonlocal total variation. Inverse Problems and Imaging, 4(1): 191–210

[50] Zheng S, Shi W, Liu J, Zhu G, Tian J (2007). Multisource image fusion method using support value transform. IEEE Trans Image Process, 16: 1

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg