A pan-sharpening method based on the ADMM algorithm

Abstract

Pan-sharpening is a method of integrating low-resolution multispectral images with corresponding high-resolution panchromatic images to obtain multispectral images with high spectral and spatial resolution. A novel variational model for pan-sharpening is proposed in this paper. The model is mainly based on three hypotheses: 1) the pan-sharpened image can be linearly represented by the corresponding panchromatic image; 2) the low-resolution multispectral image is down-sampled from the high-resolution multispectral image through the down-sampling operator; and 3) the satellite image has the low-rank property. Three energy components corresponding to these assumptions are integrated into a variational framework to obtain a total energy function. We adopt the alternating direction method of multipliers (ADMM) to optimize the total energy function. The experimental results show that the proposed method performs better than other mainstream methods in spectral and spatial information preserving aspect.

Keywords

pan-sharpening / multispectral image / panchromatic image / variational framework / energy function / ADMM

References

[1] Alparone L, Baronti S, Garzelli A, Nencini F (2004). A global quality measurement of pan-sharpened multispectral imagery. IEEE Geosci Remote Sens Lett, 1(4): 313–317

[2] Alparone L, Wald L, Chanussot J, Thomas C, Gamba P, Bruce L M (2007). Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data-fusion contest. IEEE Trans Geosci Remote Sens, 45(10): 3012–3021

[3] Aiazzi B, Alparone L, Baronti S, Garzelli A (2002). Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Trans Geosci Remote Sens, 40(10): 2300–2312

[4] Aly H, Sharma J (2014). A regularized model-based optimization framework for pan-sharpening. IEEE Trans Image Process, 23(6): 2596–2608

[5] Ballester C, Caselles V, Igual L, Verdera J, Roug B (2006). A variational model for P+XS image fusion. Int J Comput Vis, 69(1): 43–58

[6] Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2010). Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn, 3(1): 1–122

[7] Bredies K, Lorenz D A (2008). Soft-thresholding. J Fourier Anal Appl, 14(5–6): 813–837

[8] Cai J, Candes E, Shen Z (2010). A singular value thresholding algorithm for matrix completion. SIAM J Optim, 20(4): 1956–1982

[9] Candes E, Li X, Ma Y, Wright J (2011). Robust principal component analysis. J Assoc Comput Mach, 58(3): 1–37

[10] Chen C, Li Y, Liu W, Huang J (2015). SIRF: simultaneous satellite image registration and fusion in a unified framework. IEEE Transactions on Image Processing, A Publication of the IEEE Signal Processing Society, 24(11): 4213

[11] Chen C, Li Y, Liu W, Huang J (2014). Image fusion with local spectral consistency and dynamic gradient sparsity. IEEE Conference on Computer Vision & Pattern Recognition, 2760–2765

[12] Choi M (2006). A new intensity-hue-saturation fusion approach to image fusion with a tradeoff parameter. IEEE Trans Geosci Remote Sens, 44(6): 1672–1682

[13] Ding X, Jiang Y, Huang Y, Paisley J (2014). Pan-sharpening with a Bayesian nonparametric dictionary learning model. In: Proceeding of 17th International Conference of Artificial Intelligence and Statistics, 176–184

[14] Dong W, Shi G, Li X (2013). Nonlocal image restoration with bilateral variance estimation: a low-rank approach. IEEE Trans Image Process, 22(2): 700–711

[15] Fang F, Li F, Shen C, Zhang G (2013). A variational approach for pan-sharpening. IEEE Trans Image Process, 22(7): 2822–2834

[16] Goldstein T, O’Donoghue B, Setzer S, Baraniuk R (2014). Fast alternating direction optimization methods. SIAM J Imaging Sci, 7(3): 225–231

[17] Laben C, Brower B (2000). Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening. Websterny Uspenfieldny. USA: Eastman Kodak Company, 275–299

[18] Lu X, Zhang J (2014). Panchromatic and multispectral images fusion based on modified GS-SWT. Geoscience & Remote Sensing Symposium (IGARSS), 2530–2533

[19] Masi G, Cozzolino D, Verdoliva L, Scarpa G (2016). Pansharpening by convolutional neural networks. Remote Sens, 8(7): 594

[20] Metwalli M, Nasr A, Faragallah O, Rabaie E E, Abbas A (2014). Efficient pan-sharpening of satellite images with the contourlet transform. Int J Remote Sens, 35(5): 1979–2002

[21] Mezouar M E, Taleb N, Kpalma K, Ronsin J (2011). An IHS-based fusion for color distortion reduction and vegetation enhancement in IKONOS imagery. IEEE Trans Geosci Remote Sens, 49(5): 1590–1602

[22] Moller M, Wittman T, Bertozzi A, Burger M (2013). A variational approach for sharpening high dimensional images. SIAM J Imaging Sci, 5(1): 150–178

[23] Moller M, Wittman T, Bertozzi A (2008). Variational wavelet pan-sharpening. IEEE Trans Geosci Remote Sens, 1–9

[24] Park J, Kim K, Yang K (2001). Image fusion using multiresolution analysis. IEEE Int Geosci Remote Se, 2(2): 864–866

[25] Pushparaj J, Hegde A (2016). Evaluation of pan-sharpening methods for spatial and spectral quality. Appl Geomatics, 9(1): 1–12

[26] Rahmani S, Strait M, Merkurjev D, Moeller M, Wittman T (2010). An adaptive HIS pan-sharpening method. IEEE Geosci Remote Sens Lett, 7(4): 746–750

[27] Shah V, Younan N, King R (2008). An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Trans Geosci Remote Sens, 46(5): 1323–1335

[28] Shah V, Younan N, King R (2007). An adaptive PCA-based approach to pan-sharpening. Remote Sens, 6748: 1–9

[29] Toet A, Hogervorst M (2003). Performance comparison of different gray level image fusion schemes through a universal image quality index. In: Kadar I (Ed.), Signal Processing, Sensor Fusion, and Target Recognition XII, SPIE-5096, 552–561

[30] Vivone G, Alparone L, Chanussot J, Mura M, Garzelli A (2015). A critical comparison among pansharpening algorithms. IEEE Trans Geosci Remote Sens, 53(5): 2265–2586

[31] Wang Q, Yu D, Shen Y (2009). An overview of image fusion metrics. IEEE Instrumentation & Measurement Technology Conference, Singapore, 918–923

[32] Wang S, Zhang Z (2012). Colorization by Matrix Completion. In: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence (AAAI'12), AAAI Press, 1169–1175

[33] Wang Z, Bovik A (2002). A universal image quality index. IEEE Signal Process Lett, 9(3): 81–84

[34] Wei Y, Yuan Q, Shen H, Zhang L (2017). Boosting the accuracy of multispectral image pansharpening by learning a deep residual network. IEEE Geosci Remote Sens Lett, 14(10): 1795–1799

[35] Yang J, Fu X, Hu Y, Huang Y, Ding X, Paisley J (2017). PanNet: A deep network architecture for pan-sharpening. Proceedings of IEEE International Conference on Computer Vision.,1753–1761

[36] Yocky D (1995). Image merging and data fusion by means of the discrete two-dimensional wavelet transform. J Opt Soc Am A Opt Image Sci Vis, 12(9): 1834–1841

[37] Yuan Q, Wei Y, Meng X, Shen H, Zhang L (2018). A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(3): 978–989

[38] Zhang B (2010). Study on image fusion based on different fusion rules of wavelet transform. International Conference on Advanced Computer Theory & Engineering, 3(7): 649–653

[39] Zhang H, Roy D (2016). Computationally inexpensive Landsat 8 operational land imager (OLI) pansharpening. Remote Sens, 8(3): 180

[40] Zhou J, Civco D, Silander J (1998). A wavelet transform method to merge Landsat TM and SPOT panchromatic data. Int J Remote Sens, 19(4): 743–757

Acknowledgments

We sincerely thank the editor and reviewers for their insightful comments and constructive suggestions on the article, and thank Dr. Thomas James Godfrey for helping us to revise the grammar. This work was supported in part by “Chenguang Program” supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission (Grant No. 17CG25), in part by the Key Project of the National Natural Science Foundation of China (Grant No. 61731009), and in part by the National Natural Science Foundation of China (Grant No. 61871185).

RIGHTS & PERMISSIONS

2019 Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature