Hybrid fusion and interpolation algorithm with near-infrared image

Abstract

Silicon-based digital cameras can record visible and near-infrared (NIR) information, in which the full color visible image (RGB) must be restored from color filter array (CFA) interpolation. In this paper, we propose a unified framework for CFA interpolation and visible/NIR image combination. To obtain a high quality color image, the traditional color interpolation from raw CFA data is improved at each pixel, which is constrained by the corresponding monochromatic NIR image in gradient difference. The experiments indicate the effectiveness of this hybrid scheme to acquire joint color and NIR information in real-time, and show that this hybrid process can generate a better color image when compared to treating interpolation and fusion separately.

Keywords

image fusion / color filter array (CFA) / interpolation / demosaicing

References

[1] Guarnera M, Messina G, Tomaselli V. Adaptive color demosaicing and false color removal. Journal of Electronic Imaging, 2010, 19(2): 1-16

[2] Menon D, Andriani S, Calvagno G. Demosaicing with directional filtering and a posteriori decision. IEEE Transactions on Image Processing, 2007, 16(1): 132-141

[3] Su C Y, Kao W C. Effective demosaicing using subband correlation. IEEE Transactions on Consumer Electronics, 2009, 55(1): 199-204

[4] Chung K H, Chan Y H. Low-complexity color demosaicing algorithm based on integrated gradients. Journal of Electronic Imaging, 2010, 19(2): 1-15

[5] Itoh Y. Similarity-based demosaicing algorithm using unified highfrequency map. IEEE Transactions on Consumer Electronics, 2011, 57(2): 597-605

[6] Hirakawa K, Parks T W. Adaptive homogeneity-directed demosaicing algorithm. IEEE Transactions on Image Processing, 2005, 14(3): 360-369

[7] Wu X, Zhang N. Primary-consistent soft-decision color demosaicking for digital cameras. IEEE Transactions on Image Processing, 2004, 13(9): 1263-1274

[8] Gunturk B K, Altunbasak Y, Mersereau R M. Color plane interpolation using alternating projections. IEEE Transactions on Image P<?Pub Caret?>rocess, 2002, 11(9): 997-1013

[9] Lian N, Chang L, Tan Y, Zagorodnov V. Adaptive filtering for color filter array demosaicking. IEEE Transactions on Image Processing, 2007,16(10): 2515-2525

[10] Lu Y M, Karzand M, Vetterli M. Demosaicking by alternating projections: theory and fast one-step implementation. IEEE Transactions on Image Processing, 2010, 19(8): 2085-2098

[11] Bennett E P, Mason J L, McMillan L. Multispectral bilateral video fusion. IEEE Transactions on Image Processing, 2007, 16(5): 1185-1194

[12] Zhang X, Sim T, Miao X. Enhancing photographs with near infrared images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2008, 1-8

[13] Schaul L, Fredembach C, Süsstrunk S. Color image dehazing using near-infrared. In: Proceedings of the 16th IEEE International Conference on Image Processing. 2009, 1629-1632

[14] Süsstrunk S, Fredembach C, Tamburrino D. Automatic skin enhancement with visible and near-infrared image fusion. In: Proceedings of the International Conference on Multimedia. 2010, 1693-1696

[15] Fredembach C, Süsstrunk S. Illuminant estimation and detection using near-infrared. In: Proceedings of the International Society for Optics and Photonics Digital Photography. 2009

[16] Salamati N, Fredembach C, Süsstrunk S. Material classification using color and NIR images. In: Proceedings of the 17th Color Imaging Conference Final Program and Proceedings. 2009, 216-227

[17] Fredembach C, Süsstrunk S. Colouring the near-infrared. In: Proceedings of the 16th Color Imaging Conference Final Program and Proceedings. 2008, 176-182

[18] Krishnan D, Fergus R. Dark flash photography. ACM Transactions on Graphics, 2009, 28(3), Artile No. 96

[19] Matsui S, Okabe T, Shimano M, Sato Y. Image enhancement of lowlight scenes with near-infrared flash images. Lecture Notes in Computer Science, 2010, 5994: 213-223

[20] Hirakawa K, Parks TW. Joint demosaicing and denoising. IEEE Transactions on Image Processing, 2006, 15(8): 2146-2157

[21] Donoho D L. Sparse components of images and optimal atomic decomposition. Constructive Approximation, 2001, 17(3): 353-382

[22] Donoho D L. Compressed sensing. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306

[23] Wang Y, Yang J, Yin W, Zhang Y. A new alternating minimization algorithm for total variation image reconstruction. SIAM Journal on Imaging Sciences, 2008, 1(3): 248-272

[24] Lu Y M, Fredembach C, Vetterli M, Süsstrunk S. Designing color filter arrays for the joint capture of visible and near-infrared images. In: Proceedings of the 16th IEEE International Conference on Image Processing. 2009, 3797-3800

[25] Smith P R. Bilinear interpolation of digital images. Ultramicroscopy, 1981, 6(2): 201-204

[26] Hamilton J F, Adams J E. Adaptive color plane interpolation in single sensor color electronic camera. US Patent, 5 629 734, 1997-<month>07</month>-<day>29</day>

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg