High-resolution spectral video acquisition

Abstract

Compared with conventional cameras, spectral imagers providemany more features in the spectral domain. They have been used invarious fields such as material identification, remote sensing, precisionagriculture, and surveillance. Traditional imaging spectrometers usegenerally scanning systems. They cannot meet the demands of dynamicscenarios. This limits the practical applications for spectral imaging.Recently, with the rapid development in computational photographytheory and semiconductor techniques, spectral video acquisition hasbecome feasible. This paper aims to offer a review of the state-of-the-artspectral imaging technologies, especially those capable of capturingspectral videos. Finally, we evaluate the performances of the existingspectral acquisition systems and discuss the trends for future work.

Keywords

Multispectral/hyperspectral video acquisition / Snapshot / Under-sampling and reconstruction

References

[1] Abed, F.M., Amirshahi, S.H., Abed, M.R.M., 2009. Reconstructionof reflectance data using an interpolation technique. J. Opt. Soc. Am. A, 26(3):613–624. https://doi.org/10.1364/JOSAA.26.000613

[2] Adelson, E.H., Bergen, J.R., 1991. The plenoptic function and the elements of early vision. In: Landy, M.S.,Movshon, J.A. (Eds.), Computational Models of Visual Processing. MITPress, Cambridge, p.3–20.

[3] Arce, G.R., Brady, D.J., Carin, L., , 2014. Compressivecoded aperture spectral imaging: an introduction. IEEE Signal Process. Mag., 31(1):105–115. https://doi.org/10.1109/MSP.2013.2278763

[4] Bao, J., Bawendi, M.G., 2015. A colloidal quantum dot spectrometer. Nature, 523(7558):67–70. https://doi.org/10.1038/nature14576

[5] Bioucas-Dias, J.M., Figueiredo, M.A., 2007. A new TwIST: two-step iterative shrinkage/thresholdingalgorithms for image restoration. IEEE Trans. Imag. Process., 16(12):2992–3004. https://doi.org/10.1109/TIP.2007.909319

[6] Bodkin, A., Sheinis, A., Norton, A., , 2009. Snapshot hyperspectral imaging: the hyperpixel array camera. SPIE, 7334:73340H. https://doi.org/10.1117/12.818929

[7] Boyd, S., Parikh, N., Chu, E., , 2011. Distributed optimization and statistical learning via the alternatingdirection method of multipliers. Found. Trends Mach. Learn., 3(1):1–122. https://doi.org/10.1561/2200000016

[8] Candès, E.J., Wakin, M.B., 2008. An introduction to compressive sampling. IEEE Signal Process. Mag., 25(2): 21–30. https://doi.org/10.1109/MSP.2007.914731

[9] Candès, E.J., Romberg, J., Tao, T., 2006. Robustuncertainty principles: exact signal reconstruction from highly incompletefrequency information. IEEE Trans. Inform. Theory, 52(2):489–509. https://doi.org/10.1109/TIT.2005.862083

[10] Cao, X., Du, H., Tong, X., , 2011a. A prism-masksystem for multispectral video acquisition. IEEE Trans. Patt. Anal. Mach. Intell., 33(12):2423–2435. https://doi.org/10.1109/TPAMI.2011.80

[11] Cao, X., Tong, X., Dai, Q., , 2011b. High resolutionmultispectral video capture with a hybrid camera system. IEEE Conf. on ComputerVision and Pattern Recognition, p.297–304. https://doi.org/10.1109/CVPR.2011.5995418

[12] Cao, X., Yue, T., Lin, X., , 2016. Computationalsnapshot multispectral cameras. IEEE Signal Process. Mag., 33(5):95–108. https://doi.org/10.1109/MSP.2016.2582378

[13] Chakrabarti, A., Zickler, T., 2011. Statistics of real-world hyperspectral images. IEEE Conf. on ComputerVision and Pattern Recognition, p.193–200. https://doi.org/10.1109/CVPR.2011.5995660

[14] Descour, M., Dereniak, E., 1995. Computed-tomography imaging spectrometer: experimentalcalibration and reconstruction results. Appl. Opt., 34(22):4817–4826. https://doi.org/10.1364/AO.34.004817

[15] Descour, M., Volin, C.E., Ford, B.K., , 2001. Snapshothyperspectral imaging. In: Integrated Computational Imaging Systems. OSA Publishing,Washington, D.C., paper IWB4.

[16] Donoho, D.L., 2006. Compressed sensing. IEEE Trans. Inform. Theory, 52(4):1289–1306. https://doi.org/10.1109/TIT.2006.871582

[17] Du, H., Tong, X., Cao, X., , 2009. A prism-basedsystem for multispectral video acquisition. IEEE 12th Int. Conf. on Computer Vision, p.175–182. https://doi.org/10.1109/ICCV.2009.5459162

[18] Gao, L., Kester, R.T., Hagen, N., , 2010. Snapshot image mapping spectrometer (IMS) with high sampling densityfor hyperspectral microscopy. Opt. Expr., 18(14):14330–14344. https://doi.org/10.1364OE.18.014330

[19] Gat, N., 2000. Imaging spectroscopy using tunablefilters: a review. SPIE, 4056:50–64. https://doi.org/10.1117/12.381686

[20] Golbabaee, M., Vandergheynst, P., 2012. Compressed sensing of simultaneous low-rank and joint-sparsematrices. arXiv:1211.5058. http://arxiv.org/abs/1211.5058

[21] Green, R.O., Eastwood, M.L., Sarture, C.M., , 1998. Imaging spectroscopy and the airborne visible/infrared imaging spectrometer(AVIRIS). RemoteSens. Environ., 65(3):227–248. https://doi.org/10.1016/S0034-4257(98)00064-9

[22] Harvey, A.R., Beale, J.E., Greenaway, A.H., , 2000. Technology options for imaging spectrometry. Int. Symp. on Optical Science and Technology, p.13–24. https://doi.org/10.1117/12.406592

[23] Herrala, E., Okkonen, J.T., Hyvarinen, T.S., , 1994. Imaging spectrometer for process industry applications. SPIE, 2248:33–40. https://doi.org/10.1117/12.194344

[24] Hunicz, J., Piernikarski, D., 2001. Investigation of combustion in a gasoline engine usingspectrophotometric methods. SPIE, 4516:307–314. https://doi.org/10.1117/12.435940

[25] Kindzelskii, A.L., Yang, Z.Y., Nabel, G.J., , 2000. Ebola virus secretory glycoprotein (sGP) diminishes FcγRIIIB-to-CR3proximity on neutrophils. J. Immun., 164(2):953–958. https://doi.org/10.4049/jimmunol.164.2.953

[26] Kittle, D., Choi, K., Wagadarikar, A., , 2010. Multiframe image estimation for coded aperture snapshot spectralimagers. Appl.Opt., 49(36):6824–6833.

[27] Lawlor, J., Fletcher-Holmes, D., Harvey, A., , 2002. In vivo hyperspectral imaging of human retina and optic disc. Invest. Ophthalmol. Vis.Sci., 43(13):4350–4350.https://doi.org/10.1364/AO.49.006824

[28] Liao, X., Li, H., Carin, L., 2014. Generalized alternatingprojection for weighted-2,1 minimization with applications to model-basedcompressive sensing. SIAM J. Imag. Sci., 7(2):797–823. https://doi.org/10.1137/130936658

[29] Lin, X., Liu, Y., Wu, J., , 2014a. Spatial-spectralencoded compressive hyperspectral imaging. ACM Trans. Graph., 33(6), Article 233. https://doi.org/10.1145/2661229.2661262

[30] Lin, X., Wetzstein, G., Liu, Y., , 2014b. Dualcoded compressive hyperspectral imaging. Opt. Lett., 39(7):2044–2047. https://doi.org/10.1364/OL.39.002044

[31] Ma, C., Cao, X., Wu, R., , 2014. Content-adaptivehigh-resolution hyperspectral video acquisition with a hybrid camerasystem. Opt. Lett., 39(4):937–940. https://doi.org/10.1364/OL.39.000937

[32] Mansfield, C.L., 2005. Seeing into the Past. http://www. nasa.gov/vision/earth/technologies/scrolls.html

[33] Mitchell, P.A., 1995. Hyperspectral digital imagery collectionexperiment (HYDICE). SPIE, 2587:70–95. https://doi.org/10.1117/12.226807

[34] Mooney, J.M., Vickers, V.E., An, M., , 1997. Highthroughput hyperspectral infrared camera. J. Opt. Soc. Am. A, 14(11):2951–2961. https://doi.org/10.1364/JOSAA.14.002951

[35] Morovic, P., Finlayson, G.D., 2006. Metamer-set-based approach to estimating surface reflectancefrom camera RGB. J. Opt. Soc. Am. A, 23(8):1814–1822. https://doi.org/10.1364/JOSAA.23.001814

[36] Morris, H.R., Hoyt, C.C., Treado, P.J., 1994. Imagingspectrometers for fluorescence and Raman microscopy: acousto-opticand liquid crystal tunable filters. Appl. Spectr., 48(7):857–866.

[37] Nguyen, R.M., Prasad, D.K., Brown, M.S., 2014. Trainingbasedspectral reconstruction from a single RGB image. European Conf. on Computer Vision, p.186–201. https://doi.org/10.1007/978-3-319-10584-0_13

[38] Oh, W.S., Brown, M.S., Pollefeys, M., , 2016. Do it yourself hyperspectral imaging with everyday digital cameras. IEEE Conf. on ComputerVision and Pattern Recognition, p.2461–2469. https://doi.org/10.1109/CVPR.2016.270

[39] Radon, J., 1917. Über die Bestimmung von Funktionendurch ihre Integralwerte längs gewisser Mannigfaltigkeiten. Akad. Wiss., 69:262–277 (in German).

[40] Rørslett, B., 2004. All you ever wantedto know about digital UV and IR photography, but could not affordto ask. http://www.naturfotograf.com/UV_IR_rev00.html

[41] Schechner, Y.Y., Nayar, S.K., 2002. Generalized mosaicing: wide field of view multispectralimaging. IEEE Trans.Patt. Anal. Mach. Intell., 24(10):1334–1348. https://doi.org/10.1109/TPAMI.2002.1039205

[42] Shepp, L.A., Vardi, Y., 1982. Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imag., 1(2):113–122. https://doi.org/10.1109/TMI.1982.4307558

[43] Su, L., Zhou, Z., Yuan, Y., , 2015. A snapshotlight field imaging spectrometer. Opt.-Int. J. Light Electr. Opt., 126(9):877–881. https://doi.org/10.1016/j.ijleo.2015.01.034

[44] Wagadarikar, A.A., Pitsianis, N.P., Sun, X., , 2009. Video rate spectral imaging using a coded aperture snapshot spectralimager. Opt. Expr., 17(8):6368–6388. https://doi.org/10.1364/OE.17.006368

[45] Willett, R.M., Duarte, M.F., Davenport, M.A., , 2014. Sparsity and structure in hyperspectral imaging: sensing, reconstruction,and target detection. IEEE Signal Process. Mag., 31(1):116–126. https://doi.org/10.1109/MSP.2013.2279507

[46] Wu, Y., Mirza, I.O., Arce, G.R., , 2011. Developmentof a digital-micromirror-device-based multishot snapshot spectralimaging system. Opt. Lett., 36(14):2692–2694. https://doi.org/10.1364/OL.36.002692

[47] Yamaguchi, M., Haneishi, H., Fukuda, H., , 2006. Highfidelity video and still-image communication based on spectralinformation: natural vision system and its applications. SPIE, 6062:60620G. https://doi.org/10.1117/12.649454

[48] Yasuma, F., Mitsunaga, T., Iso, D., , 2010. Generalized assorted pixel camera: postcapture control of resolution,dynamic range, and spectrum. IEEE Trans. Imag. Process., 19(9):2241–2253. https://doi.org/10.1109/TIP.2010.2046811

[49] Zhou, Z., Yuan, Y., Bin, X.L., 2010. Light field imagingspectrometer: conceptual design and simulated performance. Frontiers in Optics/LaserScience XXVI, paper FThM3. https://doi.org/10.1364/FIO.2010.FThM3

RIGHTS & PERMISSIONS

2017 Zhejiang University and Springer-Verlag GmbHGermany