Transient imaging with a time-of-flight camera and its applications

Abstract

Transient imaging is a technique in photography that recordsthe process of light propagation before it reaches a stationary statesuch that events at the light speed level can be observed. In thisreview we introduce three main models for transient imaging with atime-of-flight (ToF) camera: correlation model, frequency-domain model,and compressive sensing model. Transient imaging applications usuallyinvolve resolving the problem of light transport and separating thelight rays arriving along different paths. We discuss two of the applications:imaging objects inside scattering media and recovering both the shapeand texture of an object around a corner.

Keywords

Transient imaging / Time-of-flight(ToF) camera / Scattering media / Around corners

References

[1] Bhandari , A., Raskar , R., 2016. Signal processing for time-offlight imaging sensors:an introduction to inverse problems in computational 3-D imaging. IEEE Signal Process. Mag., 33(5):45–58. https://doi.org/10.1109/MSP.2016.2582218

[2] Bhandari , A., Feigin , M., Izadi , S., , 2014. Resolvingmultipath interference in Kinect: an inverse problem approach. Proc. IEEE Sensors, p.614–617. https://doi.org/10.1109/ICSENS.2014.6985073

[3] Heide , F., Hullin , M.B., Gregson , J., , 2013. Low-budget transient imaging using photonic mixer devices. ACM Trans. Graph., 32(4), Article 45. https://doi.org/10.1145/2461912.2461945

[4] Heide , F., Xiao , L., Heidrich , W., , 2014. Diffuse mirrors: 3D reconstruction from diffuse indirect illuminationusing inexpensive time-of-flight sensors. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, p.3222–3229. https://doi.org/10.1109/CVPR.2014.418

[5] Kadambi , A., Whyte , R., Bhandari , A., , 2013. Coded time of flight cameras: sparse deconvolution to address multipathinterference and recover time profiles. ACM Trans. Graph., 32(6), Article 167. https://doi.org/10.1145/2508363.2508428

[6] Kirmani , A., Hutchison , T., Davis , J., , 2009. Looking around the corner using transient imaging. Proc. IEEE 12th Int. Conf. on Computer Vision,p.159–166. https://doi.org/10.1109/ICCV.2009.5459160

[7] Lin , J.Y., Liu , Y.B., Hullin , M.B., , 2014. Fourier analysis on transient imaging with a multifrequency time-of-flightcamera. Proc. IEEE Conf. on Computer Visionand Pattern Recognition, p.3230–3237. https://doi.org/10.1109/CVPR.2014.419

[8] Lin , J.Y., Liu , Y.B., Suo , J.L., , 2017. Frequency-domain transient imaging. IEEE Trans. Patt. Anal. Mach. Intell., 39(5):937–950. https://doi.org/10.1109/TPAMI.2016.2560814

[9] Nishita , T., Miyawaki , Y., Nakamae , E., 1987. A shadingmodel for atmospheric scattering considering luminous intensity distributionof light sources. ACM SIGGRAPH Comput. Graph., 21(4):303–310. https://doi.org/10.1145/37402.37437

[10] Peters , C., Klein , J., Hullin , M.B., , 2015. Solving trigonometric moment problems for fast transient imaging. ACM Trans. Graph., 34(6), Article 220. https://doi.org/10.1145/2816795.2818103

[11] Qiao , H., Lin , J.Y., Liu , Y.B., , 2015. Resolving transient time profile in ToF imaging via log-sum sparseregularization. Opt. Lett., 40(6):918–921. https://doi.org/10.1364/OL.40.000918

[12] Shim , H., Lee , S., 2016. Recovering translucent objects using a single time-of-flightdepth camera. IEEETrans. Circ. Syst. Video Technol., 26(5):841–854. https://doi.org/10.1109/TCSVT.2015.2397231

[13] Tropp , J.A., Gilbert , A.C., 2007. Signal recovery from random measurements viaorthogonal matching pursuit. IEEE Trans. Inform. Theory, 53(12):4655–4666. https://doi.org/10.1109/TIT.2007.909108

[14] Velten , A., Wu , D., Jarabo , A., , 2013. Femtophotography:capturing and visualizing the propagation of light. ACM Trans. Graph., 32(4), Article 44. https://doi.org/10.1145/2461912.2461928

RIGHTS & PERMISSIONS

2017 Zhejiang University and Springer-Verlag GmbHGermany