Colored 3D surface reconstruction using Kinect sensor

Abstract

A colored 3D surface reconstruction method which effectively fuses the information of both depth and color image using Microsoft Kinect is proposed and demonstrated by experiment. Kinect depth images are processed with the improved joint-bilateral filter based on region segmentation which efficiently combines the depth and color data to improve its quality. The registered depth data are integrated to achieve a surface reconstruction through the colored truncated signed distance fields presented in this paper. Finally, the improved ray casting for rendering full colored surface is implemented to estimate color texture of the reconstruction object. Capturing the depth and color images of a toy car, the improved joint-bilateral filter based on region segmentation is used to improve the quality of depth images and the peak signal-to-noise ratio (PSNR) is approximately 4.57 dB, which is better than 1.16 dB of the joint-bilateral filter. The colored construction results of toy car demonstrate the suitability and ability of the proposed method.

Keywords

Augmented Reality / Depth Image / Iterative Close Point / Kinect Sensor / Region Segmentation

References

[1] CuiY, SchuonS, ThrunS, StrickerD, TheobaltC. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35: 1039

[2] KhilarR, ChitrakalaS, SelvamParvathyS3D Image Reconstruction: Techniques, Applications and ChallengesIEEE International Conference on Optical Imaging Sensor and Security, 2013, 1

[3] FreedmanB, ShpuntA, MachlineM, ArieliY. Depth Mapping Using Projected Patterns, 2012,

[4] ChengZ, HairongY, HongC, SuiW. Journal of Optoelectronics·Laser, 2013, 24: 805

[5] LaiK, BoL, RenX, FoxDSparse Distance Learning for Object Recognition Combining RGB and Depth InformationIEEE International Conference on Robotics and Automation (ICRA), 2011, 4007

[6] KhoshelhamK, ElberinkS O. Sensors, 2012, 12: 1437

[7] HerbstE, HenryP, RenX, FoxDToward Object Discovery and Modeling via 3-D Scene ComparisonIEEE International Conference on Robotics and Automation (ICRA), 2011, 2623

[8] MennaF, RemondinoF, BattistiR, NocerinoE. Proceedings of SPIE, 2011, 8085: 80850G

[9] NewcombeR A, IzadiS, HilligesO, MolyneauxDKinectFusion: Real-time Dense Surface Mapping and Tracking10th IEEE International Symposium on Mixed and Augmented Reality, 2011, 127

[10] IzadiS, KimD, HilligesO, MolyneauxD, NewcombeR, KohliP, FitzgibbonAKinectFusion: Real-Time 3D Reconstruction and Interaction Using a Moving Depth CameraProceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, 2011, 559

[11] MassimoC, SalgadoL. Proceedings of SPIE, 2012, 8920: 82900E

[12] MatyuninS, VatolinD, BerdnikovY, SmirnovMTemporal Filtering for Depth Maps Generated by Kinect Depth Camera3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, 2011, 1

[13] RusinkiewiczS, Hall-HoltO, LevoyM. ACM Transactions on Graphics (TOG), 2002, 21: 438

[14] BrianC, LevoyMA Volumetric Method for Building Complex Models from Range ImagesProceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, 1996, 303

[15] Lorensen WilliamE, ClineH E. ACM Siggraph Computer Graphics, 1987, 21: 163

Just Accepted

This article has successfully passed peer review and final editorial review, and will soon enter typesetting, proofreading and other publishing processes. The currently displayed version is the accepted final manuscript. The officially published version will be updated with format, DOI and citation information upon launch. We recommend that you pay attention to subsequent journal notifications and preferentially cite the officially published version. Thank you for your support and cooperation.