Defocus blur detection using novel local directional mean patterns (LDMP) and segmentation via KNN matting

Abstract

Detection and segmentation of defocus blur is a challenging task in digital imaging applications as the blurry images comprise of blur and sharp regions that wrap significant information and require effective methods for information extraction. Existing defocus blur detection and segmentation methods have several limitations i.e., discriminating sharp smooth and blurred smooth regions, low recognition rate in noisy images, and high computational cost without having any prior knowledge of images i.e., blur degree and camera configuration. Hence, there exists a dire need to develop an effective method for defocus blur detection, and segmentation robust to the above-mentioned limitations. This paper presents a novel features descriptor local directional mean patterns (LDMP) for defocus blur detection and employ KNN matting over the detected LDMP-Trimap for the robust segmentation of sharp and blur regions. We argue/hypothesize that most of the image fields located in blurry regions have significantly less specific local patterns than those in the sharp regions, therefore, proposed LDMP features descriptor should reliably detect the defocus blurred regions. The fusion of LDMP features with KNN matting provides superior performance in terms of obtaining high-quality segmented regions in the image. Additionally, the proposed LDMP features descriptor is robust to noise and successfully detects defocus blur in high-dense noisy images. Experimental results on Shi and Zhao datasets demonstrate the effectiveness of the proposed method in terms of defocus blur detection. Evaluation and comparative analysis signify that our method achieves superior segmentation performance and low computational cost of 15 seconds.

Graphical abstract

Keywords

defocus blur detection / local directional mean patterns / image matting / sharpness metrics / blur segmentation

References

[1] Krishnamurthy B, Sarkar M. Deep-learning network architecture for object detection. U.S. Patents 10, 019, 655, 2018

[2] Price B L, Schiller S, Cohen S, Xu N. Image matting using deep learning. Ed: Google Patents, 2019

[3] Liu C , Liu W , Xing W . A weighted edge-based level set method based on multi-local statistical information for noisy image segmentation. Journal of Visual Communication and Image Representation, 2019, 59 : 89– 107

[4] Gast J, Roth S. Deep video deblurring: the devil is in the details. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2019

[5] Gvozden G , Grgic S , Grgic M . Blind image sharpness assessment based on local contrast map statistics. Journal of Visual Communication and Image Representation, 2018, 50 : 145– 158

[6] Shi J, Xu L, Jia J. Discriminative blur detection features. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 2014, 2965–2972

[7] Vu C T , Phan T D , Chandler D M . S ₃: a spectral and spatial measure of local perceived sharpness in natural images. IEEE Transactions on Image Processing, 2011, 21( 3): 934– 945

[8] Su B, Lu S, Tan C L. Blurred image region detection and classification. In: Proceedings of the 19th ACM International Conference on Multimedia, Scottsdale, Arizona. 2011

[9] Zhuo S , Sim T . Defocus map estimation from a single image. Pattern Recognition, 2011, 44( 9): 1852– 1858

[10] Zhu X , Cohen S , Schiller S , Milanfar P . Estimating spatially varying defocus blur from a single image. IEEE Transactions on Image Processing, 2013, 22( 12): 4879– 4891

[11] Tang C , Hou C , Song Z . Defocus map estimation from a single image via spectrum contrast. Optics letters, 2013, 38( 10): 1706– 1708

[12] Zhang X , Wang R , Jiang X , Wang W , Gao W . Spatially variant defocus blur map estimation and deblurring from a single image. Journal of Visual Communication and Image Representation, 2016, 35 : 257– 264

[13] Wing T Y, Brown M S. Single image defocus map estimation using local contrast prior. In: Proceedings of the 16th IEEE International Conference on Image Processing. 2009, 1797–1800

[14] Shan Q , Jia J , Agarwala A . High-quality motion deblurring from a single image. ACM Transactions on Graphics (Tog), 2008, 27( 3): 1– 10

[15] Rajabzadeh T, Vahedian A, Pourreza H. Static object depth estimation using defocus blur levels features. In: Proceedings of the 6th International Conference on Wireless Communications Networking and Mobile Computing. 2010, 1–4

[16] Mavridaki E, Mezaris V. No-reference blur assessment in natural images using Fourier transform and spatial pyramids. In: Proceedings of IEEE International Conference on Image Processing (ICIP). 2014, 566–570

[17] Lin J , Ji X , Xu W , Dai Q . Absolute depth estimation from a single defocused image. IEEE Transactions on Image Processing, 2013, 21( 11): 4545– 4550

[18] Zhou C , Lin S , Nayar S K . Coded aperture pairs for depth from defocus and defocus deblurring. International Journal of Computer Vision, 2011, 93( 1): 53– 72

[19] Liu R, Li Z, Jia J. Image partial blur detection and classification. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition. 2008, 1–8

[20] Tang C , Wu J , Hou Y , Wang P , Li W . A spectral and spatial approach of coarse-to-fine blurred image region detection. IEEE Signal Processing Letters, 2016, 23( 11): 1652– 1656

[21] Yi X , Eramian M . LBP-Based Segmentation of Defocus Blur. IEEE Transactions on Image Processing, 2016, 25( 4): 1626– 1638

[22] Hassen R , Wang Z , Salama M M . Image sharpness assessment based on local phase coherence. IEEE Transactions on Image Processing, 2013, 22( 7): 2798– 2810

[23] Xiao H , Lu W , Li R , Zhong N , Yeung Y , Chen J . Defocus blur detection based on multiscale SVD fusion in gradient domain. Journal of Visual Communication and Image Representation, 2019, 59 : 52– 61

[24] Chakrabarti A, Zickler T, Freeman W T. Analyzing spatially-varying blur. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2010

[25] Golestaneh S A, Karam L J. Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5800–5809

[26] Zhao W, Zheng B, Lin Q, Lu H. Enhancing diversity of defocus blur detectors via cross-ensemble network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019, 8905–8913

[27] Zhang Y, Hirakawa K. Blur processing using double discrete wavelet transform. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 1091–1098

[28] Shi J, Xu L, Jia J. Just noticeable defocus blur detection and estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 657–665

[29] Pang Y , Zhu H , Li X , Li X . Classifying discriminative features for blur detection. IEEE Transactions on Cybernetics, 2015, 46( 10): 2220– 2227

[30] Kim B, Son H, Park S J, Cho S, Lee S. Defocus and Motion Blur Detection with Deep Contextual Features. In: Proceedings of Computer Graphics Forum. 2018, 277−288

[31] Park J, Tai Y W, Cho D, Kweon I S. A unified approach of multi-scale deep and hand-crafted features for defocus estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 1736–1745

[32] Tang C, Zhu X, Liu X, Wang L, Zomaya A. DeFusionNET: defocus blur detection via recurrently fusing and refining multi-scale deep features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019, 2700–2709

[33] Nigam S, Singh R, Misra A. Local binary patterns based facial expression recognition for efficient smart applications. In: Hassanien A, Elhoseny M, Ahmed S, Singh A, eds. Security in Smart Cities: Models, Applications and Challenges. Springer, Cham, 2019, 297−322

[34] Kumar G S , Mohan P K . Local mean differential excitation pattern for content based image retrieval. SN Applied Sciences, 2019, 1( 1): 1– 10

[35] Zhao W, Zhao F, Wang D, Lu H. Defocus blur detection via multi-stream bottom-top-bottom fully convolutional network. In: Proceedings of the IEEE Conference on Computer vision and Pattern Recognition. 2018, 3080–3088

Acknowledgements

This work was supported and funded by the Directorate ASR&TD of UET-Taxila.

RIGHTS & PERMISSIONS

2022 Higher Education Press