A primary-secondary background model with sliding window PCA algorithm

Abstract

Rain and snow seriously degrade outdoor video quality. In this work, a primary-secondary background model for removal of rain and snow is built. First, we analyze video noise and use a sliding window sequence principal component analysis de-nosing algorithm to reduce white noise in the video. Next, we apply the Gaussian mixture model (GMM) to model the video and segment all foreground objects primarily. After that, we calculate von Mises distribution of the velocity vectors and ratio of the overlapped region with referring to the result of the primary segmentation and extract the interesting object. Finally, rain and snow streaks are inpainted using the background to improve the quality of the video data. Experiments show that the proposed method can effectively suppress noise and extract interesting targets.

Key words： sliding window sequence principal component analysis; primary-secondary background model; removal of rain and snow; Gaussian mixture model (GMM)

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant No. 60702032), the Natural Science Foundation of Heilongjiang Province (No. F201021), the Natural Scientific Research Innovation Foundation in Harbin Institute of Technology (No. HIT.NSRIF.2008.63).

References

1 Drew M S, Wei J, Li Z N. Illumination — invariant image retrieval and video segmentation. Pattern Recognition, 1999, 32(8): 1369–1388

2 Bianco S, Cusano C. Color target localization under varying illumination conditions. Computational Color Imaging. Lecture Notes in Computer Science, 2011, 6626: 245–255

3 Freedman D, Turek M W. Illumination — Invariant tracking via graph cuts. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington DC: IEEE Computer Society, 2005, 2: 10–17

4 Narasimhan S G, Nayar S K. Vision and the atmosphere. International Journal of Computer Vision, 2002, 48(3): 233–254

5 Stauffer C, Grimson W E L. Adaptive background mixture models for real-time tracking. In: Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington DC: IEEE Computer Society, 1999, 2: 2246–2252

6 Greenspan H, Goldberger J, Mayer A. Probabilistic space-time video modeling via piecewise GMM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(3): 384–396

PMID

7 Garg K, Nayar S K. Detection and removal of rain from videos. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington DC: IEEE Computer Society, 2004, 1: 528–535

8 Garg K, Nayar S K. Photorealistic rendering of rain streaks. ACM Transactions on Graphics, 2006, 25(3): 996–1002

9 Garg K, Nayar S K. Vision and rain. International Journal of Computer Vision, 2007, 75(1): 3–27

10 Barnum P C, Narasimhan S, Kanade T. Analysis of rain and snow in frequency space. International Journal of Computer Vision, 2009, 86(2-3): 256–274

11 Barnum P, Kanade T, Narasimhan S G. Spatio-temporal frequency analysis for removing rain and snow from videos. In: Proceedings of the First International Workshop on Photometric Analysis for Computer Vision, in conjunction with International Conference of Computer Vision. Rio de Janeiro: INRIA, 2007, 1–8

12 Starik S, Werman M. Simulation of rain in videos. In: Proceedings of the 3rd International Workshop on Texture Analysis and Synthesis. Edinburgh: IEEE Press, 2003, 95–100

13 Zhang X P, Li H, Qi Y Y, Leow W K, Ng T K. Rain removal in video by combining temporal and chromatic properties. In: Proceedings of the 2006 IEEE International Conference on Multimedia and Expo. Washington DC: IEEE Computer Society, 2006, 461–464

14 Faraji H, MacLean W J. CCD noise removal in digital images. IEEE Transactions on Image Processing, 2006, 15(9): 2676–2685

PMID

15 Khatri C G, Mardia K V. The von Mises-Fisher matrix distribution in orientation statistics. Journal of the Royal Statistical Society. Series B (Methodological), 1977, 39(1): 95–106

16 Rousseau P, Jolivet V, Ghazanfarpour D. Realistic real-time rain rendering. Computers & Graphics, 2006, 30(4): 507–518

17 Foote G B, Du Toit P S. Terminal velocity of raindrops aloft. Journal of Applied Meteorology, 1969, 8(2): 249–253