A pixel-based outlier-free motion estimation algorithm for scalable video quality enhancement

Abstract

Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to search for correspondence between high quality and low quality frames. Previous algorithms usually use block-based motion estimation to search for correspondences. Such an approach can hardly estimate scale and rotation transforms and always introduces outliers to the motion estimation results. In this paper, we propose a pixel-based outlier-free motion estimation algorithm to solve this problem. In our algorithm, the motion vector for each pixel is calculated with respect to estimate translation, scale, and rotation transforms. The motion relationships between neighboring pixels are considered via the Markov random field model to improve the motion estimation accuracy. Outliers are detected and avoided by taking both blocking effects and matching percentage in scaleinvariant feature transform field into consideration. Experiments are conducted in two scenarios that exhibit spatial scalability and quality scalability, respectively. Experimental results demonstrate that, in comparison with previous algorithms, the proposed algorithm achieves better correspondence and avoids the simultaneous introduction of outliers, especially for videos with scale and rotation transforms.

Keywords

motion estimation / scalable video coding / video super resolution

References

[1] Sodagar I. TheMPEG-DASH standard for multimedia Streaming Over the Internet. IEEE Multimedia, 2011, 18(4): 62―67

[2] Schwarz H, Marpe D, Wiegand T. Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Transactions on Circuits and Systems for Video Technology, 2007, 17(9): 1103―1120

[3] Song B C, Jeong S C, Choi Y. Video super-resolution algorithm using bi-directional overlapped block motion compensation and onthefly dictionary training. IEEE Transactions on Circuits and Systems for Video Technology, 2011, 21(3): 274―285

[4] Hung E M, de Queiroz R L, Brandi F, de Oliveira K F, Mukherjee D. Video super-resolution using codebooks derived from keyframes. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(9): 1321―1331

[5] Ferreira R U, Hung EM, de Queiroz R L. Video super resolution based on local invariant features matching. In: Proceedings of the 19th IEEE International Conference on Image Processing. 2012, 877―880

[6] Lowe D G. Object recognition from local scale-invariant features. In: Proceedings of the 17th IEEE International Conference on Computer Vision. 1999, 1150―1157

[7] Freeman WT, Jones T R, Pasztor E C. Example-based superresolution. IEEE Computer Graphics and Applications, 2002, 22(2): 56―65

[8] Brandi F, de Queiroz R, Mukherjee D. Super resolution of video using key-frames. In: Proceedings of the IEEE International Symposium on Circuits Systems. 2008, 1608―1611

[9] Brandi F, de Queiroz R L, Mukherjee D. Super-resolution of video using key-frames and motion estimation. In: Proceedings of the 15th IEEE International Conference on Image Processing. 2008, 321―324

[10] Oliveira K F, Brandi F, Hung E M, de Queiroz R L, Mukherjee D. Bipredictive video super-resolution using key-frames. In: Proceedings of SPIE Symposium on Electronic Image, Visual Information Processing and Communication. 2010, 1―5

[11] Hung E M, de Queiroz R L, Mukherjee D. Inter-frame postprocessing for intra-coded video. Journal of Communication and Information Systems, 2013, 28(1): 1―7

[12] Wen J, Li S, Lu Y, Fang M, Dong X, Chang H, Tao P. Cross segment decoding for improved quality of experience for video applications. In: Proceedings of the 2013 IEEE Data Compression Conference. 2013, 231―240

[13] Wang Q, Tang X, Shum H. Patch based blind image super resolution. In: Proceedings of the 10th IEEE International Conference on Computer Vision. 2005, 709―716

[14] Stephenson T A, Chen T. Adaptive Markov random fields for examplebased super-resolution of faces. Journal on Applied Signal Processing, 2006, 2006: 1―11

[15] Qiu G. Interresolution look-up table for improved spatial magnification of image. Journal of Visual Communication and Image Representation. 2000, 11: 360―373

[16] Elad M, Datsenko D. Example-based regularization deployed to superresolution reconstruction of single image. The Computer Journal Advance Access, 2007, 20: 15―30

[17] Besag J. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society, Series B, 1974, 36: 192―293

[18] Sun D, Roth S, Lewis J, Black M J. Learning optical flow. Lecture Notes in Computer Science, 2008, 5304: 83―97

[19] Liu C, Yuen J, Torralba A, Sivic J, Freeman W T. SIFT flow: dense correspondence across different scenes. Lecture Notes in Computer Science, 2008, 5304: 28―42

[20] Pan F, Lin X, Rahardja S, Lin W, Ong E, Yao S, Lu Z, Yang X. A locally adaptive algorithm for measuring blocking artifacts in images and videos. Signal Processing: Image Communication, 2004, 19(6): 499―506

[21] Brown M, Lowe D G. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 2007, 74(1): 59―73

[22] Horn B, Schunck B. Determining optical flow. Artificial Intelligence, 1981, 16: 185―203

[23] Wang S, Uchida S, Liwicki M, Feng Y K. Part-based methods for handwritten digit recognition. Frontiers of Computer Science, 2013, 7(4): 514―525

[24] Mehrotra H, Majhi B. Local feature based retrieval approach for iris biometrics. Frontiers of Computer Science, 2013, 7(5): 767―781

[25] PRIYA R, Shanmugama T H. Comprehensive review of significant researches on content based indexing and retrieval of visual information. Frontiers of Computer Science, 2013, 7(5): 782―799

[26] Wang Y W, Zhou Y C, Liu Y, Luo Z, Guo D H, Shao J, Tan F, Wu L, Li J H, Yan B P. A grid-based clustering algorithm for wild bird distribution. Frontiers of Computer Science, 2013, 7(4): 475―485

[27] Kang L, Wu L D, Yang Y H. A novel unsupervised approach for multilevel image clustering from unordered image collection. Frontiers of Computer Science, 2013, 7(1): 69―82

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg