Efficient fast mode decision using mode complexity for multi-view video coding

Abstract

The variable block-size motion estimation (ME) and disparity estimation (DE) are adopted in multi-view video coding (MVC) to achieve high coding efficiency. However, much higher computational complexity is also introduced in coding system, which hinders practical application of MVC. An efficient fast mode decision method using mode complexity is proposed to reduce the computational complexity. In the proposed method, mode complexity is firstly computed by using the spatial, temporal and inter-view correlation between the current macroblock (MB) and its neighboring MBs. Based on the observation that direct mode is highly possible to be the optimal mode, mode complexity is always checked in advance whether it is below a predefined threshold for providing an efficient early termination opportunity. If this early termination condition is not met, three mode types for the MBs are classified according to the value of mode complexity, i.e., simple mode, medium mode and complex mode, to speed up the encoding process by reducing the number of the variable block modes required to be checked. Furthermore, for simple and medium mode region, the rate distortion (RD) cost of mode 16×16 in the temporal prediction direction is compared with that of the disparity prediction direction, to determine in advance whether the optimal prediction direction is in the temporal prediction direction or not, for skipping unnecessary disparity estimation. Experimental results show that the proposed method is able to significantly reduce the computational load by 78.79% and the total bit rate by 0.07% on average, while only incurring a negligible loss of PSNR (about 0.04 dB on average), compared with the full mode decision (FMD) in the reference software of MVC.

Keywords

multi-view video coding / mode decision / mode complexity / computational complexity

References

[1] TanimotoM, TehraniM P, FujiiT, YendoT. Free-viewpoint TV [J]. IEEE Signal Processing Magazine, 2011, 28(1): 67-76

[2] MullerK, MerkleP, WiegendT. 3-D video representation using depth maps [J]. Proceedings of the IEEE, 2011, 99(4): 643-656

[3] ISO/IEC 14496-10: 2008/FDAM 1:2008(E). Information technology-Coding of audio-visual objects-part 10: Advanced video coding, Amendment 1: multiview video coding [S].

[4] VetroA, WiegandT, SullivanG J. Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard [J]. Proceedings of the IEEE, 2011, 99(4): 626-642

[5] DingL-f, TsungP-k, ChienS-y, ChenW-y, ChenL-gee. Content-aware prediction algorithm with inter-view mode decision for multiview video coding [J]. IEEE Transactions on Multimedia, 2008, 10(8): 1553-1563

[6] HuoJ-y, ChangY-l, LiM, MaY-zhuo. Scalable prediction structure for multiview video coding [C]. IEEE International Symposium on Circuits and Systems, 2009, Piscataway, IEEE: 2593-2596

[7] ChanC C, LinJ P, TangC W. On-line statistical analysis based fast mode decision for multi-view video coding [C]. Picture Coding Symposium, 2010, Piscataway, IEEE: 478-481

[8] ShenL-q, YanT, LiuZ, ZhangZ-y, AnP, YangLei. Fast mode decision for multiview video coding [C]. IEEE International Conference on Image Processing, 2009, Piscataway, IEEE: 2953-2956

[9] ZengH-q, MaK-k, CaiC-hui. Mode-correlation-based early termination mode decision for multi-view video coding [C]. IEEE International Conference on Image Processing, 2010, Piscataway, IEEE: 3405-3408

[10] SeoJ, SohnK. Early disparity estimation skipping for multi-view video coding [J]. EURASIP Journal on Wireless Communications and Networking, 2012, 2012(1): 1-12

[11] MerkleP, SmolicA, MullerK, WiegandT. Efficient prediction structure for multiview video coding [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2007, 17(11): 1461-1473

[12] SullivanG, WiegandT. Rate-distortion optimization for video compression [J]. IEEE Signal Processing Magazine, 1998, 15(6): 74-90

[13] WiegandT, SchwarzH, JochA, KossentiniF, SullivanG J. Rate-constrained coder control and comparison of video coding standards [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(7): 688-703

[14] ChoiI, LeeJ, JeonB. Fast coding mode selection with rate-distortion optimization for MPEG-4 part-10 AVC/H.264 [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2006, 16(12): 1557-1561

[15] ZengH-q, CaiC-h, MaK-kuang. Fast mode decision for H.264/AVC based on macroblock motion activity [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2009, 19(4): 491-499

[16] KooH S, JeonY J, JeonB MMVC motion skip mode, document JVT-W081 [R], 2007, San Jose, JVT

[17] WangF-s, ZengH-q, ShenQ-h, DuS-dan. Efficient early direct mode decision for multi-view video coding [J]. Signal Processing-Image Communication, 2013, 28(7): 736-744