Network-aware perceptual error concealment method for H.264 video with side information

Abstract

In order to improve the video quality of transmission with data loss, a spatial and temporal error concealment method was proposed, which considered both the state information of the network and the perceptual weight of the video content. The proposed method dynamically changed the reliability weight of the neighboring macroblock, which was used to conceal the lost macroblocks according to the packet loss rate of the current channel state. The perceptual weight map was utilized as side information to do weighted pixel interpolation and side-match based motion compensation for spatial and temporal error concealment, respectively. And the perceptual weight of the neighboring macroblocks was adaptively modified according to the perceptual weight of the lost macroblocks. Compared with the method used in H.264 joint model, experiment results show that the proposed method performs well both in subjective video quality and objective video quality, and increases the average peak signal-to-noise ratio (PSNR) of the whole frame by about 0.4 dB when the video bitstreams are transmitted with packets loss.

Keywords

error concealment / reliability weight / perceptual weight map / weighted pixel interpolation / motion compensation / side information / H.264

References

[1] HsuC.-t., ChenM.-j., LiaoW.-w., LoS.-yi.. High-performance spatial and temporal error-concealment algorithms for block-based video coding techniques [J]. ETRI Journal, 2005, 27(1): 53-63

[2] WangY., ZhuQ.-fan.. Error control and concealment for video communication-A review [J]. Proceedings of the IEEE, 1998, 86(5): 974-997

[3] WangY., WengerS., WenJ. T., KatsaggelosA. K.. Error resilient video coding technologies [J]. IEEE Signal Processing Magazine, 2000, 17(4): 61-82

[4] ZengW. J., LiuB. D.. Geometric-structure-based error concealment with novel application in block-based low-bit-rate coding [J]. IEEE Transactions on Circuits and Systems for Video Technology, 1999, 9(4): 648-665

[5] ChenT. P., ChenT.. Second-generation error concealment for video transport over error-prone channels [J]. Wireless Communications and Mobile Computing, 2002, 2(6): 607-624

[6] YeS.-m., LinX.-g., SunQ.-bin.. Content based error detection and concealment for image transmission over wireless channel [C]. Proceedings of IEEE International Symposium on Circuits and Systems, 2003, New York, IEEE: II368-II371

[7] ChenC.-m., ChenC.-m., LinC.-w., ChenY.-chang.. Error-resilient video streaming over wireless networks using combined scalable coding and multiple-description coding [J]. Signal Processing: Image Communication, 2007, 22(4): 403-420

[8] MaM. Y., AuO. C., GuoL. W., Cary ChanS. H., WongP. H. W.. Error concealment for frame losses in MDC [J]. IEEE Transactions on Multimedia, 2008, 10(9): 1638-1647

[9] TuW., SteinbachE.. Proxy-based reference picture selection for error resilient conversational video in mobile networks [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2009, 19(2): 151-164

[10] ZhuC. B., WangY. K., HannukselaM. M., LiH. Q.. Error resilient video coding using redundant pictures [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2009, 19(1): 3-14

[11] RaneS., AaronA., GirodB.. Systematic lossy forward error protection for error-resilient digital video broadcasting — A Wyner-Ziv coding approach [C]. Proceedings of International Conference on Image Processing, 2004, New York, IEEE: 3103-3104

[12] ZengW.-jun.. Adaptive spatial-temporal error concealment with embedded side information [J]. Journal of Visual Communication and Image Representation, 2005, 16(4/5): 499-511

[13] KangL.-w., LeouJ.-jang.. An error resilient coding scheme for H.264/AVC video transmission based on data embedding [J]. Journal of Visual Communication and Image Representation, 2005, 16(1): 93-114

[14] AdedoyinS., FernandoW. A. C., KarimH. A., HewageC., KondozA. M.. Scalable multiple description coding with side information using motion interpolation [J]. IEEE Transactions on Consumer Electronics, 2008, 54(4): 2045-2052

[15] KumarS., XuL. Y., MandalM. K., PanchanathanS.. Error resiliency schemes in H.264/AVC standard [J]. Journal of Visual Communication and Image Representation, 2006, 17(2): 425-450

[16] WangY. K., HannukselaM. M., VarsaV., HourunrantaA., GabboujM.. The error concealment feature in the H.26L test model [C]. IEEE International Conference on Image Processing, 2002, New York, IEEE: II729-II732

[17] XuY.-l., ZhouY.-hua.. H.264 video communication based refined error concealment schemes [J]. IEEE Transactions on Consumer Electronics, 2004, 50(4): 1135-1141

[18] BuJ.-j., MoL.-j., ShaoG.-f., YangZ., ChenChun.. A novel fast error-resilient video coding scheme for H.264 [C]. Proceedings of 2006 IEEE 8th Workshop on Multimedia Signal Processing, 2006, Piscataway, IEEE: 252-257

[19] ChungY. Y., ChenL. L. F., ZhengZ., ChenX. M., FaithC.. An optimized and adaptive error-resilient coding for H.264 Video [C]. TENCON 2006–2006 IEEE Region 10 Conference, 2006, Piscataway, IEEE: 414-417

[20] ITU-T H.264, ISO/IEC 14496-10 AVC. Advanced video coding for generic audiovisual services [S]. 2009.

[21] FrossardP., VerscheureO.. AMISP: A complete content-based MPEG-2 error-resilient scheme [J]. IEEE Transactions on Circuits and Systems for Video Technology, 2001, 11(9): 989-998

[22] KungW. Y., KimC. S., KuoC. C. J.. Packet video transmission over wireless channels with adaptive channel rate allocation [J]. Journal of Visual Communication and Image Representation, 2005, 16(4/5): 475-498

[23] H.264/AVC Reference Software [EB/OL]. 2007-08-07. http://iphome.hhi.de/suehring/tml/download/.

[24] WangZ., LuL. Q., BovikA. C.. Video quality assessment based on structural distortion measurement [J]. Signal Processing: Image Communication, 2003, 19(2): 121-132