Motion-adaptive adjacent-reference skipping for distributed video compressive sensing with general decoders

Abstract

On an internet of video things (IoVT), an encoder needs to collect a large number of signal samples to improve the reconstruction quality. It is challenging to some occasions where the resources of an encoder are extremely limited. The distributed video compressive sensing (DVCS) can save a lot of resources for the encoder. For the skip-block coding at such an encoder, this paper proposes a motion-adaptive adjacent-reference skipping (MAS) algorithm for DVCS with general decoders. The proposed algorithm makes full use of the spatial-temporal correlation between consecutive frames, and the reconstruction quality can be improved significantly. What’s more, the skipping ratio of non-keyframes is adaptive to the difference of their motion-speeds. The proposed algorithm does not need to change any decoder, so it can be easily applied to general decoders. The simulation results show that under different skipping ratios, the proposed algorithm can achieve better reconstruction quality than other existing algorithms, and thus improve the energy-efficiency of the encoder.

References

[1] CHENC W. Internet of video things: next-generation IoT with visual sensors[J]. IEEE internet of things journal, 2020, 7(8):6676-6685

[2] YANGY, GUOS, LIUG, et al.. Two-layer compressive sensing based video encoding and decoding framework for WMSN[J]. Journal of network and computer applications, 2018, 117: 72-85

[3] ZHANGY, ZHANGC, FANR, et al.. Recent advances on HEVC inter-frame coding: from optimization to implementation and beyond[J]. IEEE transactions on circuits and systems for video technology, 2020, 30(11):4321-4339

[4] YANGH. Subblock-based motion derivation and inter prediction refinement in the versatile video coding standard[J]. IEEE transactions on circuits and systems for video technology, 2021, 31(10):3862-3877

[5] SHANGF, YEJ J. Review of distributed video coding[C]//13th IEEE International Conference on Electronic Measurement & Instruments, October 20–22, 2017, Yangzhou, China, 2017, New York, IEEE: 315-320

[6] RANIM, DHOKS B, DESHMUKHR B. A systematic review of compressive sensing: concepts, implementations and applications[J]. IEEE access, 2018, 6: 4875-4894

[7] GANL. Block compressed sensing of natural images[C]//15th International Conference on Digital Signal Processing, 2007, Cardiff, UK, 2007, New York, IEEE: 403-406

[8] BAIGY, LAIE M K, PUNCHIHEWAA. Distributed video coding based on compressed sensing[C]//2012 International Conference on Multimedia and Expo Workshops, July 9–13, 2012, Melbourne, Australia, 2012, New York, IEEE: 325-330

[9] COSSALTERM, VALENZISEG, TAGLIASACCHIM, et al.. Joint compressive video coding and analysis[J]. IEEE transactions on multimedia, 2010, 12(3):168-183

[10] MUNS, FOWLERJ E. Residual reconstruction for block-based compressed sensing of video[C]//2011 Data Compression Conference, March 23–26, 2011, Snowbird, USA, 2011, New York, IEEE: 183-192

[11] ZHOUJ, LIUH. Video compressive sensing reconstruction via long-short-term double-pattern prediction[J]. Optoelectronics letters, 2020, 16(3):230-236

[12] ZHAOC, MAS, ZHANGJ, et al.. Video compressive sensing reconstruction via reweighted residual sparsity[J]. IEEE transactions on circuits and systems for video technology, 2017, 27(6):1182-1195

[13] ZHENGS, ZHANGX P, CHENJ, et al.. A high-efficiency compressed sensing-based terminal-to-cloud video transmission system[J]. IEEE transactions on multimedia, 2019, 21(8):1905-1920

[14] ZHENGS, CHENJ, ZHANGX P, et al.. A new multihypothesis-based compressed video sensing reconstruction system[J]. IEEE transactions on multimedia, 2021, 23: 3577-3589

[15] YUEY, LUOJ, HUAL. Distributed video compressed sensing secondary reconstruction based on inter-frames similarity structure[J]. IOP conference series: materials science and engineering, 2021, 1043(5):052012

[16] FOWLERJ E. Block-based compressed sensing of images and video[J]. Foundations & trends in signal processing, 2012, 4(4): 297-416

[17] UNDEA S, PATTATHILD P. Adaptive compressive video coding for embedded camera sensors: compressed domain motion and measurements estimation[J]. IEEE transactions on mobile computing, 2020, 19(10):2250-2263

[18] WANGJ, WANGW, CHENJ. Adaptive rate block compressive sensing based on statistical characteristics estimation[J]. IEEE transactions on image processing, 2022, 31: 734-747

[19] LIR, YANGY, SUNF. Green visual sensor of plant: an energy-efficient compressive video sensing in the internet of things[J]. Frontiers in plant science, 2022, 13: 849606