Clustering-based temporal deep neural network denoising method for event-based sensors

Abstract

To enhance the denoising performance of event-based sensors, we introduce a clustering-based temporal deep neural network denoising method (CBTDNN). Firstly, to cluster the sensor output data and obtain the respective cluster centers, a combination of density-based spatial clustering of applications with noise (DBSCAN) and Kmeans++ is utilized. Subsequently, long short-term memory (LSTM) is employed to fit and yield optimized cluster centers with temporal information. Lastly, based on the new cluster centers and denoising ratio, a radius threshold is set, and noise points beyond this threshold are removed. The comprehensive denoising metrics F1_score of CBTDNN have achieved 0.893 1, 0.773 5, and 0.921 5 on the traffic sequences dataset, pedestrian detection dataset, and turntable dataset, respectively. And these metrics demonstrate improvements of 49.90%, 33.07%, 19.31%, and 22.97% compared to four contrastive algorithms, namely nearest neighbor (NNb), nearest neighbor with polarity (NNp), Autoencoder, and multilayer perceptron denoising filter (MLPF). These results demonstrate that the proposed method enhances the denoising performance of event-based sensors.

References

[1] ChenG Z, LiuS, XuJ T. Memory-boosting RNN with dynamic graph for event-based action recognition[J]. Optoelectronics letters, 2023, 19(10): 629-634

[2] LiuS, ChengH H, HuangS Y, et al.. Fast event-inpainting based on lightweight generative adversarial nets[J]. Optoelectronics letters, 2021, 17(8): 507-512

[3] StromatiasE, SotoM, SerranoG M T, et al.. An event-based classifier for dynamic vision sensor and synthetic data[J]. Applied sciences, 2020, 10(1): 2024-2042

[4] CzechD, OrchardG. Evaluating noise filtering for event-based asynchronous change detection image sensors[C]. 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics, June 26–29, 2016, Singapore, 2016 New York IEEE 19-24

[5] LvY, LiuZ, ZhouL, et al.. Denoising algorithm based on event camera[C]//6th Conference on Frontiers in Optical Imaging and Technology: Novel Detector Technologies, October 22–24, 2023, Nanjing, China. Washington: SPIE, 2024, 13154: 59-65

[6] FengY, LvH, LiuH, et al.. Event density based denoising method for dynamic vision sensor[J]. Applied sciences, 2020, 10(6): 2024

[7] TangS, LvH, ZhaoY, et al.. Denoising method based on salient region recognition for the spatiotemporal event stream[J]. Sensors, 2023, 23(15): 6655

[8] DuanP, WangZ W, ZhouX, et al.. EventZoom: learning to denoise and super resolve neuromorphic events[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, June 19–25, 2021, California, USA, 2021 New York IEEE 12824-12833

[9] FangH, WuJ, LiL, et al.. AEDNet: asynchronous event denoising with spatial-temporal correlation among irregular data[C]. Proceedings of the 30th ACM International Conference on Multimedia, October 10–14, 2022, Lisbon, Portugal, 2022 New York ACM 1427-1435

[10] GuoS, DelbruckT. Low cost and latency event camera background activity denoising[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(1): 785-795

[11] YaoY H, PanY G, LiJ, et al.. Sanitized clustering against confounding bias[J]. Machine learning, 2024, 113(6): 3711-3730

[12] HuoC, BaiH F, YinZ B, et al.. Naive-LSTM based services awareness of edge computing elastic optical networks[J]. Optoelectronics letters, 2023, 19(5): 279-283

[13] KhodamoradiA, KastnerR. O(N)-space spatiotemporal filter for reducing noise in neuromorphic vision sensors[J]. IEEE transactions on emerging topics in computing, 2021, 9(1): 15-23

[14] RoyP, ZhongG Q, CherietM. Tandem hidden Markov models using deep belief networks for offline handwriting recognition[J]. Frontiers of information technology & electronic engineering, 2017, 18(7): 978-988

[15] LeungW K, LiuX, MengH. CNN-RNN-CTC based end-to-end mispronunciation detection and diagnosis[C]. 2019 IEEE International Conference on Acoustics, Speech and Signal Processing, May 12–17, 2019, Brighton, UK, 2019 New York IEEE 8132-8136

[16] El ShairZ, RawashdehS A. High-temporal-resolution object detection and tracking using images and events[J]. Journal of imaging, 2022, 8(8): 210

[17] MiaoS, ChenG, NingX, et al.. Neuromorphic vision datasets for pedestrian detection, action recognition, and fall detection[J]. Frontiers in neurorobotics, 2019, 13: 38

[18] ZhangZ, SuoJ, DaiQ H. Denoising of event-based sensors with deep neural networks[C]//Optoelectronic Imaging and Multimedia Technology VIII, October 10–12, 2021, Nantong, China. Washington: SPIE, 2021, 11897: 203-209

[19] LiuH, BrandliC, LiC, et al.. Design of a spatiotemporal correlation filter for event-based sensors[C]. 2015 IEEE International Symposium on Circuits and Systems, May 24–27, 2015, Lisbon, Portugal, 2015 New York IEEE 722-725

RIGHTS & PERMISSIONS

Tianjin University of Technology