Dynamic GNN-based multimodal anomaly detection for spatial crowdsourcing drone services^✩

Abstract

We introduce a pioneering anomaly detection framework within spatial crowdsourcing Internet of Drone Things (IoDT), specifically designed to improve bushfire management in Australia’s expanding urban areas. This framework innovatively combines Graph Neural Networks (GNN) and advanced data fusion techniques to enhance IoDT capabilities. Through spatial crowdsourcing, drones collectively gather diverse, real-time data across multiple locations, creating a rich dataset for analysis. This method integrates spatial, temporal, and various data modalities, facilitating early bushfire detection by identifying subtle environmental and operational changes. Utilizing a complex GNN architecture, our model effectively processes the intricacies of spatially crowdsourced data, significantly increasing anomaly detection accuracy. It incorporates modules for temporal pattern recognition and spatial analysis of environmental impacts, leveraging multimodal data to detect a wide range of anomalies, from temperature shifts to humidity variations. Our approach has been empirically validated, achieving an F1 score of 0.885, highlighting its superior anomaly detection performance. This integration of spatial crowdsourcing with IoDT not only establishes a new standard for environmental monitoring but also contributes significantly to disaster management and urban sustainability.

Keywords

Anomaly detection / Multi-modal data / GNN / IoDT / Data fusion

References

[1] S. Sidana, D. Kumar, R.H. Jhaveri,Leveraging blockchain-as-a-certificate authority for authentication in 6g-enabled spatial crowdsourcing drone services, in: 2024 IEEE 100th Vehicular Technology Conference (VTC2024-Fall), IEEE, 2024, pp. 1-5.

[2] H.S. Munawar, Z. Gharineiat, S. Imran Khan, A framework for burnt area mapping and evacuation problem using aerial imagery analysis, Fire 5 (4) (2022) 122.

[3] J. Akram, A. Anaissi, R.S. Rathore, R.H. Jhaveri, A. Akram, Galtrust: Generative ad- verserial learning-based framework for trust management in spatial crowdsourcing drone services, IEEE Trans. Consum. Electron. 70 (3) (2024) 6196-6207, https://doi.org/10.1109/TCE.2024.3384978.

[4] J. Akram, M. Umair, R.H. Jhaveri, M.N. Riaz, H. Chi, S. Malebary, Chained-drones: Blockchain-based privacy-preserving framework for secure and intelligent service provisioning in internet of drone things, Comput. Electr. Eng. 110 (2023) 108772.

[5] H. Rizvi, Handover management in 5g software defined network based v2x com- munication, in: 2018 12th International Conference on Open Source Systems and Technologies (ICOSST), IEEE, 2018, pp. 22-26.

[6] A. Akram, A. Alabdultif, A. Anaissi, R.H. Jhaveri, Secure and interoperable iomt- based smart homes, IEEE Consum. Electron. Mag. 14 (4) (2025) 100-105, https://doi.org/10.1109/MCE.2025.3534442.

[7] S. Ramaswamy, R. Rastogi, K. Shim, Efficient algorithms for mining outliers from large data sets, in: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, ACM Press, 2000, pp. 427-438.

[8] W. Jin, A.K. Tung, J. Han, Mining top-n local outliers in large databases, in: Proceed- ings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM Press, 2001, pp. 293-298.

[9] M. Braei, S. Wagner, Anomaly detection in univariate time-series: A survey on the state-of-the-art, preprint, arXiv:2004.00433.

[10] L. Erhan, M. Ndubuaku, M. Di Mauro, W. Song, M. Chen, G. Fortino, O. Bagdasar, A. Liotta, Smart anomaly detection in sensor systems: A multi-perspective review, Inf. Fusion 67 ( 2021) 64-79.

[11] J. Akram, A. Anaissi, R.S. Rathore, R.H. Jhaveri, A. Akram, Digital twin-driven trust management in open ran-based spatial crowdsourcing drone services, IEEE Trans. Green Commun. Netw. 8 (2) (2024) 1061-1075, https://doi.org/10.1109/TGCN.2024.3403901.

[12] X. Ma, J. Wu, S. Xue, J. Yang, C. Zhou, Q.Z. Sheng, H. Xiong, L. Akoglu, A comprehen- sive survey on graph anomaly detection with deep learning, IEEE Trans. Knowl. Data Eng. 35 (12) (2023) 12012-12038, https://doi.org/10.1109/TKDE.2021.3118815.

[13] Z. Najam, H. Rizwi, Energy efficient localization in wireless sensor networks using computational intelligence, in: 2018 15th International Conference on Smart Cities: Improving Quality of Life Using ICT & IoT (HONET-ICT), IEEE, 2018, pp. 78-82.

[14] J. Akram, A. Anaissi, A blockchain-enhanced framework for privacy and data in- tegrity in crowdsourced drone services, in: Proceedings of the 22nd International Conference on Service-Oriented Computing, Springer, 2024.

[15] S. Malik, S. Ansari, H. Rizvi, D. Kim, R. Hasnain,Intelligent target coverage in wireless sensor networks with adaptive sensors, in: 2020 IEEE 92nd Vehicular Tech- nology Conference (VTC2020-Fall), IEEE, pp. 1-5.

[16] J. Akram, A. Anaissi, A. Akram, R.S. Rathore, R.H. Jhaveri, Adversarial label-flipping attack and defense for anomaly detection in spatial crowdsourcing uav services, IEEE Trans. Consum. Electron. 71 (1) (2024) 2163-2174, https://doi.org/10.1109/TCE.2024.3448541.

[17] J. Akram, A. Akram, R.H. Jhaveri, M. Alazab, H. Chi, Bc-iodt: blockchain-based framework for authentication in internet of drone things,in:Proceedings of the 5th International ACM Mobicom Workshop on Drone Assisted Wireless Communications for 5G and Beyond, ACM Press, 2022, pp. 115-120.

[18] A. Tahir, A. Akram, A.Z. Kouzani, M.P. Mahmud, Cloud- and fog-integrated smart grid model for efficient resource utilisation, Sensors 21 (23) (2021) 7846.

[19] Z. Najam, A. Rafi, Efficient resource utilization in cloud-fog environment integrated with smart grids, in: 2018 International Conference on Frontiers of Information Tech- nology (FIT), IEEE, 2018, pp. 188-193.

[20] J. Chung, C. Gulcehre, K. Cho, Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, preprint, arXiv:1412.3555.

[21] W. Othman, A. Alabdulatif, Dronessl: Self-supervised multimodal anomaly detection in internet of drone things, IEEE Trans. Consum. Electron. 70 (1) (2024) 4287-4298.

[22] M.T.R. Khan, M.M. Saad, M.A. Tariq, D. Kim, Spice-it: Smart covid-19 pandemic controlled eradication over ndn-iot, Inf. Fusion 74 ( 2021) 50-64.

[23] I. Kiss, B. Genge, P. Haller, G. Sebestyén, Data clustering-based anomaly detection in industrial control systems, in: 2014 IEEE 10th International Conference on Intelligent Computer Communication and Processing (ICCP), IEEE, 2014, pp. 275-281.

[24] R. Blanquero, E. Carrizosa, C. Molero-Río, D.R. Morales, Optimal randomized clas- sification trees, Comput. Oper. Res. 132 (2021) 105281.

[25] J. Ma, S. Perkins, Time-series novelty detection using one-class support vector ma- chines, Proceedings of the International Joint Conference on Neural Networks, 2003, vol. 3, IEEE, 2003, pp. 1741-1745.

[26] L. Puggini, S. McLoone, An enhanced variable selection and isolation forest based methodology for anomaly detection with oes data, Eng. Appl. Artif. Intell. 67 (2018) 126-135.

[27] B. Zong, Q. Song, M.R. Min, W. Cheng, C. Lumezanu, D. Cho, H. Chen,Deep autoen- coding gaussian mixture model for unsupervised anomaly detection, in:International Conference on Learning Representations, 2018, pp. 1-19. Online: OpenReview.net.

[28] Y. Su, Y. Zhao, C. Niu, R. Liu, W. Sun, D. Pei, Robust anomaly detection for multi- variate time series through stochastic recurrent neural network, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM Press, 2019, pp. 2828-2837.

[29] L. Shen, Z. Yu, Q. Ma, J.T. Kwok, Time series anomaly detection with multiresolution ensemble decoding, in: Proceedings of the AAAI Conference on Artificial Intelligence, AAAI Press, 2021, pp. 9567-9575.

[30] K. Hundman, V. Constantinou, C. Laporte, I. Colwell, T. Soderstrom, Detecting space- craft anomalies using lstms and nonparametric dynamic thresholding, in: Proceed- ings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM Press, 2018, pp. 387-395.

[31] A. Deng, B. Hooi, Graph neural network-based anomaly detection in multivariate time series, in: Proceedings of the AAAI Conference on Artificial Intelligence, AAAI Press, 2021, pp. 4027-4035.

[32] H. Wen, Y. Liu, I. Rekik, S. Wang, Z. Chen, J. Zhang, Y. Zhang, Y. Peng, H. He, Multi-modal multiple kernel learning for accurate identification of tourette syn- drome children, Pattern Recognit. 63 (2017) 601-611.

[33] Y. Zhen, D.-Y. Yeung, A probabilistic model for multimodal hash function learning, in: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM Press, 2012, pp. 940-948.

[34] Y. Wang, W. Huang, F. Sun, T. Xu, Y. Rong, J. Huang, Deep multimodal fusion by channel exchanging, in: 34thInt. Conf. on Neural Information Processing Sys- tems (NeurIPS), Curran AssociatesInc., 2020, pp. 4835-4845.

[35] B.K. Iwana, S. Uchida, Time series classification using local distance-based features in multi-modal fusion networks, Pattern Recognit. 97 (2020) 107024.

[36] P. Yang, B. Chen, P. Zhang, X. Sun, Visual agreement regularized training for multi- modal machine translation, in: Proceedings of the AAAI Conference on Artificial Intelligence, AAAI Press, 2020, pp. 9418-9425.

[37] D. Park, Z. Erickson, T. Bhattacharjee, C.C. Kemp, Multimodal execution monitor-ing for anomaly detection during robot manipulation, in: 2016 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2016, pp. 407-414.

[38] D. Park, Y. Hoshi, C.C. Kemp, A multimodal anomaly detector for robot-assisted feeding using an lstm-based variational autoencoder, IEEE Robot. Autom. Lett. 3 (3) (2018) 1544-1551, https://doi.org/10.1109/lra.2018.2801475.

[39] S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Comput. 9 (8)(1997) 1735-1780.

[40] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio,Graph attention networks, preprint, arXiv:1710.10903.

[41] F. Ullah, D. Shahzad, A. Heravi, S. Qayyum, Civil infrastructure damage and corro-sion detection: An application of machine learning, Buildings 12 (2) (2022) 156.

[42] H.S. Munawar, A.Z. Kouzani, M.P. Mahmud, Using adaptive sensors for optimised target coverage in wireless sensor networks, Sensors 22 (3) (2022) 1083.

[43] H.S. Munawar, J. Akram, S.I. Khan, F. Ullah, B.J. Choi,Drone-as-a-service (daas) for covid-19 self-testing kits delivery in smart healthcare setups: a technological perspective, ICT Express 9 (4) (2023) 748-753.

[44] J. Akram, A. Anaissi, Decentralized pki framework for data integrity in spatial crowd-sourcing drone services, in: 2024 IEEE International Conference on Web Services (ICWS), IEEE, 2024, pp. 643-653.

[45] J. Akram, A. Anaissi, Ddrm: Distributed drone reputation management for trust and reliability in crowdsourced drone services, in: 2024 IEEE International Conference on Web Services (ICWS), IEEE, 2024, pp. 921-931.

[46] J. Akram, A. Anaissi, Privacy-first crowdsourcing: Blockchain and local differential privacy in crowdsourced drone services, in: 2024 IEEE International Conference on Web Services (ICWS), IEEE, 2024, pp. 1412-1414.

[47] A. Mukherjee, N. Dey, A. Mondal, D. De, R.G. Crespo, isocialdrone: Qos aware mqtt middleware for social internet of drone things in 6g-sdn slice, Soft Comput. 27 (8) (2023) 5119-5135.

[48] A. Mukherjee, D. De, N. Dey, R.G. Crespo, E. Herrera-Viedma, Disastdrone: A disaster aware consumer internet of drone things system in ultra-low latent 6g network, IEEE Trans. Consum. Electron. 69 (1) (2022) 38-48.

[49] H. Chai, S. Leng, Y. Chen, K. Zhang, A hierarchical blockchain-enabled federated learning algorithm for knowledge sharing in internet of vehicles, IEEE Trans. Intell. Transp. Syst. 22 (7) (2020) 3975-3986.

[50] Q. Zhang, M. Ye, X. Deng, A novel anomaly detection method for multimodal wsn data flow via a dynamic graph neural network, Connect. Sci. 34 (1) (2022) 1609-1637.

[51] S. Lu, F. Liu, Y. Li, K. Zhang, H. Huang, J. Zou, X. Li, Y. Dong, F. Dong, J. Zhu, Y. Xiong, W. Yuan, Y. Cui, L. Hanzo, Integrated sensing and communications: Re- cent advances and ten open challenges, IEEE Internet Things J. 11 (11) (2024) 19094-19120, https://doi.org/10.1109/JIOT.2024.3361173.

[52] M. Datar, A. Gionis, P. Indyk, R. Motwani, Maintaining stream statistics over slid- ing windows, SIAM J. Comput. 31 (6) (2002) 1794-1813, https://doi.org/10.1137/s0097539701398363.

[53] https://www.bunburymail.com.au/story/8048594/noggerup-bushfire-downgraded-after-overnight-spread/.

[54] S. Madden, Intel Berkeley Research lab, http://db.csail.mit.edu/labdata/labdata.html, 2004.