Online soft measurement for wastewater treatment system based on hybrid deep learning

Abstract

● A hybrid model is proposed to overcome limitations of single model with time series.

● CNN and bidirectional NLSTM are combined to solve complex nonlinear monitoring issue.

● Attention mechanism is suitably introduced to hybrid model for better convergence.

● TPE is used to find the optimal parameter combination faster rather than manual.

The existing automated wastewater treatment control systems encounter challenges such as the utilization of specialized testing instruments, equipment repair complications, high operational costs, substantial operational errors, and low detection accuracy. An effective soft measure model offers a viable approach for real-time monitoring and the development of automated control in the wastewater treatment process. Consequently, a novel hybrid deep learning CNN-BNLSTM-Attention (CBNLSMA) model, which incorporates convolutional neural networks (CNN), bidirectional nested long and short-term memory neural networks (BNLSTM), attention mechanisms (AM), and Tree-structure Parzen Estimators (TPE), has been developed for monitoring effluent water quality during the wastewater treatment process. The CBNLSMA model is divided into four stages: the CNN module for feature extraction and data filtering to expedite operations; the BNLSTM module for temporal data’s temporal information extraction; the AM module for model weight reassignment; and the TPE optimization algorithm for the CBNLSMA model’s hyperparameter search optimization. In comparison with other models (TPE-CNN-BNLSTM, TPE-BNLSTM-AM, TPE-CNN-AM, PSO-CBNLSTMA), the CBNLSMA model reduced the RMSE for effluent COD prediction by 25.4%, decreased the MAPE by 32.9%, and enhanced the R² by 14.9%. For the effluent SS prediction, the CBNLSMA model reduced the RMSE by 26.4%, the MAPE by 21.0%, and improved the R² by 35.7% compared to other models. The simulation results demonstrate that the proposed CBNLSMA model holds significant potential for real-time effluent quality monitoring, indicating its high potential for automated control in wastewater treatment processes.

Graphical abstract

Keywords

Prediction model / Soft measurement / CNN-BNLSTM-AM model / TPE optimization algorithm

References

[1] Abdeljaber O, Avci O, Kiranyaz S, Gabbouj M, Inman D J. (2017). Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks. Journal of Sound and Vibration, 388: 154–170

[2] AhmedA N, Othman F B, AfanH A, IbrahimR K, FaiC M, HossainM S, Ehteram M, ElshafieA (2019). Machine learning methods for better water quality prediction. Journal of Hydrology, 578: 124084

[3] Bergstra J, Komer B, Eliasmith C, Yamins D, Cox D D. (2015). Hyperopt: a Python library for model selection and hyperparameter optimization. Computational Science and Discovery, 8(1): 014008–014024

[4] ChateM, Gohokar V (2020). Vehicle Detection Using Faster Recurrent Convolution Neural Network. ICDSMLA 2019. Proceedings of the 1st International Conference on Data Science, Machine Learning and Applications. Lecture Notes in Electrical Engineering, (LNEE 601): 1190–1195

[5] Chen Q P, Xie Q S, Yuan Q N, Huang H S, Li Y T. (2019). Research on a real-time monitoring method for the wear state of a tool based on a convolutional bidirectional LSTM model. Symmetry, 11(10): 1233

[6] Chen Z B, Hu D X, Ren N Q, Zhang Z P. (2008). Simultaneous removal of organic substances and nitrogen in pilot-scale submerged membrane bioreactors treating digested traditional Chinese medicine wastewater. International Biodeterioration & Biodegradation, 62(3): 250–256

[7] Choi H, Cho K, Bengio Y. (2018). Fine-grained attention mechanism for neural machine translation. Neurocomputing, 284: 171–176

[8] Farhi N, Kohen E, Mamane H, Shavitt Y. (2021). Prediction of wastewater treatment quality using LSTM neural network. Environmental Technology & Innovation, 23(2): 101632

[9] Fulcher B D, Little M A, Jones N S. (2013). Highly comparative time-series analysis: the empirical structure of time series and their methods. Journal of the Royal Society, Interface, 10: 20130048

[10] Galassi A, Lippi M, Torroni P. (2021). Attention in natural language processing. IEEE Transactions on Neural Networks and Learning Systems, 32(10): 4291–4308

[11] Gao L L, Li X P, Song J K, Shen H T. (2020). Hierarchical lstms with adaptive attention for visual captioning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(5): 1112–1131

[12] Han H G, Zhang H J, Liu Z, Qiao J F. (2020). Data-driven decision-making for wastewater treatment process. Control Engineering Practice, 96: 104305

[13] Hao S Y, Lee E O R, Zhao D. (2019). Sequence to sequence learning with attention mechanism for short-term passenger flow prediction in large-scale metro system. Transportation Research Part C, Emerging Technologies, 107: 287–300

[14] Hu L L, Wang J L, Wen X G, Qian Y. (2005). Study on performance characteristics of SBR under limited dissolved oxygen. Process Biochemistry (Barking, London, England), 40(1): 293–296

[15] HuY XGuo C KMeiNZhangJGongZ K ZhaoJ (2022). Prediction of Boiler Control Parameters Based on LSTM Neural Network. 2022 4th Asia Energy and Electrical Engineering Symposium (AEEES), Chengdu, China, 2022: 451−457

[16] Inoue T, Mulligan C N, Zadeh E M, Fukue M. (2009). Effect of contaminated suspended solids on water and sediment qualities and their treatment. Journal of ASTM International, 6(3): JAI102185

[17] Jin N, Zeng Y K, Yan K, Ji Z W. (2021). Multivariate air quality forecasting with nested long short term memory neural network. IEEE Transactions on Industrial Informatics, 17(12): 8514–8522

[18] Khullar S, Singh N. (2022). Water quality assessment of a river using deep learning Bi-LSTM methodology: forecasting and validation. Environmental Science and Pollution Research International, 29(9): 12875–12889

[19] Kiranyaz S, Avci O, Abdeljaber O, Ince T, Gabbouj M, Inman D J. (2021). 1D convolutional neural networks and applications: a survey. Mechanical Systems and Signal Processing, 151: 107398

[20] Kong J B, Minseok J. (2019). Association analysis of convolution layer, kernel and accuracy in cnn. Journal of the Korea Institute of Electronic Communication Sciences, 14(6): 1153–1160

[21] LiR, ZhengS Y, DuanC X, Yang Y, WangX Q (2020). Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sensing (Basel), 12(3): 582

[22] Li X Y, Yi X H, Liu Z H, Liu H B, Chen T, Niu G Q, Yan B, Chen C, Huang M Z, Ying G G. (2021). Application of novel hybrid deep leaning model for cleaner production in a paper industrial wastewater treatment system. Journal of Cleaner Production, 2: 126343

[23] Li J, Chen Z, Li X, Yi X, Zhao Y, He X, Huang Z, Hassaan M A, Nemr A E, Huang M. (2023). Water quality soft-sensor prediction in anaerobic process using deep neural network optimized by Tree-structured Parzen Estimator. Frontiers of Environmental Science & Engineering, 17(6): 67

[24] LiangJ, Luo F, XuY G (2009). Wastewater DO concentration control through NH4 prediction based on evolutionary radial basis function neural network. Proceedings of the 2009 Fifth International Conference on Natural Computation, Tianjian, China, 2009. (ICNC 2009): 378–381

[25] Liu L, Chen J, Fieguth P, Zhao G Y, Chellappa R, Pietikainen M. (2019). From bow to CNN: two decades of texture representation for texture classification. International Journal of Computer Vision, 127(1): 74–109

[26] LiuR M, Ning X, CaiW W, LiG J (2021). Multiscale dense cross-attention mechanism with covariance pooling for hyperspectral image scene classification. Mobile Information Systems, 2021(4): 9962057

[27] Ma X L, Zhong H Y, Li Y, Ma J Y, Cui Z Y, Wang Y H. (2021). Forecasting transportation network speed using deep capsule networks with nested LSTM models. IEEE Transactions on Intelligent Transportation Systems, 22(8): 4813–4824

[28] MonizJ, Krueger D (2018). Nested LSTMs. JMLR: Workshop and Conference Proceedings, 15−17 Nov., Yonsei University, Seoul, Republic of Korea

[29] NguyenH P, Liu J, ZioE (2020). A long-term prediction approach based on long short-term memory neural networks with automatic parameter optimization by tree-structured parzen estimator and applied to time-series data of NPP steam generators. Applied Soft Computing, 165: 89

[30] Niu G Q, Yi X H, Chen C, Li X Y, Han D H, Yan B, Huang M Z, Ying G G. (2020). A novel effluent quality predicting model based on genetic-deep belief network algorithm for cleaner production in a full-scale paper-making wastewater treatment. Journal of Cleaner Production, 265: 121787

[31] Peng Y Q, Kondo N S, Fujiura T, Suzuki T, Ouma S, Wulandari Y, Hidetsugu I, Erina I. (2020). Dam behavior patterns in Japanese black beef cattle prior to calving: automated detection using lstm-rnn. Computers and Electronics in Agriculture, 169: 105178

[32] PhanH, Hertel L, MaassM, MertinsA, Int Speech Commun A (2016). Robust audio event recognition with 1-Max pooling convolutional. Neural Networks, 2016: 3653–3657,

[33] Pu Z, Yan J, Chen L, Li Z, Tian W, Tao T, Xin K. (2023). A hybrid Wavelet-CNN-LSTM deep learning model for short-term urban water demand forecasting. Frontiers of Environmental Science & Engineering, 17(2): 1–14

[34] RagiN M, Holla R, ManjuG (2019). Predicting Water Quality Parameters Using Machine Learning. 2019 4th International Conference on Recent Trends on Electronics, Information, Communication & Technology (RTEICT) , Bangalore, India, 2019: 1109–1112

[35] Roscher R, Bohn B, Duarte M F, Garcke J. (2020). Explainable machine learning for scientific insights and discoveries. IEEE Access: Practical Innovations, Open Solutions, 8: 42200–42216

[36] Schmidt-Hieber J. (2020). Nonparametric regression using deep neural networks with ReLU activation function. Annals of Statistics, 48(4): 1875–1897

[37] ShaJ, LiX, ZhangM, Wang Z L (2021). Comparison of forecasting models for real-time monitoring of water quality parameters based on hybrid deep learning neural networks. Water (Basel) 13: 1547

[38] Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15: 1929–1958

[39] Tan W W, Zhang J J, Wu J, Lan H, Liu X, Xiao K, Wang L, Lin H J, Sun G, Guo P. (2022). Application of CNN and long short-term memory network in water quality predicting. Intelligent Automation and Soft Computing, 34(3): 1943–1958

[40] Wan J Q, Huang M Z, Ma Y W, Guo W J, Wang Y, Zhang H P, Li W J, Sun X F. (2011). Prediction of effluent quality of a paper mill wastewater treatment using an adaptive network-based fuzzy inference system. Applied Soft Computing, 11(3): 3238–3246

[41] WangX, Chen S H, SuJ S (2020). Real network traffic collection and deep learning for mobile app identification. Wireless Communications and Mobile Computing, 2020, : 4707909: 1–4707909:14

[42] Wang Z F, Man Y, Hu Y S, Li J G, Hong M N, Cui P Z. (2019). A deep learning-based dynamic COD prediction model for urban sewage. Environmental Science. Water Research & Technology, 5(12): 2210–2218

[43] Wu C, Guo L. (2017). Influence of temperature and dissolved oxygen on nitrogen and phosphorus removal of integrated bioreactor. International Journal Bioautomation, 21(2): 207–216

[44] WuJ, LiZ B, ZhuL, LiG Y, NiuB S, Peng F (2018). Optimized BP neural network for dissolved oxygen prediction, 596–601

[45] Xu L Q, Liu S Y. (2013). Study of short-term water quality prediction model based on wavelet neural network. Mathematical and Computer Modelling, 58(3–4): 807–813

[46] YanC G, Tu Y B, WangX Z, ZhangY B, HaoX H, ZhangY D, Dai Q H (2020). STAT: spatial-temporal attention mechanism for video captioning. 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass, CO, USA, 2020, 1606–1615

[47] Yang F, Xie H, Li H X. (2019). Retracted article: video associated cross-modal recommendation algorithm based on deep learning. Applied Soft Computing, 82: 105597

[48] Yang L, Shami A. (2020). On hyperparameter optimization of machine learning algorithms: theory and practice. Neurocomputing, 415: 295–316

[49] Yang Y R, Xiong Q Y, Wu C, Zou Q H, Yu Y, Yi H L, Gao M. (2021). A study on water quality prediction by a hybrid CNN-LSTM model with attention mechanism. Environmental Science and Pollution Research International, 28(39): 55129–55139

[50] Yu Y, Si X S, Hu C H, Zhang J X. (2019). A review of recurrent neural networks: LSTM cells and network architectures. Neural Computation, 31(7): 1235–1270

[51] Zhang J, Peng Y, Ren B, Li T. (2021). PM_2.5 concentration prediction based on CNN-BiLSTM and attention mechanism. Algorithms, 14(7): 208

[52] Zhao J F, Mao X, Chen L J. (2018). Learning deep features to recognize speech emotion using merged deep CNN. IET Signal Processing, 12(6): 713–721

RIGHTS & PERMISSIONS

Higher Education Press