Water quality soft-sensor prediction in anaerobic process using deep neural network optimized by Tree-structured Parzen Estimator
Junlang Li, Zhenguo Chen, Xiaoyong Li, Xiaohui Yi, Yingzhong Zhao, Xinzhong He, Zehua Huang, Mohamed A. Hassaan, Ahmed El Nemr, Mingzhi Huang
Water quality soft-sensor prediction in anaerobic process using deep neural network optimized by Tree-structured Parzen Estimator
● Hybrid deep-learning model is proposed for water quality prediction.
● Tree-structured Parzen Estimator is employed to optimize the neural network.
● Developed model performs well in accuracy and uncertainty.
● Usage of the proposed model can reduce carbon emission and energy consumption.
Anaerobic process is regarded as a green and sustainable process due to low carbon emission and minimal energy consumption in wastewater treatment plants (WWTPs). However, some water quality metrics are not measurable in real time, thus influencing the judgment of the operators and may increase energy consumption and carbon emission. One of the solutions is using a soft-sensor prediction technique. This article introduces a water quality soft-sensor prediction method based on Bidirectional Gated Recurrent Unit (BiGRU) combined with Gaussian Progress Regression (GPR) optimized by Tree-structured Parzen Estimator (TPE). TPE automatically optimizes the hyperparameters of BiGRU, and BiGRU is trained to obtain the point prediction with GPR for the interval prediction. Then, a case study applying this prediction method for an actual anaerobic process (2500 m3/d) is carried out. Results show that TPE effectively optimizes the hyperparameters of BiGRU. For point prediction of CODeff and biogas yield, R2 values of BiGRU, which are 0.973 and 0.939, respectively, are increased by 1.03%–7.61% and 1.28%–10.33%, compared with those of other models, and the valid prediction interval can be obtained. Besides, the proposed model is assessed as a reliable model for anaerobic process through the probability prediction and reliable evaluation. It is expected to provide high accuracy and reliable water quality prediction to offer basis for operators in WWTPs to control the reactor and minimize carbon emission and energy consumption.
Water quality prediction / Soft-sensor / Anaerobic process / Tree-structured Parzen Estimator
Fig.2 Flowchart of construction process of proposed model. The blue arrows represent the construction process, and the red arrows represent dataset combination. X denotes the features, Y denotes the labels. The superscript “tr” and “te” respectively denote training set and test set. The subscript “1” and “2” respectively denote the first and the second. |
Tab.1 Comparison of R2 over the last 30 trials between TPE and RS |
Optimization methods | Max R2 | Average R2 | Variance of R2 | p value of the t-test |
---|---|---|---|---|
TPE | 0.975 | 0.954 | 0.000297 | 0.0127 |
RS | 0.970 | 0.935 | 0.000732 |
Tab.2 Metrics of three models. |
Feature | Model | Point prediction | Interval prediction | Probability prediction | Test time (s) | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
R2 | RMSE* | MAPE (%) | CP | MWP | MC | CRPS | |||||
CODeff | BiGRU-GPR | 0.973 | 24.94 | 2.38 | 0.93 | 0.12 | 0.13 | 0.0329 | 2093 | ||
BiLSTM-GPR | 0.963 | 30.31 | 2.84 | 0.97 | 0.15 | 0.15 | 0.0412 | 2356 | |||
BiRNN-GPR | 0.899 | 48.54 | 5.22 | 1 | 0.24 | 0.24 | 0.0504 | 1769 | |||
Biogas | BiGRU-GPR | 0.939 | 169.32 | 4.66 | 0.9 | 0.24 | 0.27 | 0.0988 | 1953 | ||
BiLSTM-GPR | 0.927 | 184.89 | 5.26 | 0.97 | 0.31 | 0.31 | 0.110 | 2238 | |||
BiRNN-GPR | 0.842 | 275.37 | 8.06 | 0.93 | 0.39 | 0.42 | 0.186 | 1641 |
Notes: *, The unit for RMSE of CODeff and biogas production is mg/L and m3/d, respectively. |
WWTPs | Wastewater Treatment Plants |
SVM | Support Vector Machine |
RF | Random Forest |
GPR | Gaussian Process Regression |
ANN | Artificial Neural Network |
RNN | Recurrent Neural Network |
LSTM | Long Short-Time Memory |
GRU | Gated Recurrent Unit |
BiGRU | Bidirectional Gated Recurrent Unit |
TPE | Tree-structured Parzen Estimator |
RS | Random Search |
GS | Grid Search |
RBF | Radial Basis Function |
SE | Square Exponential Covariance function |
RQ | Rational Quadratic |
ALK | Alkalinity |
OLR | Organic Loading Rate |
HRT | Hydraulic Retention Time |
COD | Chemical Oxygen Demand |
MSE | Mean Square Error |
RMSE | Root Mean Square Error |
MAPE | Mean Absolute Percentage Error |
CP | Coverage Percentage |
MWP | Mean Width Percentage |
CRPS | Continuous Ranked Probability Score |
CDF | Cumulative Distribution Function |
Probability Density Function | |
PIT | Probability Integral Transform |
[1] |
Chen Q L, Chai W, Qiao J F. (2010). Modeling of Wastewater Treatment Process Using Recurrent Neural Network. Jinan: IEEE,
|
[2] |
Ching P M L, So R H Y, Morck T. (2021). Advances in soft sensors for wastewater treatment plants: a systematic review. Journal of Water Process Engineering, 44: 102367
CrossRef
Google scholar
|
[3] |
Darvishi H, Ciuonzo D, Eide E R, Rossi P S. (2021). Sensor-fault detection, isolation and accommodation for digital twins via modular data-driven architecture. IEEE Sensors Journal, 21(4): 4827–4838
CrossRef
Google scholar
|
[4] |
Di Maria F, Micale C. (2015). The contribution to energy production of the aerobic bioconversion of organic waste by an organic Rankine cycle in an integrated anaerobic-aerobic facility. Renewable Energy, 81: 770–778
CrossRef
Google scholar
|
[5] |
Ferro C A T. (2014). Fair scores for ensemble forecasts. Quarterly Journal of the Royal Meteorological Society, 140(683): 1917–1923
CrossRef
Google scholar
|
[6] |
Han H G, Zhang J C, Du S L, Sun H Y, Qiao J F. (2021). Robust optimal control for anaerobic-anoxic-oxic reactors. Science China. Technological Sciences, 64(7): 1485–1499
CrossRef
Google scholar
|
[7] |
HauckM, Maalcke-Luesken F A, JettenM S M, HuijbregtsM A J (2016). Removing nitrogen from wastewater with side stream anammox: What are the trade-offs between environmental impacts? Resources, Conservation and Recycling, 107: 212–219
CrossRef
Google scholar
|
[8] |
Heydari B, Sharghi E A, Rafiee S, Mohtasebi S S. (2021). Use of artificial neural network and adaptive neuro-fuzzy inference system for prediction of biogas production from spearmint essential oil wastewater treatment in up-flow anaerobic sludge blanket reactor. Fuel, 306: 121734
CrossRef
Google scholar
|
[9] |
Hochreiter S, Schmidhuber J. (1997). Long short-term memory. Neural Computation, 9(8): 1735–1780
CrossRef
Pubmed
Google scholar
|
[10] |
Jiang Y, Yin S, Dong J, Kaynak O. (2021). A review on soft sensors for monitoring, control, and optimization of industrial processes, control, and optimization of industrial processes. IEEE Sensors Journal, 21(11): 12868–12881
CrossRef
Google scholar
|
[11] |
Jupp P E, Kume A. (2020). Measures of goodness of fit obtained by almost-canonical transformations on Riemannian manifolds. Journal of Multivariate Analysis, 176: 104579
CrossRef
Google scholar
|
[12] |
Kadlec P, Gabrys B, Strandt S. (2009). Data-driven Soft Sensors in the process industry. Computers & Chemical Engineering, 33(4): 795–814
CrossRef
Google scholar
|
[13] |
Kang L, Chen R S, Xiong N, Chen Y C, Hu Y X, Chen C M. (2019). Selecting hyper-parameters of gaussian process regression based on non-inertial particle swarm optimization in Internet of things. IEEE Access: Practical Innovations, Open Solutions, 7: 59504–59513
CrossRef
Google scholar
|
[14] |
Kim M, Yang Y N, Morikawa-Sakura M S, Wang Q H, Lee M V, Lee D Y, Feng C P, Zhou Y L, Zhang Z Y. (2012). Hydrogen production by anaerobic co-digestion of rice straw and sewage sludge. International Journal of Hydrogen Energy, 37(4): 3142–3149
CrossRef
Google scholar
|
[15] |
Laio F, Tamea S. (2007). Verification tools for probabilistic forecasts of continuous hydrological variables. Hydrology and Earth System Sciences, 11(4): 1267–1277
CrossRef
Google scholar
|
[16] |
Li X Y, Yi X H, Liu Z H, Liu H B, Chen T, Niu G Q, Yan B, Chen C, Huang M Z, Ying G G. (2021). Application of novel hybrid deep leaning model for cleaner production in a paper industrial wastewater treatment system. Journal of Cleaner Production, 294: 126343
CrossRef
Google scholar
|
[17] |
Newhart K B, Holloway R W, Hering A S, Cath T Y. (2019). Data-driven performance analyses of wastewater treatment plants: a review. Water Research, 157: 498–513
CrossRef
Pubmed
Google scholar
|
[18] |
Nguyen H P, Liu J, Zio E. (2020). A long-term prediction approach based on long short-term memory neural networks with automatic parameter optimization by Tree-structured Parzen Estimator and applied to time-series data of NPP steam generators. Applied Soft Computing, 89: 106116
CrossRef
Google scholar
|
[19] |
Ozcan G, Pajovic M, Sahinoglu Z, Wang Y B, Orlik P V, Wada T. (2016). Online State of Charge Estimation for Lithium-Ion Batteries Using Gaussian Process Regression. Florence: IEEE,
|
[20] |
Pham V, Bluche T, Kermorvant C, Louradour J. (2014). Dropout Improves Recurrent Neural Networks for Handwriting Recognition. Hersonissos, Greece: IEEE,
|
[21] |
PutatundaS, Rama K, Acm (2018). A Comparative Analysis of Hyperopt as Against Other Approaches for Hyper-Parameter Optimization of XGBoost. Shanghai: ACM
|
[22] |
Qiao S, Wang Q, Zhang J, Pei Z. (2020). Detection and classification of early decay on blueberry based on improved deep residual 3D convolutional neural network in hyperspectral images. Scientific Programming, 2020: 1–12
CrossRef
Google scholar
|
[23] |
Safari M a M, Masseran N, Majid M H A. (2020). Robust reliability estimation for lindley distribution: a probability integral transform statistical approach. Mathematics, 8(9): 1634
CrossRef
Google scholar
|
[24] |
Samuelsson O, Björk A, Zambrano J, Carlsson B. (2017). Gaussian process regression for monitoring and fault detection of wastewater treatment processes. Water Science and Technology, 75(12): 2952–2963
CrossRef
Pubmed
Google scholar
|
[25] |
Şenol H. (2021). Methane yield prediction of ultrasonic pretreated sewage sludge by means of an artificial neural network. Energy, 215: 119173
CrossRef
Google scholar
|
[26] |
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15: 1929–1958
|
[27] |
Szelag B, Gawdzik A, Gawdzik A. (2017). Application of selected methods of black box for modelling the settleability process in wastewater treatment plant. Ecological Chemistry and Engineering S-Chemia I Inzynieria Ekologiczna S, 24(1): 119–127
|
[28] |
Wang H T, Yang Y, Keller A A, Li X, Feng S J, Dong Y N, Li F T. (2016). Comparative analysis of energy intensity and carbon emissions in wastewater treatment in USA, Germany, China and South Africa. Applied Energy, 184: 873–881
CrossRef
Google scholar
|
[29] |
Wang J, Cui Q, Sun X. (2021). A novel framework for carbon price prediction using comprehensive feature screening, bidirectional gate recurrent unit and Gaussian process regression. Journal of Cleaner Production, 314: 128024
CrossRef
Google scholar
|
[30] |
Wei J P, Liang G F, Alex J, Zhang T C, Ma C B. (2020). Research progress of energy utilization of agricultural waste in China: Bibliometric analysis by citespace. Sustainability (Basel), 12(3): 812
CrossRef
Google scholar
|
[31] |
Wu X, Wang Y, Wang C, Wang W, Dong F. (2021). Moving average convergence and divergence indexes based online intelligent expert diagnosis system for anaerobic wastewater treatment process. Bioresource Technology, 324: 124662
CrossRef
Pubmed
Google scholar
|
[32] |
Xu Y, Gao W, Qian F, Li Y. (2021). Potential analysis of the attention-based LSTM model in ultra-short-term forecasting of building HVAC energy consumption. Frontiers in Energy Research, 9: 730640
CrossRef
Google scholar
|
[33] |
Yaginuma K, Tanabe S, Kano M. (2022). Gray-box soft sensor for water content monitoring in fluidized bed granulation. Chemical & Pharmaceutical Bulletin, 70(1): 74–81
CrossRef
Pubmed
Google scholar
|
[34] |
Zeng G M, Li X D, Jiang R, Li J B, Huang G H. (2006). Fault diagnosis of WWTP based on improved support vector machine. Environmental Engineering Science, 23(6): 1044–1054
CrossRef
Google scholar
|
[35] |
Zhang C, Wei H, Zhao X, Liu T, Zhang K. (2016). A Gaussian process regression based hybrid approach for short-term wind speed prediction. Energy Conversion and Management, 126: 1084–1092
CrossRef
Google scholar
|
[36] |
Zhang Z, Ye L, Qin H, Liu Y, Wang C, Yu X, Yin X, Li J. (2019). Wind speed prediction method using shared weight long short-term memory network and gaussian process regression. Applied Energy, 247: 270–284
CrossRef
Google scholar
|
Supplementary files
FSE-22097-OF-LJL_suppl_1 (221 KB)
Part of a collection:
Artificial Intelligence/Machine Learning on Environmental Science & Engineering
/
〈 | 〉 |