Prediction and cause investigation of ozone based on a double-stage attention mechanism recurrent neural network

Abstract

● Used a double-stage attention mechanism model to predict ozone.

● The model can autonomously select the appropriate time series for forecasting.

● The model outperforms other machine learning models and WRF-CMAQ.

● We used the model to analyze the driving factors of VOCs that cause ozone pollution.

Ozone is becoming a significant air pollutant in some regions, and VOCs are essential for ozone prediction as necessary ozone precursors. In this study, we proposed a recurrent neural network based on a double-stage attention mechanism model to predict ozone, selected an appropriate time series for prediction through the input attention and temporal attention mechanisms, and analyzed the cause of ozone generation according to the contribution of feature parameters. The experimental data show that our model had an RMSE of 7.71 μg/m³ and a mean absolute error of 5.97 μg/m³ for 1-h predictions. The DA-RNN model predicted ozone closer to observations than the other models. Based on the importance of the characteristics, we found that the ozone pollution in the Jinshan Industrial Zone mainly comes from the emissions of petrochemical enterprises, and the good generalization performance of the model is proved through testing multiple stations. Our experimental results demonstrate the validity and promising application of the DA-RNN model in predicting atmospheric pollutants and investigating their causes.

Graphical abstract

Keywords

Ozone prediction / Deep learning / Time series / Attention / Volatile organic compounds

1 Introduction

The emission of atmospheric pollutants is gradually increasing with the acceleration of industrialization and urbanization, and society is becoming increasingly concerned with environmental issues (Hui et al., 2018). Air pollution has evolved into a more complex type of pollution with a high ozone content. Ozone, as an air quality index (AQI) contaminant, has become one of the most serious environmental pollution issues in many cities and regions (Ou et al., 2015).

As it is more common in East China, especially during summer, ozone has become the primary air pollutant in urban areas (Wang et al., 2018). High levels of ozone have negative effects on human health, climate change, and the ecological balance (Kampa and Castanas, 2008). Volatile organic compounds (VOCs) and NO_x are major precursors. They catalyze the conversion of oxygen to ozone, which is affected by several atmospheric variables such as temperature, humidity, radiation intensity, particulate matter concentration, and other unclear factors (Avery, 2006). Each VOC can play a different role during ozone formation. The complexity of VOCs inevitably makes this situation extremely complicated and difficult to predict. The establishment of a reliable ozone forecasting model, based on the non-linear relationship between the affecting components and conditions, will be critically important to predict its formation and set up strategies for its elimination (Shao et al., 2009).

The establishment of ozone prediction systems is mainly based on two types of conventional methods: chemistry-transport models (CTMs) and statistical methods (Donnelly et al., 2015). CTMs commonly use the method of solving mathematical equations to numerically calculate the chemical and physical processes of pollutant emission, diffusion, and transformation, and obtain the fitting results of the relationship between the atmosphere and emissions (Vautard et al., 2007). Generally, there are three CTMs: the Community Multiscale Air Quality (CMAQ) model (Byun and Schere, 2006), the Weather Research and Forecasting model coupled with Chemistry (WRF-Chem) (Zhang et al., 2010), and the Nested Air Quality Prediction Modeling System (NAQPMS) (Ge et al., 2014). CTMs have the merits of accuracy of the emission inventory and meteorological data and completeness of the simulated chemical mechanism (Zhang et al., 2011). It often takes a long time to perform preliminary preparations and calculation simulations to obtain accurate results (Liu et al., 2021). Frequently used statistical methods include the autoregressive moving average (ARMA) model (Erdem and Shi, 2011), autoregressive integrated moving average (ARIMA) model (Kavasseri and Seetharaman, 2009), multiple linear regression (MLR) (del Carmen Bas et al., 2017), and geographically weighted regression (GWR) (Wang et al., 2014). Statistical methods are limited to capturing the complex nonlinear relationship between air quality data in a time series regardless of high-dimensional data (Delle Monache et al., 2006). Meanwhile, both types of methods struggle to solve the problem of dimension disaster owing to the lack of a flexible multi-scale framework, resulting in poor prediction performance (Garaud and Mallet, 2011). Recently, machine learning techniques have become the standard for researchers to solve problems. For example, Chen et al. used a combined representation learning algorithm for the super-resolution reconstruction of face images (Chen et al., 2021). Xia et al. (2022) used an improved algorithm to reduce occlusion interference during target tracking and improve the object recognition accuracy. Zhang et al. (2022a) used a multi-feature fusion algorithm to achieve a better success rate and accuracy than other trackers. Therefore, some researchers have begun using machine learning models to solve the problem of forecasting air pollutants. For example, Hu et al. (2017) estimated ground-level PM_2.5 concentrations in the United States using a random forest model. Ding et al. (2021) used the CatBoost model to reconstruct satellite aerosol optical depth (AOD) data. An object tracking framework with recapture based on correlation filters and Siamese networks developed by Zhang et al. (2022b) has application prospects in the visual detection of environmental pollutants. Wong et al. (2021) used an XGBoost-based boosting algorithm to predict NO₂ concentrations.

Deep learning technology has been developed owing to its ability to extract features from large amounts of data using neural networks; this process has the potential to address the limitations of conventional air quality prediction methods (Molina-Gomez et al., 2021). Ozone concentration prediction can be genuinely recognized as a multidimensional time-series prediction problem from the perspective of deep learning (Jia et al., 2021). According to the complexity of the multi-dimensional data series and the spatiotemporal characteristics of deep learning (Xu et al., 2021), the main structures suitable for time series forecasting include the recurrent neural network (RNN) (Qin et al., 2017), long short-term memory (LSTM) network (Wang et al., 2021), and gated recurrent unit (GRU) network (Tao et al., 2019). Based on these structures, corresponding models can extract spatiotemporal characteristics and apply them to air quality prediction (Reichstein et al., 2019). However, atmospheric pollution is a complex and long-time scale problem. These structures hardly capture long-time scale correlations and identify potential nonlinear relationships to improve prediction accuracy (Bengio et al., 1994).

In this study, we used a neural network model based on a double-stage attention mechanism recurrent neural network (DA-RNN) to predict ozone concentration. This model appropriately captures the long-term temporal dependence based on the previous values of the multi-dimensional time series and multiple driving sequences, and selects the relevant driving sequence for prediction. The importance and relevance of all characteristics to ozone were analyzed, and multiple evaluation standards were selected to assess the accuracy of the model prediction. The performance of the optimized DA-RNN model was compared with that of the CMAQ for the final test and verification. The main contributions of this study are as follows: 1) The DA-RNN model was used to accurately predict the ozone concentration in the Jinshan Industrial Zone. 2) Based on the importance of these features, we found that petrochemical enterprises were the main causes of ozone pollution in the Jinshan Industrial Zone. 3) The model has a good generalization performance and good application prospects in the field of ozone prediction.

2 Study station and dataset

2.1 Research station and monitoring data

The selected station in this study was Jinshan station, which is a representative atmospheric monitoring site in the Shanghai Chemical Industry District. Shanghai, a chemical industry district, is located near the southern border of Shanghai, north of Hangzhou Bay, and close to the East China Sea. With a planned area of 29.4 km² and a management area of 36.1 km², it ranges within the Jinshan and Feng Xian districts. Jinshan is located near a chemical engineering park and has high research value. Therefore, except for ozone, significant precursors, including hourly monitoring data for NO_x, CO, PM₁₀, PM_2.5, and VOCs, were identified. Weather conditions, including wind speed, temperature, humidity, and radiation intensity, which could promote ozone formation, were also recorded. To study the model’s generalization performance, we also studied the Jinshan industrial district station and Jinshan boundary station, using the same data categories as for the Jinshan station.

All data were obtained from the Shanghai Jinshan Environmental Monitoring Center. Fig.7 shows scatter plots of hourly ozone concentration values versus NO_x, CO, PM₁₀, PM_2.5, wind speed, temperature, humidity, and radiation intensity; there was a significant non-linear relationship between the variables and ozone concentration. The data were used as training and test sets for the DA-RNN model.

2.2 Data preprocessing

The dataset used in this study contained outliers removed from the original records. However, the maintenance of equipment at a station can result in missing values. Therefore, in order to make the dataset as realistic as possible, the missing data must be filled in. Furthermore, it is well known that the variation in atmospheric pollutant concentrations can be considered a continuous, non-discrete process. Therefore, we can choose linear interpolation to complete the missing data. The linear interpolation equation is as follows:

(1)y=y_0+\frac{y_1-y_0}{x_1-x_0}\left(x-x_0\right),

where (x₀, y₀) and (x₁, y₁) are the coordinates of the two points before and after the value to be filled, and (x, y) are the coordinates of the points that need to be supplemented.

In total, 8760 records were shared in the final dataset. Seventy percent of the data were used as the training set and the remaining 30 % as the testing set to evaluate the prediction performance of the model.

3 Method

In this study, the model was divided into two parts based on the input attention and temporal attention mechanisms. In the first stage, a new input attention mechanism that can automatically select a suitable time series was introduced. In the second stage, we used a temporal attention mechanism to select the relevant encoder hidden states across all time steps. Through these two attention mechanisms, the DA-RNN can adaptively select the most relevant input features and capture the long-term temporal dependence of the time series. A flowchart of the DA-RNN model is shown in Fig.2. The details of the double-stage attention mechanism recurrent neural network are described in .

3.1 Input attention mechanism

The input attention mechanism captures dependencies on a multidimensional time series. For the time series, X=\left(x_1,x_2,\dots ,x_T\right) with x_t\in X. There is mapping from x_t to h_t (at time step t) with

(2)h_t= f\left(h_{t-1},x_t\right),

whereh_t is the hidden state of the input attention layer at time t and f is the non-linear activation function of the LSTM. Each LSTM is controlled by the control unit C_t at timet. Access to the control unit is controlled by three S-shaped doors: the forget gate f_t, input gate i_t, and output gate o_t. The status update of LSTM can be represented by the following equations:

(3)f_t= \sigma \left(W_f\cdot \left[h_{t-1};x_t\right]+b_f\right),

(4)i_t= \sigma \left(W_i\cdot \left[h_{t-1};x_t\right]+b_i\right),

(5){\tilde{C}}_t=\mathrm{t}\mathrm{a}\mathrm{n}h\left(W_C\cdot \left[h_{t-1};x_t\right]+b_C\right),

(6)C_t= f_t\ast C_{t-1}+i_t\ast {\stackrel{}{\tilde{C}}}_t,

(7)o_t= \sigma \left(W_o\cdot \left[h_{t-1};x_t\right]+b_o\right),

(8)h_t= o_t\ast \mathrm{t} \mathrm{a}\mathrm{n}h\left(C_t\right),

where \left[h_{t-1};x_t\right] is the gate that reads the last hidden state h_{t-1} and the current input x_t. W_f, W_i, W_C, \mathrm{a}\mathrm{n}\mathrm{d} W_o, and b_f, b_i, b_C, \mathrm{a}\mathrm{n}\mathrm{d} b_o are the parameters that the model must learn. \sigma is the sigmoid function that is used as the activation function of the neural network to map variables between 0 and 1. There is one memory cell per LSTM cell with state C_t at time t. {\tilde{C}}_t represents the cell state update value obtained from input data x_t and hidden state h_{t-1}. The reason for choosing LSTM is that it can overcome the problem of gradient disappearance caused by a long time, to better capture the correlation of time series over a long time span.

In this structure, we use an algorithm based on input attention, which can adaptively select the relevant time series, given the k-th input time series x^k=\left(x_1^k,x_2^k,\dots ,x_T^k\right), by reading the previous hidden state h_{t-1} and unit state C_{t-1} in the LSTM cell; thus, we can construct the input attention mechanism through the model:

(9)e_t^k= v_e^T\mathrm{t}\mathrm{a}\mathrm{n}h\left(W_e\cdot \left[h_{t-1};C_{t-1}\right]+U_e{x}^k\right),

(10){\alpha }_t^k= \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(e_t^k\right)}{{\sum }_{t=1}^n\mathrm{e}\mathrm{x}\mathrm{p}\left(e_t^i\right)},

where v_e, U_e, and W_e are the parameters that the model must learn. e_t^k is the attention weight of the k-th input sequence at time t. {\alpha }_t^k is the attention weight, which measures the importance of the k-th input time series at time t. In Eq. (10), we use the softmax function to ensure that the sum of all the attention weights is 1. Using these attention weights, the driving sequence can be extracted adaptively.

According to this structure, we obtain a time series with attention by combining the attention weights:

(11)\widetilde{x_t}=\begin{array}{c}{(\alpha }^1{x}_t^1\end{array},{\alpha }^2{x}_t^2,\dots ,{\alpha }^n{x}_t^n).

According to the obtained \widetilde{x_t}, h_t is updated as follows:

(12)h_t= f\left(h_{t-1},\widetilde{x_t}\right).

Therefore, in the input attention mechanism, the relevant time series can be selected according to the attention weight for the subsequent training.

3.2 Temporal attention mechanism

To make accurate predictions, we introduced an LSTM structure based on an RNN in the temporal attention mechanism. As the length of the time series increases, the performance of the model decreases rapidly. Therefore, we used a temporal attention mechanism to adaptively select the appropriate hidden state at the time step. By reading the previous hidden state h_{t-1} and unit state C_{t-1} in the LSTM cell, we constructed a temporal attention mechanism using the following model:

(13)l_t^i= {\mathrm{v}}_d^T\mathrm{t} \mathrm{a}\mathrm{n}h\left(W_d\right[h_{t-1};C_{t-1}]+U_d{h}_i),

(14){\beta }_t^i= \frac{\mathrm{e}\mathrm{x}\mathrm{p}\left(l_t^i\right)}{{\sum }_{j=1}^T\mathrm{e}\mathrm{x}\mathrm{p}{\left(l_t^j\right)}^{\text{'}}},

where v_d, W_d, and U_d are the parameters that the model must learn. l_t^i is the temporal attention weight of the i-th input sequence at time t. {\beta }_t^i represents the importance of the i-th encoder hidden state for prediction. In eq. (14), we use the softmax function to ensure that the sum of all the temporal attention weights is 1.

Then, we obtain the context vector c_t according to the time attention weight {\beta }_t^i and all hidden state sequences \left(h_1,h_2,\dots ,h_T\right):

(15)c_t=\sum _{i=1}^T{\beta }_t^i{h}_i.

Then, according to the given target sequence (y_1,y_2,\dots , y_T), combined with the context vector, the predicted sequence can be obtained as follows:

(16)y_t=w [y_{(t-1)};c_{(t-1)}]+b

where [y_{t-1};c_{t-1}] is a concatenation of the target sequence y_{t-1} and the context vector c_{t-1}. The parameters w and b were obtained by training the model. The calculated y_t can be used to update the hidden state h_t at time t as follows:

(17)h_t= f\left(h_{t-1},y_t\right).

The newly obtained hidden state h_t is used for the next prediction, and the prediction sequence is finally obtained. f is an LSTM cell that can be computed using Eqs. (3)–(8).

The input sequence of the model was the concentration of various pollutants and meteorological parameters per hour; the time step was set to 24, and the output sequence was the ozone concentration within the predicted time period.

3.3 Comparison with other machine learning models and WRF-CMAQ models

Machine learning models have been widely used to predict air pollutant concentrations. To prove the prediction performance of the model used in this study, we selected CatBoost, LightGBM, LightGBMLarge, XGBoost, Random Forest, ExtraTrees, NeuralNet, KNeighbors, and several common machine learning models for comparison.

Traditional chemistry-transport models are still widely used, so we also compared the WRF-CMAQ model, including the Weather Research and Forecasting Model (WRF; version 3.7), Sparse Matrix Operator Kernel Emission System (SMOKE; version 3.7), Model of Emissions of Gases and Aerosols from Nature (MEGAN; version 3), and Community Multi-scale Air Quality Modeling System (CMAQ; version 5.3.2).

Three levels of nesting were adopted for the simulation area of the model. The center latitude and longitude was 120.3 °E, 33.7 °N; we used the Lambert projection; the horizontal resolution of the three levels of nesting was 36 km, 12 km, and 4 km, respectively. The grid numbers were 185 × 148, 147 × 240, and 204 × 228. The model adopted the Yonsei University (YSU) scheme to parameterize the boundary-layer process (Hong et al., 2006). The parameterizing interaction of the land-atmosphere was generated using the Noah land surface scheme (Ek et al., 2003). The Purdue-Lin microphysical scheme was used to reproduce cloud and precipitation processes (Efstathiou et al., 2013). Meteorological input data were obtained from the meteorological reanalysis data (FNL) of the National Center for Environmental Prediction (NCEP) with a grid resolution of 1° × 1° and a time resolution of 6 h. The emission inventory was obtained from the 2017 National Air Pollution Source Emission Inventory of Tsinghua University (MEIC). On this basis, RRTM long-wave (Chin et al., 2002) and Goddard shortwave (Mlawer et al., 1997) radiation schemes were used to reflect the radiation situation.

3.4 Model evaluation method

To measure the effectiveness of the time-series forecasting method, three distinct evaluation indicators were used to assess the differences between the predicted and observed values. The three evaluation indicators were the mean absolute error (MAE), root mean square error (RMSE), and Pearson correlation coefficient (PMC).

The MAE is the sum of the absolute values of the difference between the target and predicted values. The calculation formula is as follows:

(18)MAE=\frac{1}{m}\sum _{i=1}^m\left|\right(y_i-{\hat{y}}_i\left)\right|,

where y_i is the value of the i-th forecast, {\hat{y}}_i is the value of the i-th observation, and m is the total amount of data.

The RMSE was used to measure the error between the target and predicted values. The calculation formula is as follows:

(19)RMSE= \sqrt{\frac{1}{m}\sum _{i=1}^m{(y_i-{\hat{y}}_i)}^2},

where y_i is the value of the i-th forecast, {\hat{y}}_i is the value of the i-th observation, and m is the total amount of data. The lower the MAE and RMSE, the higher the prediction accuracy of the model.

The Pearson correlation coefficient is generally used in the feature selection stage before training to determine the linear relationship between the features. This is calculated as follows:

(20)PMC=\frac{\mathrm{C}\mathrm{o}\mathrm{v}(X,Y)}{\sqrt{D\left(X\right)}\sqrt{D\left(Y\right)}},

where X,Y are two variables that require correlation coefficients, \mathrm{C} \mathrm{o}\mathrm{v}(X,Y) is the covariance of X and Y; D\left(X\right) is the variance of X, and D\left(Y\right) is the variance of Y.

4 Results and discussion

To cover most factors affecting ozone generation, we selected hourly monitoring data, including reacting substances (NO_x, CO, PM₁₀, PM_2.5, and typical VOCs), resulting ozone concentrations, and weather conditions (wind speed, temperature, humidity, and radiation intensity) from the Jinshan station in 2020. The Photochemical Assessment Monitoring Station (PAMS) program introduced 57 species of VOCs, which are prone to photochemical reactions for easy detection. Accordingly, 33 VOC species were selected: ethane, propane, n-butane, isobutane, n-pentane, isopentane, n-hexane, n-heptane, n-octane, 2,2,4-trimethyl pentane, 2,3,4-trimethyl pentane, ethylene, acetylene, propylene, cis-2-butene, trans-2-butene, 1,3-butadiene, isoprene, benzene, toluene, ethyl benzene, ortho-xylene, styrene, 1,2,3-trimethylbenzene, 1,2,4-trimethylbenzene, 1,3,5-trimethylbenzene, cumene, n-propylbenzene, 1,1-dichloroethane, 1,2-dichloroethane, trichloroethylene, perchloroethylene, chlorobenzene, and 1,2,4-trichlorobenzene.

4.1 Analysis of feature correlation

In essence, the machine learning model completes the prediction by mining the correlation between the features and the predicted values. If a certain set of features and predicted values are completely independent, then it is useless for modeling. Therefore, it was necessary to verify the correlation prior to training. In this study, characteristic correlation analysis enabled us to understand the correlation between different pollutants in Jinshan. The Pearson correlation heat map shown in Fig.3 proves a good non-linear relationship between the selected species, which provides theoretical support for follow-up work.

4.2 Analysis of feature importance

In machine learning, it is essential to analyze the importance of the features in the training model to improve the generalization ability of the model and the understanding of model features. Based on this model, the importance of features can help us understand the relative contributions of different species and conditions to ozone formation in Jinshan.

Fig.4 shows a graph of the importance of weights of different features for the ozone concentration. For convenience, we divided VOCs into alkanes, olefins, aromatic compounds, and chlorinated compounds. As shown in Fig.4(a), VOCs, as a large category of features, had the highest impact on ozone, indicating that changes in the concentration of VOCs had the greatest impact on ozone. Therefore, we can infer that the Jinshan station is in a VOC-controlled area. The Shanghai chemical industry zone is dominated by petrochemical and other chemical plants, and the main pollutants emitted by the petrochemical industry are alkanes and aromatic compounds, which accounted for more than 50 % of the importance of VOCs. Developed industries are often accompanied by massive energy consumption and transportation needs; therefore, olefins and chlorides also account for an essential portion of the critical weight of VOCs. At the same time, the impact of radiation intensity on ozone also has a higher weight because radiation can accelerate photochemical reactions and the production of ozone (Atkinson, 2000). As NO_x plays a similar catalytic role in the formation of ozone, the amount of NO_x directly affects the ozone yield. Fig.4(b) shows that alkanes and aromatic hydrocarbons, as the two species with the highest percentages, have a combined importance of over 50 %, which indicates that ozone pollution in the Jinshan Industrial District is mainly influenced by VOCs from petrochemical enterprises and vehicle exhaust emissions. Fig. S1 shows the importance of each of the 33 VOC species.

4.3 Predictive performance of the model

We used an RTX3060 graphics card for model training. We set the batch size to 128, epochs to 300, and learning rate to 0.001. The software environment of the model was Python 3.8, Pytorch 1.8.1, and CUDA 11.1. The total training time was 900 s.

We used the model to predict the ozone concentration at 1, 2, 3, 6, 12, and 24 h on the test set. Both MAE and RMSE increased with an increase in forecast duration (Tab.1). The MAE was between 5.97 μg/m³ and 14.71 μg/m³; the RMSE was between 7.71 μg/m³ and 20.57 μg/m³. Accordingly, R² gradually deviated from 0.958 to 0.699. In conclusion, the model indicated a high prediction accuracy in 1 h. Although the prediction performance gradually weakened, it was still possible to accurately predict the ozone concentration within a period of time.

Fig.5 shows the scatter plots for different forecast durations that illustrate the correlation between the forecast and observed values. The dashed line represents the perfect case, where the predicted value is equal to the observed value, the color represents the density of points, and the model’s prediction performance gradually decreases as the prediction duration increases. However, at 24 h, there was still a strong correlation between the predicted and observed values.

4.4 Performance of the model at low and high ozone values

To verify the performance of the model in predicting low and high values, we selected July and August as the typical months with high ozone values and November and December as the typical months with low ozone values. In Shanghai, ozone pollution incidents occur frequently, especially during summer when the peak ozone concentration exceeds 250 μg/m³. In winter, the ozone concentration is low because of the weak sunlight and low temperature. Fig.6 shows a comparison between the model’s predicted hourly ozone concentration and the observed values in July, August, November, and December 2020. The predicted values of the ozone concentration in most time periods were comparable to the observed values. The RMSE values for July, August, November, and December were 11.72 μg/m³, 8.91 μg/m³, 9.04 μg/m³, and 7.85 μg/m³, respectively. The results show that the DA-RNN model can be used to predict ozone concentration over a short period of time.

However, as shown in Fig.6, Jinshan Station had ozone peaks exceeding 250 μg/m³ in some time periods in July and August 2020. For example, at 16:00 on July 3, 2020, the observed value was 281.46 μg/m³ and the predicted value was 196.37 μg/m³. At 18:00 on August 24, 2020, the observed value was 262.43 μg/m³ and the predicted value was 192.48 μg/m³. As shown in Fig.6, during the period when the ozone concentration was low, the prediction performance of the model was better. Therefore, the model can make accurate predictions for low ozone values but is not suitable for prediction at extremely high ozone values. Extremely high values appear less frequently in the training set, which could account for the model’s impaired performance in identifying the characteristics at high ozone concentrations.

Fig.7 shows the prediction performance of the DA-RNN model at different sites. For example, the RMSE (μg/m³), MAE (μg/m³), and R² values of the Jinshan Industrial District Station were 9.49, 6.63, and 0.95, respectively; the corresponding values of the Jinshan Boundary Station were 10.91, 8.39, and 0.93, respectively. This shows that the DA-RNN model has good generalization performance.

4.5 Comparing results with other models

Tab.2 presents a comparison of the 1 h prediction performance of the DA-RAN model and other machine learning models. The DA-RNN model shows a significant improvement in the evaluation indicators of R², MAE, and RMSE compared with the other models. We speculate that this is due to the excellent ability of the DA-RNN model to capture multidimensional time-series features, and its performance on training sets with more dimensions is significantly better than that of other models.

We compared the prediction results of the DA-RNN model with those of the WRF-CMAQ model. As the simulation results of WRF-CMAQ are time-consuming, we chose only two representative months, July and November, 2020. We selected the grid points closest to Jinshan station and used the ozone concentration data at the grid points as the predicted values of the model. As shown in Fig.8, the ozone concentration predicted by the DA-RNN model was closer to the observed value than the WRF-CMAQ model in most time periods, and the ozone concentration predicted by the WRF-CMAQ model was generally lower than the observed value. For extremely high values, neither the DA-RNN model nor the WRF-CMAQ model could predict very well, which was caused by too few extremely high-value samples.

Tab.3 lists the prediction performance of the WRF-CMAQ and DA-RNN models. Both models have better predictions for low-value time periods, and the prediction performance of DA-RNN is better than that of WRF-CMAQ. Owing to the impact of the accuracy of the emission inventory, the accuracy of the meteorological data, and the completeness of the simulated chemical mechanism, WRF-CMAQ cannot predict the ozone concentration well, and it takes a long time to calculate. Compared with WRF-CMAQ, DA-RNN can predict future ozone concentration by learning the existing data samples to determine the dependencies and importance weights of features, which can efficiently and conveniently achieve good prediction results.

5 Conclusions

Currently, ozone has surpassed PM_2.5 and is the primary pollutant in some high-incidence seasons. Therefore, it is important to conduct real-time forecasts and ozone monitoring and study the direct synergistic relationship between ozone precursors and ozone reduction. In this study, we used a double-stage attention-based recurrent neural network (DA-RNN) to monitor and forecast ozone in real time and analyze the importance of these features. This model appropriately captures the long-term time dependence based on the previous value of the time series and multiple time series and selects the relatively important time series for prediction.

We calculated the results of different prediction periods, RMSE, MAE, and R² to assess the predictive performance of the model. The results showed that the short-term prediction results of the DA-RNN model were more accurate. Within the most precise prediction time (1 h), the values of MAE, RMSE, and R² were 5.97 μg/m³, 7.71 μg/m³, and 0.958, respectively. The model’s accuracy decreased as the forecast time increased but still had good accuracy in the 24-h forecast, with MAE, RMSE, and R² values of 14.71 μg/m³, 20.57 μg/m³, and 0.70, respectively. By analyzing the importance of the characteristics of the Jinshan Industrial Zone, we found that ozone pollution in this site comes from other ozone precursors such as VOCs in the petrochemical industry. We explored the relationship between the concentrations of precursors and ozone based on the usual ozone prediction. We predicted the relationship between the concentration change of ozone and the evolution of the precursor concentration. Therefore, our model can effectively help environmental protection departments monitor and provide early warnings of ozone pollution and take targeted preventive measures.

To verify the model’s prediction performance, we compared the DA-RNN model with other machine learning models. The DA-RNN model outperformed other machine learning models in 1-h predictions. We also compared the DA-RNN model with the WRF-CMAQ model. In the two representative months we selected, the average RMSE and MAE of the DA-RNN model were 8.63 μg/m³, which was much lower than the 33.70 μg/m³ of WRF-CMAQ. Compared with the WRF-CMAQ model, the DA-RNN model requires less data for prediction, is shorter and more time consuming, and has better versatility. To verify the generalization performance of the model, we selected three different types of stations for the experiments. In the 1-h prediction performance of the three sites, MAE, RMSE, and R² outperformed the other machine learning models. The excellent performance of the DA-RNN model in terms of generalization ability and generality was explained.

Owing to the structure of its deep neural network, the DA-RNN model can identify complex synergistic and nonlinear relationships from the concentration data of different pollutants through learning. Therefore, compared to traditional models, it can predict the concentration of pollutants more efficiently and accurately. However, owing to the limitation of the number of samples in the training set, the model cannot predict a high concentration value. Further research can improve the performance of the model in predicting extremely high values by collecting samples with extremely high values.

References

[1] AtkinsonR ( 2000). Atmospheric chemistry of VOCs and NO _x . Atmospheric Environment, 34( 12– 14): 2063– 2101

[2] AveryR J. (2006). Reactivity-based VOC control for solvent products: more efficient ozone reduction strategies. Environmental Science & Technology, 40( 16): 4845– 4850

[3] BasM D, OrtizJ, BallesterosL, MartorellS. (2017). Evaluation of a multiple linear regression model and SARIMA model in forecasting ⁷Be air concentrations. Chemosphere, 177 : 326– 333

[4] BengioY, SimardP, FrasconiP. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5( 2): 157– 166

[5] ByunD, SchereK L. (2006). Review of the governing equations, computational algorithms, and other components of the Models-3 community multiscale air quality (CMAQ) modeling system. Applied Mechanics Reviews, 59( 2): 51– 77

[6] ChenY, PhonevilayV, TaoJ, ChenX, XiaR, ZhangQ, YangK, XiongJ, XieJ. (2021). The face image super-resolution algorithm based on combined representation learning. Multimedia Tools and Applications, 80( 20): 30839– 30861

[7] ChinM, GinouxP, KinneS, TorresO, HolbenB N, DuncanB N, MartinR V, LoganJ A, HigurashiA, NakajimaT. (2002). Tropospheric aerosol optical thickness from the GOCART model and comparisons with satellite and Sun photometer measurements. Journal of the Atmospheric Sciences, 59( 3): 461– 483

[8] Delle MonacheL, NipenT, DengX X, ZhouY M, StullR. (2006). Ozone ensemble forecasts: 2. A Kalman filter predictor bias correction. Journal of Geophysical Research, 111( D5): D05308

[9] DingY, ChenZ, LuW, WangX. (2021). A CatBoost approach with wavelet decomposition to improve satellite-derived high-resolution PM_2.5 estimates in Beijing-Tianjin-Hebei. Atmospheric Environment, 249 : 118212

[10] DonnellyA, MisstearB, BroderickB. (2015). Real time air quality forecasting using integrated parametric and non-parametric regression techniques. Atmospheric Environment, 103 : 53– 65

[11] EfstathiouG A, ZoumakisN M, MelasD, LolisC J, KassomenosP ( 2013). Sensitivity of WRF to boundary layer parameterizations in simulating a heavy rainfall event using different microphysical schemes : e ffect on large-scale processes. Atmospheric Research, 132– 133: 125– 143

[12] EkM B, MitchellK E, LinY, RogersE, GrunmannP, KorenV, GaynoG, TarpleyJ D. (2003). Implementation of Noah land surface model advances in the national centers for environmental prediction operational mesoscale Eta model. Journal of Geophysical Research, 108( D22): 2002JD003296

[13] ErdemE, ShiJ. (2011). ARMA based approaches for forecasting the tuple of wind speed and direction. Applied Energy, 88( 4): 1405– 1414

[14] GaraudD, MalletV. (2011). Automatic calibration of an ensemble for uncertainty estimation and probabilistic forecast: application to air quality. Journal of Geophysical Research, 116( D19): D19304

[15] GeB Z, WangZ F, XuX B, WuJ B, YuX L, LiJ. (2014). Wet deposition of acidifying substances in different regions of China and the rest of East Asia: modeling with updated NAQPMS. Environmental Pollution, 187 : 10– 21

[16] HongS Y, NohY, DudhiaJ. (2006). A new vertical diffusion package with an explicit treatment of entrainment processes. Monthly Weather Review, 134( 9): 2318– 2341

[17] HuX, BelleJ H, MengX, WildaniA, WallerL A, StricklandM J, LiuY. (2017). Estimating PM_2.5 concentrations in the conterminous United States using the random forest approach. Environmental Science & Technology, 51( 12): 6936– 6944

[18] HuiL, LiuX, TanQ, FengM, AnJ, QuY, ZhangY, JiangM. (2018). Characteristics, source apportionment and contribution of VOCs to ozone formation in Wuhan, Central China. Atmospheric Environment, 192 : 55– 71

[19] JiaP, CaoN, YangS. (2021). Real-time hourly ozone prediction system for Yangtze River Delta area using attention based on a sequence to sequence model. Atmospheric Environment, 244 : 117917

[20] KampaM, CastanasE. (2008). Human health effects of air pollution. Environmental Pollution, 151( 2): 362– 367

[21] KavasseriR G, SeetharamanK. (2009). Day-ahead wind speed forecasting using f-ARIMA models. Renewable Energy, 34( 5): 1388– 1393

[22] LiuC, ZhangH, ChengZ, ShenJ, ZhaoJ, WangY, WangS, ChengY. (2021). Emulation of an atmospheric gas-phase chemistry solver through deep learning: case study of Chinese mainland. Atmospheric Pollution Research, 12( 6): 101079

[23] MlawerE J, TaubmanS J, BrownP D, IaconoM J, CloughS A. (1997). Radiative transfer for inhomogeneous atmospheres: RRTM, a validated correlated-k model for the longwave. Journal of Geophysical Research, 102( D14): 16663– 16682

[24] Molina-GómezN I, Diaz-ArevaloJ L, Lopez-JimenezP A. (2021). Air quality and urban sustainable development: the application of machine learning tools. International Journal of Environmental Science and Technology, 18( 4): 1029– 1046

[25] OuJ, Zheng J, LiR, HuangX, ZhongZ, ZhongL, LinH ( 2015). Speciated OVOC and VOC emission inventories and their implications for reactivity-based ozone control strategy in the Pearl River Delta region, China. Sci ence of the Total Environ ment, 530– 531: 393– 402

[26] QinY, SongD, ChengH, ChengW, JiangG, CottrellG W. (2017). A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction. Melbourne, Australia : 2627– 2633

[27] ReichsteinM, Camps-VallsG, StevensB, JungM, DenzlerJ, CarvalhaisN. (2019). Deep learning and process understanding for data-driven Earth system science. Nature, 566( 7743): 195– 204

[28] ShaoM, ZhangY, ZengL, TangX, ZhangJ, ZhongL, WangB. (2009). Ground-level ozone in the Pearl River Delta and the roles of VOC and NO_x in its production. Journal of Environmental Management, 90( 1): 512– 518

[29] TaoQ Liu F LiY SidorovD ( 2019). Air pollution forecasting using a deep learning model based on 1D convnets and bidirectional GRU. IEEE Access: Practical Innovations, Open Solutions, 7: 76690– 76698

[30] VautardR, BuiltjesP H J, ThunisP, CuvelierC, BedogniM, BessagnetB, HonoreC, MoussiopoulosN, PirovanoG, SchaapM, SternR, TarrasonL, WindP. (2007). Evaluation and intercomparison of ozone and PM₁₀ simulations by several chemistry transport models over four European cities within the CityDelta project. Atmospheric Environment, 41( 1): 173– 188

[31] WangB, YuanQ, YangQ, ZhuL, LiT, ZhangL. (2021). Estimate hourly PM_2.5 concentrations from Himawari-8 TOA reflectance directly using geo-intelligent long short-term memory network. Environmental Pollution, 271 : 116327

[32] WangH, XiangZ, WangL, JingS, LouS, TaoS, LiuJ, YuM, LiL, LinL, ChenY, WiedensohlerA, ChenC. (2018). Emissions of volatile organic compounds (VOCs) from cooking and their speciation: A case study for Shanghai with implications for China. Science of the Total Environment, 621 : 1300– 1309

[33] WangS, FangC, MaH, WangY, QinJ. (2014). Spatial differences and multi-mechanism of carbon footprint based on GWR model in provincial China. Journal of Geographical Sciences, 24( 4): 612– 630

[34] WongP Y, SuH J, LeeH Y, ChenY C, HsiaoY P, HuangJ W, TeoT A, WuC D, SpenglerJ D. (2021). Using land-use machine learning models to estimate daily NO₂ concentration variations in Taiwan, China. Journal of Cleaner Production, 317 : 128411

[35] XiaR, Chen Y, RenB ( 2022). Improved anti-occlusion object tracking algorithm using Unscented Rauch-Tung-Striebel smoother and kernel correlation filter. Journal of King Saud University - Computer and Information Sciences

[36] XuX, ZhangC, LiangY. (2021). Review of satellite-driven statistical models PM_2.5 concentration estimation with comprehensive information. Atmospheric Environment, 256 : 118302

[37] ZhangH, LinfordJ C, SanduA, SanderR. (2011). Chemical mechanism solvers in air quality models. Atmosphere (Toronto), 2( 3): 510– 532

[38] ZhangJ M, FengW J, YuanT Y, WangJ, SangaiahA K. (2022a). SCSTCF: spatial-channel selection and temporal regularized correlation filters for visual tracking. Applied Soft Computing, 118 : 108485

[39] ZhangJ M, SunJ, WangJ, LiZ P, ChenX. (2022b). An object tracking framework with recapture based on correlation filters and Siamese networks. Computers & Electrical Engineering, 98 : 107730

[40] ZhangY, WenX Y, JangC J. (2010). Simulating chemistry-aerosol-cloud-radiation-climate feedbacks over the continental US using the online-coupled Weather Research Forecasting Model with chemistry (WRF/Chem). Atmospheric Environment, 44( 29): 3568– 3582

RIGHTS & PERMISSIONS

Higher Education Press