Quality control of specific humidity from surface stations based on EOF and FFT—Case study

Abstract

Comparisons between observations and background fields indicate that amplitude and phase differences in oscillations result in a non-Gaussian distribution in observation minus background vectors (OMB). Empirical Orthogonal Function (EOF) quality control (QC) and Fast Fourier Transform (FFT) quality control are proposed from the perspective of data assimilation and are applied to the surface specific humidity from ground-based stations. The QC results indicate that the standard deviation between observations and background is reduced effectively, and the frequency distribution for the observation increment is closer to a normal distribution. The specific humidity outliers occur primarily in mountainous and coastal regions. Comparing the two QC methods, it is found that the EOF QC performs better than the FFT QC as it can keep large scale of fluctuation information from the original field, preventing these waves from entering into the residual field and being removed by the QC process.

Keywords

specific humidity / quality control / EOF / FFT

References

[1] Anderson E, Järvinen H (1999). Variational quality control. Q J R Meteorol Soc, 125(554): 697–722

[2] Baker N L (1992). Quality control for the Navy operational atmospheric database. Weather Forecast, 7(2): 250–261

[3] Chen Q Y, Guan C G, Yao M M, Tong H, Mariano H, Li Z C (2007). Development of key techniques and experiments in global model upgrading. Acta Meteorologica Sinica, 65(4): 478–492 (in Chinese)

[4] Cooley J W, Tukey O W (1965). An algorithm for the machine calculation of complex Fourier series. Math Comput, 19(90): 297–301

[5] Eischeid J K, Bruce Baker C, Karl T R, Diaz H F (1995). The quality control of long-term climatological data using objective data analysis. J Appl Meteorol, 34(12): 2787–2795

[6] Fan S Y, Zhang C L (2006). Numerical assessing experiments on individual component impact of the meteorological observation network on the “July 2000” torrential rain in Beijing. Acta Meteorologica Sinica, 20(4): 389–401 (in Chinese)

[7] Feng S, Hu Q, Qian W (2004). Quality control of daily meteorological data in China, 1951–2000: a new dataset. Int J Climatol, 24(7): 853–870

[8] Gandin L S (1988). Complex quality control of meteorological observations. Mon Weather Rev, 116(5): 1137–1156

[9] Ghil M, Malanotte-Rizzoli P (1991). Data assimilation in meteorology and oceanography. Adv Geophys, 33: 151‒156

[10] Guo Y R, Shin D H, Lee J H, Xiao Q N, Barker D M, Kuo Y H (2002). Application of the MM5 3DVAR system for a heavy rain case over the Korean Peninsula. In: Proccedings: the Twelfth PSU/NCAR Mesoscale Model Users’ Workshop NCAR, Boulder, 24–25

[11] Ha K J, Jeon E H, Oh H M (2007). Spatial and temporal characteristics of precipitation using an extensive network of ground gauge in the Korean Peninsula. Atmos Res, 86(3–4): 330–339

[12] Hong S Y, Pan H L (1996). Nonlocal boundary layer vertical diffusion in a medium-range forecast model. Mon Weather Rev, 124(10): 2322–2339

[13] Hubbard K G, You J (2005). Sensitivity analysis of quality assurance using the spatial regression approach—A case study of the maximum/minimum air temperature. J Atmos Ocean Technol, 22(10): 1520–1530

[14] Ingleby N B, Lorenc A C (1993). Bayesian quality control using multivariate normal distributions. Q J R Meteorol Soc, 119(513): 1195–1225

[15] Kalnay E, Kanamitsu M, Kistler R, Collins W, Deaven D, Gandin L, Iredell M, Saha S, White G, Woolen J, Zhu Y, Leetmaa A, Reynolds R, Chelliah M, Ebisuzaki W, Higgins W, Janowiak J, Mo K C, Ropelewski C, Wang J, Jenne R, Joseph D (1996). The NCEP/NCAR 40-year reanalysis project. Bull Am Meteorol Soc, 77(3): 437–471

[16] Lanzante J R (1996). Resistant, robust and non-parametric techniques for the analysis of climate data: theory and examples, including applications to historical radiosonde station data. Int J Climatol, 16(11): 1197–1226

[17] Li L, Njoku E G, Im E, Chang P S, Germain K S (2004). A preliminary survey of radio-frequency interference over the US in Aqua AMSR-E data. Geoscience and Remote Sensing. IEEE Transactions on, 42(2): 380–390

[18] Lorenc A C, Hammon O (1988). Objective quality control of observations using Bayesian methods. Theory, and a practical implementation. Q J R Meteorol Soc, 114(480): 515–543

[19] Lorenz E N (1956). Empirical orthogonal functions and statistical weather prediction, scientific Report No. 1, Statistical Forecasting Project. Massachusetts Institute of Technology department of meteorology

[20] Mardia K V (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3): 519–530

[21] Mohanty U C, Kasahara A, Errico R (1986). The impact of diabatic heating on the initialization of a global forecast model. J Meteorol Soc Jpn, 64(6): 805–817

[22] Onogi K, Tsutsui J, Koide H, Sakamoto M, Kobayashi S, Hatsushika H, Matsumoto T, Yamazaki N, Kamahori H, Takahashi K, Kadokura S, Wada K, Kato K, Oyama R, Ose T, Mannoji N, Taira R (2007). The JRA-25 reanalysis. J Meteorol Soc Jpn, 85(3): 369–432

[23] Qin Z K, Zou X L, Li G, Ma X L (2010). Quality control of surface station temperature data with non-Gaussian observation-minus-background distributions. J Geophys Res, 115(D16): D16312

[24] Ramanathan V, Barkstrom B R, Harrison E F (1989). Climate and the earth’s radiation budget. Phys Today, 42(5): 22–32

[25] Reek T, Doty S R, Owen T W (1992). A deterministic approach to the validation of historical daily temperature and precipitation data from the cooperative network. Bull Am Meteorol Soc, 73(6): 753–762

[26] Sasaki Y (1970). Some basic formalisms in numerical variational analysis. Mon Weather Rev, 98(12): 875–883

[27] Shafer M A, Fiebrich C A, Arndt D S, , Fredrickson S E, Hughes T W (2000). Quality assurance procedures in the Oklahoma Mesonetwork. J Atmos Ocean Technol, 17(4): 474–494

[28] Simmons A, Uppala S, Dee D, Kobayashi S (2007). ERA-Interim: new ECMWF reanalysis products from 1989 onwards. ECMWF newsletter, 110(110): 25–35

[29] Torrence C, Compo G P (1998). A practical guide to wavelet analysis. Bull Am Meteorol Soc, 79(1): 61–78

[30] Wade C G (1987). A quality control program for surface mesometeorological data. J Atmos Ocean Technol, 4(3): 435–453

[31] Wang Y, Xu Z F, Fan G Z (2013). Research of quality control based on EOF for 2m temperature. Plateau Meteorology, 32(2): 564–574 (in Chinese)

[32] Wei F Y (2007). Statisticial Diagnosis and Prediction for Modern Climate. Beijing: China Meteorological Press, 71–76 (in Chinese)

[33] Zhao J, Zou X L, Weng F Z (2013). WindSat radio-frequency interference signature and its identification over Greenland and Antarctic. IEEE Trans Geosci Rem Sens, 51(9): 4830–4839

[34] Zou X L (2009). Theories and Applications for Data Assimilation (I). Beijing: China Meteorological Press (in Chinese)

[35] Zou X L, Ma Y, Qin Z K (2012). Fengyun-3B microwave humidity sounder (MWHS) data noise characterization and filtering using principle component analysis. IEEE Trans Geosci Rem Sens, 50(12): 4892–4902

[36] Zou X L, Qin Z K (2010). Time zone dependence of diurnal cycle errors in surface temperature analyses. Mon Weather Rev, 138(6): 2469–2475

[37] Zou X L, Zeng Z (2006). A quality control procedure for GPS radio occultation data. J Geophys Res, 111: D02112

Acknowledgements

This work was jointly supported by the National Basic Research Program of China (No. 2010CB951600), China Special Fund for Meteorological Research in the Public Interest (No. GYHY201406008), the Research Innovation Program for college graduates of Jiangsu Province (CXZZ13_0503) and the Priority Academic Program Development (PAPD) of Jiangsu Higher Education Institutions. Thanks for NCEP for providing FNL data on http://rda.ucar.edu/.

RIGHTS & PERMISSIONS

2014 Higher Education Press and Springer-Verlag Berlin Heidelberg