Variational quality control of non-Gaussian innovations and its parametric optimizations for the GRAPES m3DVAR system

Jie HE, Yang SHI, Boyang ZHOU, Qiuping WANG, Xulin MA

PDF(2953 KB)
PDF(2953 KB)
Front. Earth Sci. ›› 2023, Vol. 17 ›› Issue (2) : 620-631. DOI: 10.1007/s11707-021-0962-1
RESEARCH ARTICLE

Variational quality control of non-Gaussian innovations and its parametric optimizations for the GRAPES m3DVAR system

Author information +
History +

Abstract

The magnitude and distribution of observation innovations, which have an important impact on the analyzed accuracy, are critical variables in data assimilation. Variational quality control (VarQC) based on the contaminated Gaussian distribution (CGD) of observation innovations is now widely used in data assimilation, owing to the more reasonable representation of the probability density function of innovations that can sufficiently absorb observations by assigning different weights iteratively. However, the inaccurate parameters prevent VarQC from showing the advantages it should have in the GRAPES (Global/Regional Assimilation and PrEdiction System) m3DVAR system. Consequently, the parameter optimization methods are considerable critical studies to improve VarQC. In this paper, we describe two probable CGDs to include the non-Gaussian distribution of actual observation errors, Gaussian plus flat distribution and Huber norm distribution. The potential optimization methods of the parameters are introduced in detail for different VarQCs. With different parameter configurations, the optimization analysis shows that the Gaussian plus flat distribution and the Huber norm distribution are more consistent with the long-tail distribution of actual innovations compared to the Gaussian distribution. The VarQC’s cost and gradient functions with Huber norm distribution are more reasonable, while the VarQC’s cost function with Gaussian plus flat distribution may converge on different minimums due to its non-concave properties. The weight functions of two VarQCs gradually decrease with the increase of innovation but show different shapes, and the VarQC with Huber norm distribution shows more elasticity to assimilate the observations with a high contamination rate. Moreover, we reveal a general derivation relationship between the CGDs and VarQCs. A novel schematic interpretation that classifies the assimilated data into three categories in VarQC is presented. They are conducive to the development of a new VarQC method in the future.

Graphical abstract

Keywords

data assimilation / variational quality control / contaminated Gaussian distribution / non-Gaussian distribution / innovation

Cite this article

Download citation ▾
Jie HE, Yang SHI, Boyang ZHOU, Qiuping WANG, Xulin MA. Variational quality control of non-Gaussian innovations and its parametric optimizations for the GRAPES m3DVAR system. Front. Earth Sci., 2023, 17(2): 620‒631 https://doi.org/10.1007/s11707-021-0962-1

Jie He received his Ph.D. Degree in meteorology from Nanjing University of Information Science and Technology (NUIST), Nanjing, China in 2021, mainly focusing on the research of variational quality control and ensemble reanalysis. He continues as a postdoc to study data assimilation and ensemble forecast at the Guangzhou Institute of Tropical and Marine Meteorology.E-mail: hej@gd121.cn.

Yang Shi received her M.S. Degree in meteorology from NUIST, Nanjing, China in 2015. She is an Engineer at the Guangdong Meteorological Observatory and her current research includes ensemble forecasts and postprocessing of NWP.E-mail: shiyang@gd121.cn

Boyang Zhou received her M.S. Degree in meteorology from NUIST, Nanjing, China in 2017. ​She is an engineer at the Qingdao Air Traffic Management Station of Civil Aviation of China and her current research includes data assimilation and ensemble perturbation schemes.E-mail: zby12357@126.com.

Qiuping Wang is a graduate student in meteorology from the School of Atmospheric Sciences at NUIST, Nanjing, China. Her research interest focuses on ensemble forecast and data assimilation. E-mail: wangqp@nuist.edu.cn.

Xulin Ma received his Ph.D. Degree in meteorology from NUIST, China in 2008. His major field is data assimilation, ensemble forecast, and meso- and micro-scale numerical simulation. He works as a Professor at the School of Atmospheric Sciences, NUIST, Nanjing, China since 2008. He is devoted to studying ensemble forecast theory, hybrid data assimilation, and adaptive or targeting observation.E-mail: xulinma@nuist.edu.cn

References

[1]
Anderson E, Jarvinen H (1999). Variational quality control.Q J R Meteorol Soc, 125(554): 697–722
CrossRef Google scholar
[2]
Courtier P, Andersson E, Heckley W, Vasiljevic D, Hamrud M, Hollingsworth A, Rabier F, Fisher M, Pailleux J (1998). The ECMWF implementation of three-dimensional variational assimilation (3D-Var).Part I: formulation. Q J R Meteorol Soc, 124(550): 1783–1807
CrossRef Google scholar
[3]
Dharssi I, Lorenc A C, Ingleby N B (1992). Treatment of gross errors using maximum probability theory.Q J R Meteorol Soc, 118(507): 1017–1036
CrossRef Google scholar
[4]
Duan B H, Zhang W M, Yang X F, Dai H J, Yu Y (2017). Assimilation of typhoon wind field retrieved from scatterometer and SAR based on the Huber norm quality control.Remote Sens (Basel), 9(10): 987
CrossRef Google scholar
[5]
Fernández C, Steel M F (1998). On Bayesian modeling of fat tails and skewness.J Am Stat Assoc, 93(441): 359–371
[6]
Fowler A, van Leeuwen P J (2013). Observation impact in data assimilation: the effect of non-Gaussian observation error.Tellus, 65(1): 20035
CrossRef Google scholar
[7]
Gauthier P, Chouinard C, Brasnett B (2003). Quality control: methodology and applications. In: Swinbank R, Shutyaev V, Lahoz W A, eds. Data Assimilation for the Earth System. Dordrecht: Springer-Verlag, 177–187
[8]
Guitton A, Symes W W (2003). Robust inversion of seismic data using the Huber norm.Geophysics, 68(4): 1310–1319
CrossRef Google scholar
[9]
Hampel F R (1977). Rejection rules and robust estimates of location: an analysis of some Monte Carlo results. In: Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, and Random Processes. Dordrecht, Hingham, MA, 187–194
[10]
Hampel F R (2001). Robust statistics: a brief introduction and overview. In: Research report/Seminar für Statistik, Eidgenössische Technische Hochschule. Zurich, Switzerland, 1–5
[11]
He J, Ma X L, Ge X Y, Liu J J, Cheng W, Chan M Y, Xiao Z N (2021). Variational quality control of non-Gaussian innovations in the GRAPES m3DVAR system: mass field evaluation of assimilation experiments.Adv Atmos Sci, 38(9): 1510–1524
CrossRef Google scholar
[12]
Hollingsworth A (1989). The role of real-time four-dimensional data assimilation in the quality control, interpretation, and synthesis of climate data. In: Anderson D L T, and Willebrand J, eds. Oceanic Circulation Models: Combining Data and Dynamics. Dordrecht: Springer-Verlag, 304–339
[13]
Houtekamer P L, Mitchell H L (1998). Data assimilation using an ensemble Kalman filter technique.Mon Weather Rev, 126(3): 796–811
CrossRef Google scholar
[14]
Huber P J (1972). The 1972 wald lecture robust statistics: a review.Ann Math Stat, 43(4): 1041–1067
CrossRef Google scholar
[15]
Huber P J (2011). Robust statistics. In: Lovric M, ed. International Encyclopedia of Statistical Science. Heidelberg: Springer-Verlag, 1248–1251
[16]
Ingleby N B, Lorenc A C (1993). Bayesian quality control using multivariate normal distributions.Q J R Meteorol Soc, 119(513): 1195–1225
CrossRef Google scholar
[17]
Jarvinen H, Unden P (1997). Observation screening and first guess quality control in the ECMWF 3D-Var data assimilation system.In: ECMWF Technical Memoranda, (236): 1–33
[18]
Kalnay E (2003). Atmospheric Modeling, Data Assimilation and Predictability.New York: Cambridge University Press, 198–204
[19]
Lahoz W A, Schneider P (2014). Data assimilation: making sense of earth observation.Front Env Sci—Switz, 2: 16
[20]
Legrand R, Michel Y, Montmerle T (2016). Diagnosing non-Gaussianity of forecast and analysis errors in a convective scale model.Nonlinear Process Geophys, 23: 1–12
CrossRef Google scholar
[21]
Lorenc A C (1988). Optimal nonlinear objective analysis.Q J R Meteorol Soc, 114(479): 205–240
CrossRef Google scholar
[22]
Lorenc A C, Hammon O (1988). Objective quality control of observations using Bayesian methods: theory, and a practical implementation.Q J R Meteorol Soc, 114(480): 515–543
CrossRef Google scholar
[23]
Ma X L, Zhuang Z R, Xue J S, Lu W S (2009). Development of 3-D variational data assimilation system for the nonhydrostatic numerical weather prediction model-GRAPES.Acta Meteorol Sin, 67(1): 50–60
[24]
Ma X L, He J, Zhou B Y, Li L, Ji Y, Guo H (2017). Effect of variational quality control of Non-Gaussian distribution observation error on heavy rainfall prediction. Trans Atmos Sci, 40(2): 170−180 (in Chinese)
[25]
Parrish D F, Derber J C (1992). The national meteorological center spectral statistical interpolation analysis system.Mon Weather Rev, 120(8): 1747–1763
CrossRef Google scholar
[26]
Pires C A, Talagrand O, Bocquet M (2010). Diagnosis and impacts of non-Gaussianity of innovations in data assimilation.Physica D, 239(17): 1701–1717
CrossRef Google scholar
[27]
Rabier F, Järvinen H, Klinker E, Mahfouf J F, Simmons A (2000). The ECMWF operational implementation of four-dimensional variational assimilation.I: experimental results with simplified physics. Q J R Meteorol Soc, 126(564): 1143–1170
CrossRef Google scholar
[28]
Sondergaard T, Lermusiaux P F (2013). Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations.Part I: theory and scheme. Mon Weather Rev, 141(6): 1737–1760
CrossRef Google scholar
[29]
Storto A (2016). Variational quality control of hydrographic profile data with non-Gaussian errors for global ocean variational data assimilation systems.Ocean Model, 104: 226–241
CrossRef Google scholar
[30]
Su X J, Pursera R J (2013). A new observation error probability model for nonlinear variational quality control and applications within the NCEP gridpoint statistical interpolation. In: Sixth WMO Symposium on Data Assimilation. Sixth WMO Symposium on Data Assimilation, 33–34
[31]
Swinbank R, Shutyaev V, Lahoz W A (2003). Data assimilation for the earth system. Maratea: Springer, 31–35
[32]
Tavolato C, Isaksen L (2015). On the use of a Huber norm for observation quality control in the ECMWF 4D–Var.Q J R Meteorol Soc, 141(690): 1514–1527
CrossRef Google scholar
[33]
Tukey J W (1960). A survey of sampling from contaminated distributions. In: Olkin I, ed. Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling. Stanford, CA: Stanford University Press, 448–485
[34]
Wang X G, Lei T (2014). GSI-based four-dimensional ensemble-variational (4DEnsVar) data assimilation: formulation and single-resolution experiments with real data for NCEP global forecast system.Mon Weather Rev, 142(9): 3303–3325
CrossRef Google scholar
[35]
Yang Y X (1991). Robust bayesian estimation.J Geod, 65(3): 145–150
[36]
Zhao H, Zou X L, Qin Z K (2015). Quality control of specific humidity from surface stations based on EOF and FFT—case study.Front Earth Sci, 9(3): 381–393
CrossRef Google scholar
[37]
Zhu J J (1996). Robustness and the robust estimate.J Geod, 70(9): 586–590
CrossRef Google scholar
[38]
Zhu J J, Zeng Z Q (1999). The theory of surveying adjustment under contaminated error model. Acta Geodaetica et Cartographica Sinica, 28(3): 91–91(in Chinese)

Acknowledgments

The authors thank Yinghui Lu for his helpful advice and grammar correction. Jie He is supported by the China Scholarship Council. This work is primarily sponsored by the National Key R&D Program of China (Nos. 2018YFC1506702 and 2017YFC1502000). We acknowledge the High Performance Computing Center of Nanjing University of Information Science & Technology (NUIST) for their support of this work. The data sets in this paper are archived and accessible on the super computer of NUIST.

RIGHTS & PERMISSIONS

2022 Higher Education Press
AI Summary AI Mindmap
PDF(2953 KB)

Accesses

Citations

Detail

Sections
Recommended

/