Improving the Lag Window Estimators of the Spectrum and Memory for Long-Memory Stationary Gaussian Processes

Abstract

Gaussian process (GP) is a stochastic process that has been successfully applied in finance, black-box modeling of biosystems, machine learning, geostatistics, multitask learning or robotics and reinforcement learning. Effectively estimating the spectral density function (SDF) and degree of memory (DOM) of a long-memory stationary GP (LMSGP) is needed to get accurate information about the process. The practice demonstrated that the periodogram estimator (PE) and lag window estimator (LWE) that are the extremely used estimators of the SDF and DOM have some drawbacks, especially for LMSGPs. The behaviors of the PEs and LWEs are soundly investigated numerically; however, the theoretical justifications are limited and thus the challenge to improve them is still daunting. This paper gives a closer look at the theoretical justifications of the efficiency of the LWEs that provides new sufficient conditions (NSCs) for improving the LWEs of the SDF and DOM for LMSGPs. The precision, the convergence rate of the bias and variance, and the asymptotic distributions of the LWEs under the NSCs are studied. A comparison study among the LWEs under the NSCs, the LWEs without the NSCs and the PEs is given to investigate the significance of the NSCs. The main theoretical and simulation results show that: the LWEs under the NSCs are asymptotically unbiased and consistent and better than the LWEs without the NSCs and the PEs, and the asymptotic distributions of the LWEs under the NSCs are chi-square for SDF and normal for DOM.

Keywords

Gaussian process / Spectral density / Degree of memory / Lag window / Periodogram / Local Whittle method / Regression method

References

[1] Alvarez MA, Rosasco L, Lawrence ND. Kernels for vector-valued functions: a review. Found. Trends in Mach. Learn.. 2012, 4 3 195-266

[2] Azman K, Kocijan K. Application of Gaussian processes for black-box modelling of biosystems. ISA Trans.. 2007, 46 443-457

[3] Baillie RT. Long-memory processes and fractional integration in econometrics. J. Economet.. 1996, 73 5-59

[4] Bartlett MS. Smoothing periodograms from time series with continuous spectra. Nature. 1948, 161 686-687

[5] Beran J. Statistics for Long Memory Processes. 1994 New York: Chapman and Hall

[6] Beran J, Kulik R, Ghosh S, Feng Y. Long-Memory Processes: Probabilistic Properties and Statistical Methods. 2013 London: Springer Science and Business Media

[7] Blackman RB, Tukey JW. The Measurement of Power Spectra from the Point of View of Communications Engineering- Part I. 1959 New York: Dover

[8] Box GEP, Jenkins GM. Time Series Analysis: Forecasting and Control. 1976 San Francisco: Holden Day

[9] Brockwell PJ, Davis RA. Time Series: Theory and Methods. 1991 2 New York: Springer Verlag

[10] Chen G, Abraham B, Peiris S. Lag window estimation of the degree of differencing in fractionally integrated time series models. J. Time Ser. Anal.. 1993, 15 5 473-487

[11] Cressie N. Statistics for spatial data. Terra Nova. 1992, 4 5 613-617

[12] Deisenroth MPD, Rasmussen CE. Gaussian processes for data-efficient learning in robotics and control. IEEE Trans. Pattern Anal. Mach. Intell.. 2015, 37 2 408-423

[13] Elsawah AM. Multiple doubling: a simple effective construction technique for optimal two-level experimental designs. Statist Papers. 2021, 62 2923-2967

[14] Elsawah AM. An appealing technique for designing optimal large experiments with three-level factors. J. Comput. Appl. Math.. 2021, 384

[15] Elsawah, A.M.: Designing optimal large four-level experiments: a new technique without recourse to optimization softwares. Commun. Math. Statistics (2021). https://doi.org/10.1007/s40304-021-00241-y

[16] Elsawah AM. Building some bridges among various experimental designs. J. Korean Stat. Soc.. 2020, 49 55-81

[17] Frederiksen, P., Nielsen, F. S. and Nielsen, M.: Local polynomial Whittle estimation of perturbed fractional processes. J. Economet. (2012)

[18] Geweke J, Porter-Hudak S. The estimation and application of long memory time series models. J. Time Ser. Anal.. 1983, 4 4 221-238

[19] Granger CWJ, Joyeux R. An introduction to long-memory time series models and fractional differencing. J. Time Ser. Anal.. 1980, 1 15-29

[20] Grenander U, Rosenblatt M. Statistical spectral analysis of time series arising from stationary stochastic processes. Ann. Math. Statist.. 1953, 24 4 537-558

[21] GušŠgan D. Les chaos finance: approche statistique. 2003 Paris: Economica

[22] Hassler U. Regression of spectral estimator with fractionally integrated time series. J. Time Ser. Anal.. 1993, 14 4 369-380

[23] Hassler U. Time Series Analysis with Long Memory in View. 2019 New Jersey: John Wiley and Sons, Inc.

[24] Hunt RL, Peiris MS, Weber NC. The bias of lag window estimators of the fractional difference parameter. J. Appl. Math. Comput.. 2003, 12 67-79

[25] Hurst HE. Long-term storage capacity of reservoirs. Trans. Am. Soc. Civil Eng.. 1951, 116 770-799

[26] Hurvich C, Beltrao K. Asymptotics for the low-frequency ordinates of the periodogram of a long-memory time series. J. Time Ser. Anal.. 1993, 14 5 455-472

[27] Künsch, H. R.: Statistical aspects of self-similar processes. Proceedings of the First World Congress of the Bernoulli Society. (1987)

[28] Lahiri SN. A necessary and sufficient condition for asymptotic independence of discrete Fourier transforms under short-and long-range dependence. Ann. Statist.. 2003, 31 2 613-641

[29] Lang G, Azais JM. Non-parametric estimation of the long-range dependence exponent for Gaussian processes. J. Statistical Plan. Inference.. 1999, 80 59-80

[30] Solomon, Otis M.: Jr. PSD Computations Using Welchs Method. Sandia National Laboratories Albuquerque (1991)

[31] Parzen E. Time series model identification and prediction variance horizon in Applied Time Series Analysis II. 1981 Findley New York: Academic Press. 415-447

[32] Peiris MS, Court JR. A note on the estimation of degree of differencing in long memory time series analysis. Probab. Math. Statist.. 1993, 14 2 223-229

[33] Priestley, M.B.: Spectral analysis and Time Series. Academic Press, New York, School of Mathematics and Statistics (1981)

[34] Rashid, A.S., Allami, M.J.H., Mutasher, A.K.: Best Lag Window for Spectrum Estimation of Law Order MA Process, p. 9352453. Abstract and Applied Analysis, ID (2020)

[35] Rasmussen CE, Williams CKI. Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. 2006 Cambridge: MIT Press

[36] Robinson, P. M.: Time series with strong dependence. In: Sims, C.A. (Ed.), Proceedings of the Sixth World Congress on Advances in Econometrics. Cambridge University Press, Cambridge (1994)

[37] Robinson PM. Semiparametric analysis of long-memory time series. Ann. Statist.. 1994, 22 1 515-539

[38] Robinson PM. Log-periodogram regression of time series with long range dependence. Ann. Statist.. 1995, 23 3 1048-1072

[39] Rosenblatt, M.: Stationary sequences and random fields. Boston ; Basel ; Stuttgart : Birkhauser, ISBN -13: 978-0-8176-3264-9. (1985)

[40] Tikhonov S. Trigonometric series with general monotone coefficients. J. Maths. anal. appl.. 2007, 326 721-735

[41] Vishwakarma GK, Paul C, Elsawah AM. An algorithm for outlier detection in a time series model using backpropagation neural network. J. King Saud Univ.-Sci.. 2020, 32 3328-3336

[42] Vishwakarma GK, Paul C, Elsawah AM. A hybrid feedforward neural network algorithm for detecting outliers in non-stationary multivariate time series. Expert Syst. Appl.. 2021, 184

[43] Willinger W, Taqqu MS, Leland WE, Wilson DV. Self-similarity in high-speed packet traffic: analysis and modeling of Ethernet traffic measurements. Stat. Sci.. 1995, 10 67-85

[44] Yajima Y. A central limit theorem of Fourier transforms of strongly dependent stationary processes. J. Time Series Anal.. 1989, 10 375-83

[45] Yong CH. Asymptotic Behaviour of Trigonometric Series. 1974 Hong Kong: Chinese University of Hong Kong

[46] Zygmund A. Trigonometric Series. 1977 UK: Cambridge University Press

Funding

uic(R201810)

uic(R202010)