Analytical fitting functions of finite sample discrete entropies of white Gaussian noise

Abstract

In order to find the convergence rate of finite sample discrete entropies of a white Gaussian noise(WGN), Brown entropy algorithm is numerically tested.With the increase of sample size, the curves of these finite sample discrete entropies are asymptotically close to their theoretical values.The confidence intervals of the sample Brown entropy are narrower than those of the sample discrete entropy calculated from its differential entropy, which is valid only in the case of a small sample size of WGN. The differences between sample Brown entropies and their theoretical values are fitted by two rational functions exactly, and the revised Brown entropies are more efficient. The application to the prediction of wind speed indicates that the variances of resampled time series increase almost exponentially with the increase of resampling period.

Keywords

entropy / non-stationary time series / prediction / white Gaussian noise / sample size / wind speed

References

[1] Michalowicz J V, Nichols J M, Bucholtz F. Handbook of Differential Entropy [M], 2014, Florida, USA: CRC Press.

[2] Cover T M, Thomas J A. Elements of Information Theory [M], 2006, New Jersey, USA: John Wiley & Sons, Inc..

[3] Chen B, Principe J C. On the smoothed minimum error entropy criterion [J]. Entropy, 2012, 14(11): 2311-2323.

[4] Brown G, Pocock M, Zhao M J, et al. Conditional likelihood maximisation: A unifying framework for information theoretic feature selection [J]. Journal of Machine Learning Research, 2012, 13(1): 27-66.

[5] Zhang H, Yu Y, Liu Zhiyuan. Study on the maximum entropy principle applied to the annual wind speed probability distribution: A case study for observations of intertidal zone anemometer towers of Rudong in East China Sea [J]. Applied Energy, 2014, 114: 931-938.

[6] Bessa R J, Miranda V, Gama J. Entropy and correntropy against minimum square error in offline and online threeday ahead wind power forecasting [J]. IEEE Transactions on Power Systems, 2009, 24(4): 1657-1666.

[7] Chellali F, Khellaf A, Belouchrani A, et al. A comparison between wind speed distributions derived from the maximum entropy principle and Weibull distribution. Case of study; six regions of Algeria [J]. Renewable and Sustainable Energy Reviews, 2012, 16(1): 379-385.

[8] Ren B, Sun Y, Zhou Z, et al. Comprehensive evaluation model of reservoir operation based on improved set pair analysis [J]. Transactions of Tianjin University, 2013, 19(1): 25-28.

[9] Zeng X, Wang D, Wu Jichun. Sensitivity analysis of the probability distribution of groundwater level series based on information entropy [J]. Stochastic Environmental Research and Risk Assessment, 2012, 26(3): 345-356.

[10] Wang B, Hou Yuexian. Measurement of incompatible probability in information retrieval: A case study with user clicks [J]. Transactions of Tianjin University, 2013, 19(1): 37-42.

[11] Zhou X, Li Y F, He Bingwei. Entropy distribution and coverage rate-based birth intensity estimation in GMPHD filter for multi-target visual tracking [J]. Signal Processing, 2014, 94: 650-660.

[12] Xu G, Zhao Lina. Analysis of fluvial process based on information entropy [J]. Journal of Tianjin University( Science and Technology), 2013, 46(4): 347-353.

[13] Al-Omari A I. Estimation of entropy using random sampling [J]. Journal of Computational and Applied Mathematics, 2014, 261: 95-102.

[14] Hazewinkel M. Encyclopaedia of Mathematics [M], 2010, the Netherlands: Springer.

[15] Fan J, Yao Qiwei. Nonlinear Time Series: Nonparametric and Parametric Methods [M], 2005, New York, USA: Springer-Verlag.

[16] Concise Handbook of Mathematics [M]. Revised Edition. Shanghai Education Press, Shanghai, China, 1991(in Chinese).

[17] McClellan J H, Schafer R W, Yoder M A. Signal Processing First [M], 2003, New Jersey, USA: Prentice Hall.

[18] Zhai Xiangzhi. The Upper Limit Estimation of Forecast Accuracy for Electric Load in “Mechanism Model + Identification Model” Strategy[D], 2008, Tianjin, China: Tianjin University.