Integral-type functionals of first hitting times for continuous-time Markov chains

Abstract

We investigate integral-type functionals of the first hitting times for continuous-time Markov chains. Recursive formulas and drift conditions for calculating or bounding integral-type functionals are obtained. The connection between the subexponential integral-type functionals and the subexponential ergodicity is established. Moreover, these results are applied to the birth-death processes. Polynomial integral-type functionals and polynomial ergodicity are studied, and a sufficient criterion for a central limit theorem is also presented.

Keywords

Integral-type functional / continuous-time Markov chain (CTMC) / subexponential ergodicity / birth-death process / central limit theorem (CLT)

References

[1] Anderson W J. Continuous-Time Markov Chains: An Applications-Oriented Approach.New York: Springer, 1991

[2] Bladt M. The variance constant for the actual waiting time of the PH/PH/1 queue. Ann Appl Probab, 1996, 6(3): 766–777

[3] Chen M F. From Markov Chains to Non-Equilibrium Particle Systems.2nd ed. Singapore: Word Scientific, 2004

[4] Chen M F, Wang Y Z. Algebraic convergence of Markov chains. Ann Appl Probab, 2003, 13(2): 604–627

[5] Douc R, Fort G, Guillin A. Subgeometric rates of convergence of f-ergodic strong Markov processes. Stoch Proc Appl, 2009, 119(3): 897–923

[6] Fort G, Roberts G O. Subgeometric ergodicity of strong Markov processes. Ann Appl Probab, 2005, 15(2): 1565–1589

[7] Glynn P W, Whitt W. Necessary conditions in limit theorems for cumulative processes. Stochastic Process Appl, 2002, 98(2): 199–209

[8] Hou Z T, Guo Q F. Homogeneous Denumerable Markov Processes.New York: Springer, 1988

[9] Hou Z T, Liu Y Y, Zhang H J. Subgeometric rates of convergence for a class of continuous-time Markov processes. J Appl Probab, 2005, 42(3): 698–712

[10] Jiang S X, Liu Y Y, Yao S. Poisson’s equation for discrete-time single-birth processes. Statist Probab Lett, 2014, 85(2): 78–83

[11] Liu Y Y. Additive functionals for discrete-time Markov chains with applications to birth-death processes. J Appl Probab, 2011, 48(4): 925–937

[12] Liu Y Y, Wang P F, Xie Y M. Deviation matrix and asymptotic variance for GI/M/1-type Markov chains. Front Math China, 2014, 9(4): 863–880

[13] Liu Y Y, Wang P F, Zhao Y Q. The variance constant for continuous-time level dependent quasi-birth-and-death processes. Stoch Models, 2018, 34(1): 25–44

[14] Liu Y Y, Zhang H J, Zhao Y Q. Subgeometric ergodicity for continuous-time Markov chains. J Math Anal Appl, 2010, 368(1): 178–189

[15] Liu Y Y, Zhang Y H. Central limit theorems for ergodic continuous-time Markov chains with applications to single birth processes. Front Math China, 2015, 10(4): 933–947

[16] Mao Y H. Ergodic degrees for continuous-time Markov chains. Sci China Ser A, 2004, 47(2): 161–174

[17] Mey S P, Tweedie R L. Markov Chains and Stochastic Stability.2nd ed. Cambridge: Cambridge Univ Press, 2009

[18] Pollett P K, Stefanov V T. Path integrals for continuous-time Markov chains. J Appl Probab, 2002, 39(4): 901–904

[19] Stefanov V T, Wang S. A note on integrals for birth-death processes. Math Biosci, 2000, 168(2): 161–165

[20] Whitt W. Asymptotic formulas for Markov processes with applications to simulation. Oper Res, 1992, 40(2): 279–291

[21] Zhang Y H. A note on exponential ergodicity and ℓ-ergodicity of single birth processes. J Beijing Normal Univ, 2010, 46(1): 1–5 (in Chinese)

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature