Ergodicity of Inhomogeneous Markov Processes Under General Criteria

Abstract

This paper is concerned with ergodic properties of inhomogeneous Markov processes. Since the transition probabilities depend on initial times, the existing methods to obtain invariant measures for homogeneous Markov processes are not applicable straightforwardly. We impose some appropriate conditions under which invariant measure families for inhomogeneous Markov processes can be studied. Specifically, the existence of invariant measure families is established by either a generalization of the classical Krylov–Bogolyubov method or a Lyapunov criterion. Moreover, the uniqueness and exponential ergodicity are demonstrated under a contraction assumption of the transition probabilities on a large set. Finally, three examples, including Markov chains, diffusion processes and storage processes, are analyzed to illustrate the practicality of our method.

Keywords

Ergodicity / inhomogeneous Markov process / invariant measure family / Feller transition semigroup

References

[1] BakryD, CattiauxP, GuillinA. Rate of convergence for ergodic continuous Markov processes: Lyapunov versus Poincaré. J. Funct. Anal., 2008, 254(3): 727-759

[2] BenaïmM, BouguetF, CloezB. Ergodicity of inhomogeneous Markov chains through asymptotic pseudotrajectories. Ann. Appl. Probab., 2017, 27(5): 3004-3049

[3] BenaïmM, HurthTMarkov Chains on Metric Spaces—A Short Course, 2022, Cham, Springer

[4] BogachevVl, KrylovNV, RöcknerM, ShaposhnikovSVFokker–Planck–Kolmogorov Equations, 2015, Providence, RI, American Mathematical Society207

[5] BrahimiM, WorthingtonDJ. Queuing models for out-patient appointment systems—a case study. J. Opl. Res. Soc., 1991, 42(9): 733-746

[6] ChebanD, LiuZ. Periodic, quasi-periodic, almost periodic, almost automorphic, Birkhoff recurrent and Poisson stable solutions for stochastic differential equations. J. Differential Equations, 2020, 269(4): 3652-3685

[7] ChengM, LiuZ. The second Bogolyubov theorem and global averaging principle for SPDEs with monotone coefficients. SIAM J. Math. Anal., 2023, 55(2): 1100-1144

[8] Da PratoG, DebusscheA. 2D stochastic Navier-Stokes equations with a time-periodic forcing term. J. Dynam. Differential Equations, 2008, 20(2): 301-335

[9] Da PratoG, RïocknerM. A note on evolution systems of measures for time-dependent stochastic differential equations. Seminar on Stochastic Analysis, Random Fields and Applications V, 2008, Basel, Birkhïauser: 115-122 59

[10] Da PratoG, ZabczykJErgodicity for Infinite-dimensional Systems, 1996, Cambridge, Cambridge University Press 229

[11] DoeblinW. Exposé de la théorie des chaînes simples constants de Markoff á un nombre fini d’états. Rev. Math. Un. Interbalkanique, 1938, 2: 77-105

[12] DolgopyatD, HafoutaY. A Berry–Esseen theorem and Edgeworth expansions for uniformly elliptic inhomogeneous Markov chains. Probab. Theory Related Fields, 2023, 186(1–2): 439-476

[13] DolgopyatD, SarigOLocal Limit Theorems for Inhomogeneous Markov Chains, 2023, Cham, Springer2331

[14] DoucR, FortG, GuillinA. Subgeometric rates of convergence of f-ergodic strong Markov processes. Stochastic Process. Appl., 2009, 119(3): 897-923

[15] GeH, JiangD-Q, QianM. A simple discrete model of Brownian motors: time-periodic Markov chains. J. Stat. Phys., 2006, 123(4): 831-859

[16] GeH, JiangD-Q, QianM. Reversibility and entropy production of inhomogeneous Markov chains. J. Appl. Probab., 2006, 43(4): 1028-1043

[17] GīhmanǏĪ, SkorohodAV. Stochastic Differential Equations. Ergebnisse der Mathematik und ihrer Grenzgebiete, 1972, New York-Heidelberg, Springer-VerlagTranslated from the Russian by Kenneth Wickwire

[18] HairerMErgodic properties of Markov processes, 2018

[19] HairerMConvergence of Markov processes, 2021

[20] HairerM, MattinglyJC. Yet another look at Harris’ ergodic theorem for Markov chains. Seminar on Stochastic Analysis, Random Fields and Applications VI, 2011, Basel, Birkhäauser/Springer Basel AG: 109-117 63

[21] HajnalJ. The ergodic properties of non-homogeneous finite Markov chains. Proc. Cambridge Philos. Soc., 1956, 52: 67-77

[22] HajnalJ. Weak ergodicity in non-homogeneous Markov chains. Proc. Cambridge Philos. Soc., 1958, 54: 233-246

[23] HarrisTE. The existence of stationary measures for certain Markov processes. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, 1956, Berkeley–Los Angeles, CA, Univ. California Press: 113-124Vol. II

[24] HarrisonJM, TaylorAJ. Optimal control of a Brownian storage system. Stochastic Process. Appl., 1977, 6(2): 179-194 78

[25] HarrisonJM, WilliamsRJ. On the quasireversibility of a multiclass Brownian service station. Ann. Probab., 1990, 18(3): 1249-1268

[26] IosifescuMFinite Markov Processes and Their Applications, Corrected Reprint of the 1980 Original, 2007, Mineola, NY, Dover Publications, Inc.

[27] KhasminskiiRStochastic Stability of Differential Equations, 2012, Heidelberg, Springer With contributions by G. N. Milstein and M. B. Nevelson, Completely Revised and Enlarged Second Edition66

[28] LevitanBM, ZhikovVVAlmost Periodic Functions and Differential Equations, 1982, Cambridge-New York, Cambridge University PressTranslated from the Russian by L. W. Longdon

[29] LindvallTLectures on the coupling method, 1992, New York, John Wiley & Sons, Inc.

[30] MeynSP, TweedieRL. Stability of Markovian processes, I. Criteria for discrete-time chains. Adv. in Appl. Probab., 1992, 24(3): 542-574

[31] MeynSP, TweedieRL. Stability of Markovian processes, II. Continuous-time processes and sampled chains. Adv. in Appl. Probab., 1993, 25(3): 487-517

[32] MeynSP, TweedieRL. Stability of Markovian processes, III. Foster-Lyapunov criteria for continuous-time processes. Adv. in Appl. Probab., 1993, 25(3): 518-548

[33] MeynSP, TweedieRLMarkov Chains and Stochastic Stability, 2009Second EditionCambridge, Cambridge University Press, Cambridge

[34] Rey-BelletLErgodic Properties of Markov Processes, 2006, Berlin, Springer1881

[35] SärkkäS, SolinAApplied Stochastic Differential Equations, 2019, Cambridge, Cambridge University Press

[36] WorthingtonD, WallA. Using the discrete time modelling approach to evaluate the time-dependent behaviour of queueing systems. J. Oper. Res. Soc., 1999, 50(8): 777-788

[37] WuF, YinG, MeiH. Stochastic functional differential equations with infinite delay: existence and uniqueness of solutions, solution maps, Markov properties, and ergodicity. J. Differential Equations, 2017, 262(3): 1226-1252

[38] YoshizawaTStability Theory by Liapunov’s Second Method, 1966, Tokyo, Mathematical Society of Japan9

[39] YoshizawaTStability Theory and the Existence of Periodic Solutions and Almost Periodic Solutions, 1975, New York-Heidelberg, Springer-Verlag Vol. 14

[40] ZeifmanAI, IsaacsonD. On strong ergodicity for nonhomogeneous continuous-time Markov chains. Stochastic Process. Appl., 1994, 50(2): 263-273

[41] ZhuC, YinG. On strong Feller, recurrence, and weak stabilization of regime-switching diffusions. SIAM J. Control Optim., 2009, 48(3): 2003-2031

RIGHTS & PERMISSIONS

Peking University