Hostname: page-component-74656f9884-8r8nv Total loading time: 0 Render date: 2026-11-09T10:50:05.996Z Has data issue: false hasContentIssue false
  • English
  • Français

COUNTABLE STATE MARKOV PROCESSES: NON-EXPLOSIVENESS AND MOMENT FUNCTION

Published online by Cambridge University Press:  09 July 2015

F.M. Spieksma
F.M. Spieksma*
Affiliation:
Mathematics Institute, Leiden University, Niels Borhweg 1, 2333CA Leiden, The Netherlands Email: spieksma@math.leidenuniv.nl
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

The existence of a moment function satisfying a drift function condition is well known to guarantee non-explosiveness of the associated minimal Markov process (cf. [1,9]), under standard technical conditions. Surprisingly, the reverse is true as well for a countable space Markov process. We prove this result by showing that recurrence of an associated jump process, that we call the α-jump process, is equivalent to non-explosiveness. Non-explosiveness corresponds in a natural way to the validity of the Kolmogorov integral relation for the function identically equal to 1. In particular, we show that positive recurrence of the α-jump chain implies that all bounded functions satisfy the Kolmogorov integral relation. We present a drift function criterion characterizing positive recurrence of this α-jump chain.

Suppose that to a drift function V there corresponds another drift function W, which is a moment with respect to V. Via a transformation argument, the above relations hold for the transformed process with respect to V. Transferring the results back to the original process, allows to characterize the V-bounded functions that satisfy the Kolmogorov forward equation.

Information

Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 29 , Issue 4 , October 2015 , pp. 623 - 637
Copyright
Copyright © Cambridge University Press 2015 

References

1. Anderson, W.J. (1991). Continuous-time Markov chains. New York: Springer-Verlag.Google Scholar
2. Avrachenkov, K., Piunovskiy, A., and Zhang, Y. (2013). Markov processes with restart. J. Appl. Prob. 50: 960–968.Google Scholar
3. Brémaud, P. (1999). Markov chains. Gibbs fields, Monte Carlo Simulation, and Queues. Number 31 in Texts in Applied Mathematics. New York: Springer-Verlag.Google Scholar
4. Chen, M.F. (1986). Coupling for jump processes. Acta Mathematica Sinica 2(2): 123–136.Google Scholar
5. Chen, M.F. (1992). From Markov Chains to Non-equilibrium Particle Systems. Singapore: World Scientific Publishing Co.Google Scholar
6. Guo, X.P. and Hernández-Lerma, O. (2009). Continuous-time Markov decision processes . In Rozovskii, B. & Grimmett, G. (eds), Stochastic Modelling and Applied Probability, Vol. 62. Heidelberg: Springer-Verlag.Google Scholar
7. Guo, X.P., Hernández-Lerma, O., and Prieto–Rumeau, T. (2006). A survey of recent results on continuous-time Markov decision processes. TOP 14: 177–261.Google Scholar
8. Mertens, J.-F., Samuel-Cahn, E., and Zamir, S. (1978). Necessary and sufficient conditions for recurrence and transience of Markov chains in terms of inequalities. Journal of Applied Probability 15: 848–851.Google Scholar
9. Meyn, S.P. and Tweedie, R.L. (1995). Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Advances in Applied Probability 25: 518–548.Google Scholar
10. Norris, J.R. (2004). Markov Chains. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press.Google Scholar
11. Piunovskiy, A. and Zhang, Y. (2012). The transformation method for continuous-time Markov decision processes. Journal of Optimization Theory and Applications 154(2): 691–712.Google Scholar
12. Prieto-Rumeau, T. and Hernández-Lerma, O. (2012). Selected Topics on Continuous-Time Controlled Markov Chains and Markov Games, ICP Advanced Texts in Mathematics. Vol. 5. London: Imperial College Press.Google Scholar
13. Prieto-Rumeau, T. and Hernández-Lerma, O. (2012). Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results. Annals of Operations Research doi:10.1007/s10479-012-1184-4.Google Scholar
14. Reuter, G.E.H. (1957). Denumerable Markov processes and the associated contraction semigroups on l . Acta Mathematica 97: 1–46.Google Scholar
15. Spieksma, F.M. (2013). Kolmogorov forward equation and explosiveness in countable state Markov processes. Annals of Operations Research doi: 10.1007/s10479-012-1262-7.Google Scholar
16. Tweedie, R.L. (1974). Some ergodic properties of the Feller minimal process. Journal of Mathematics 25(2): 485–493.Google Scholar