Hostname: page-component-7b7567c8c-kwt9d Total loading time: 0 Render date: 2026-11-14T17:44:38.508Z Has data issue: false hasContentIssue false
  • English
  • Français

Stability of Markovian processes II: continuous-time processes and sampled chains

Part of:Markov processes

Published online by Cambridge University Press:  01 July 2016

Sean P. Meyn and
R. L. Tweedie
Sean P. Meyn*
Affiliation:
University of Illinois
R. L. Tweedie*
Affiliation:
Colorado State University
*
∗ Postal address: University of Illinois, Coordinated Science Laboratory, 1101 W. Springfield Ave., Urbana, IL 61801, USA.
∗∗ Postal address: Department of Statistics, Colorado State University, Fort Collins, CO 80523, USA.
Get accessRights & Permissions [Opens in a new window]

Abstract

In this paper we extend the results of Meyn and Tweedie (1992b) from discrete-time parameter to continuous-parameter Markovian processes Φ evolving on a topological space.

We consider a number of stability concepts for such processes in terms of the topology of the space, and prove connections between these and standard probabilistic recurrence concepts. We show that these structural results hold for a major class of processes (processes with continuous components) in a manner analogous to discrete-time results, and that complex operations research models such as storage models with state-dependent release rules, or diffusion models such as those with hypoelliptic generators, have this property. Also analogous to discrete time, ‘petite sets', which are known to provide test sets for stability, are here also shown to provide conditions for continuous components to exist.

New ergodic theorems for processes with irreducible and countably reducible skeleton chains are derived, and we show that when these conditions do not hold, then the process may be decomposed into an uncountable orbit of skeleton chains.

Keywords

IRREDUCIBLE MARKOV PROCESSESERGODICITYRECURRENCERESOLVENTSDIFFUSIONS

MSC classification

Primary:60J10: Markov chains (discrete-time Markov processes on discrete state spaces)

Information

Type
Research Article
Information
Advances in Applied Probability , Volume 25 , Issue 3 , September 1993, pp. 487 - 517
Copyright
Copyright © Applied Probability Trust 1993 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

This work was commenced at Bond University, and developed there and at the Australian National University, the University of Illinois, and Colorado State University.

Work supported in part by NSF initiation grant #ECS 8910088.

References

References

[1] Asmussen, A. (1987) Applied Probability and Queues. Wiley, New York.Google Scholar
[2] Azema, J. Duflo, M. and Revuz, D. (1969) Propriétés relatives des processus de Markov récurrents. Z. Wahrscheinlichkeitsth. 13, 286–314.Google Scholar
[3] Azema, J. Kaplan-Duflo, M. and Revuz, D. (1967) Measure invariante sur les classes récurrentes des processus de Markov. Z. Wahrscheinlichkeitsth. 8, 157–181.CrossRefGoogle Scholar
[4] Beneš, V. E. (1968) Finite regular invariant measures for Feller processes. J. Appl. Prob. 5, 203–209.Google Scholar
[5] Blumenthal, R. M. and Getoor, R. K. (1968) Markov Processes and Potential Theory. Academic Press, New York.Google Scholar
[6] Bonami, A. Karoui, N.B. Roynette, , and Reinhard, H. (1971) Processus de diffusion associé un opérateur elliptique dégénéré. Ann. Inst. H. Poincaré VII, 31–80.Google Scholar
[7] Bony, J. M. (1969) Principe du maximum, inégalité de Harnack et unicité du probléme de Cauchy pour les opérateurs elliptique dégénérés. Ann. Inst. Fourier, Grenoble 19, 277–304.CrossRefGoogle Scholar
[8] Duflo, M. and Revuz, D. (1969) Propriétés asymptotiques des probabilités de transition des processus de Markov récurrents. Ann. Inst. H. Poincaré B V, 233–244.Google Scholar
[9] Getoor, R. K. (1979) Transience and recurrence of Markov processes. In Séminaire de Probabilités XIV, ed. Azéma, J. and Yor, M., pp. 397–409, Springer-Verlag, Berlin.Google Scholar
[10] Glynn, P. and Sigman, K. (1992) Uniform Cesaro limit theorems for synchronous processes with applications to queues. Stoch. Proc. Appl. 40, 29–43.Google Scholar
[11] Harrison, J. M. and Resnick, S. I. (1976) The stationary distribution and first exit probabilities of a storage process with general release rule. Math. Operat. Res. 1, 347–358.Google Scholar
[12] Harrison, J. M. and Resnick, S. I. (1978) The recurrence classification of risk and storage processes. Math. Operat. Res. 3, 57–66.Google Scholar
[13] Ichihara, K. and Kunita, H. (1974) A classification of the second order degenerate elliptic operators and its probabilistic characterization. Z. Wahrscheinlichkeitsth. 30, 235–254.CrossRefGoogle Scholar
[14] Kaspi, H. and Mandelbaum, A. (1992) On Harris recurrence in continuous time. Technical report, Technion-Israel Instiute of Technology. Submitted for publication.Google Scholar
[15] Khas'Minskii, R. Z. (1960) Ergodic properties of recurrent diffusion processes and stabilization of the solution to the Cauchy problem for parabolic equations. Theory Prob. Appl. 5, 179–195.Google Scholar
[16] Khas'Minskii, R. Z. (1980) Stochastic Stability of Differential Equations. Sijthoff & Noordhoff, Dordrecht, Netherlands.Google Scholar
[17] Kliemann, W. (1987) Recurrence and invariant measures for degenerate diffusions. Ann. Prob. 15, 690–707.CrossRefGoogle Scholar
[18] Kunita, H. (1978) Supports of diffusion processes and controllability problems. In Proc. Internat. Symp. Stochastic Differential Equations, ed. Itô, K., pp. 163–185, Wiley, New York.Google Scholar
[19] Kunita, H. (1990) Stochastic Flows and Stochastic Differential Equations. Cambridge University Press.Google Scholar
[20] Meyn, S. P. (1989) Ergodic theorems for discrete time stochastic systems using a stochastic Lyapunov function. SIAM J. Control Optim. 27, 1409–1439.CrossRefGoogle Scholar
[21] Meyn, S. P. and Caines, P. E. (1991) Asymptotic behavior of stochastic systems processing Markovian realizations. SIAM J. Control Optim. 29, 535–561.Google Scholar
[22] Meyn, S. P. and Tweedie, R. L. (1992a) Generalized resolvents and Harris recurrence of Markov processes. In 50 years after Doeblin: Developments in the Theory of Markov Chains, Markov Processes and Sums of Random Variables.CrossRefGoogle Scholar
[23] Meyn, S. P. and Tweedie, R. L. (1993a) Markov Chains and Stochastic Stability. Springer-Verlag, Berlin.CrossRefGoogle Scholar
[24] Meyn, S. P. and Tweedie, R. L. (1992b) Stability of Markovian processes I: discrete time chains. Adv. Appl. Prob. 24, 542–574.CrossRefGoogle Scholar
[25] Meyn, S. P. and Tweedie, R. L. (1993b) Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes. Adv. Appl. Prob. 25, 518–548.Google Scholar
[26] Nummelin, E. (1984) General Irreducible Markov Chains and Non-Negative Operators. Cambridge University Press.CrossRefGoogle Scholar
[27] Orey, S. (1959) Recurrent Markov chains. Pacific J. Math. 9, 805–827.Google Scholar
[28] Orey, S. (1971) Limit Theorems for Markov Chain Transition Probabilities. Van Nostrand Reinhold, London.Google Scholar
[29] Sharpe, M. (1988) General Theory of Markov Processes. Academic Press, New York.Google Scholar
[30] Tuominen, P. and Tweedie, R. L. (1979) Exponential decay and ergodicity of general Markov processes and their discrete skeletons. Adv. Appl. Prob. 11, 784–803.CrossRefGoogle Scholar
[31] Tuominen, P. and Tweedie, R. L. (1979a) Markov chains with continuous components. Proc. London Math. Soc. (3) 38, 89–114.Google Scholar
[32] Tuominen, P. and Tweedie, R. L. (1979b) The recurrence structure of general Markov processes. Proc. London Math. Soc. (3) 39, 554–576.Google Scholar
[33] Tweedie, R. L. (1975) Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space. Stoch. Proc. Appl. 3, 385–403.Google Scholar
[34] Tweedie, R. L. (1976) Criteria for classifying general Markov chains. Adv. Appl. Prob. 8, 737–771.Google Scholar
[35] Wonham, W. M. (1966) A Liapunov criteria for weak stochastic stability. J. Differential Equations 2, 195–207.Google Scholar

Reference added in proof

[36] Brankovan, M. (1973) Quelques propriétés des résolvantes récurrentes au sens de Harris. Ann. Inst. H. Poincaré B 9, 1–18.Google Scholar