Hostname: page-component-6898cfcb55-zkb6p Total loading time: 0 Render date: 2026-11-11T19:30:45.805Z Has data issue: false hasContentIssue false
  • English
  • Français

Markov Processes with Restart

Part of:Markov processesGroups and semigroups of linear operators, their generalizations and applications

Published online by Cambridge University Press:  30 January 2018

Konstantin Avrachenkov,
Alexey Piunovskiy and
Yi Zhang
Konstantin Avrachenkov*
Affiliation:
Inria Sophia Antipolis
Alexey Piunovskiy*
Affiliation:
The University of Liverpool
Yi Zhang*
Affiliation:
The University of Liverpool
*
∗ Postal address: Inria Sophia Antipolis, 2004 Route des Lucioles, Sophia Antipolis, 06902, France. Email address: k.avrachenkov@sophia.inria.fr
∗∗ Postal address: Department of Mathematical Sciences, University of Liverpool, M&O Building, L69 7ZL, UK.
∗∗ Postal address: Department of Mathematical Sciences, University of Liverpool, M&O Building, L69 7ZL, UK.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

We consider a general homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such processes comes from modeling human and animal mobility patterns, restart processes in communication protocols, and from application of restarting random walks in information retrieval. We provide a connection between the transition probability functions of the original Markov process and the modified process with restarts. We give closed-form expressions for the invariant probability measure of the modified process. When the process evolves on the Euclidean space, there is also a closed-form expression for the moments of the modified process. We show that the modified process is always positive Harris recurrent and exponentially ergodic with the index equal to (or greater than) the rate of restarts. Finally, we illustrate the general results by the standard and geometric Brownian motions.

Keywords

Markov process with restartpositive Harris recurrenceexponential ergodicitystandard and geometric Brownian motions

MSC classification

Primary:60J25: Continuous-time Markov processes on general state spaces
Secondary:47D07: Markov semigroups and applications to diffusion processes

Information

Type
Research Article
Information
Journal of Applied Probability , Volume 50 , Issue 4 , December 2013, pp. 960 - 968
Copyright
© Applied Probability Trust 

References

Alt, H., et al. (1996). A method for obtaining randomized algorithms with small tail probabilities. Algorithmica 16, 543–547.CrossRefGoogle Scholar
Avrachenkov, K. E., Filar, J. and Haviv, M. (2002). Singular perturbations of Markov chains and decision processes. In Handbook of Markov Decision Processes, eds Feinberg, E. A. and Shwartz, A., Kluwer, Boston, MA, pp. 113–150.CrossRefGoogle Scholar
Brin, S. and Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Comput. Networks ISDN Systems 30, 107–117.CrossRefGoogle Scholar
Chung, F. (2007). The heat kernel as the pagerank of a graph. Proc. Nat. Acad. Sci. 104, 19735–19740.CrossRefGoogle Scholar
Down, D., Meyn, S. P. and Tweedie, R. L. (1995). Exponential and uniform ergodicity of Markov processes. Ann. Prob. 23, 1671–1691.CrossRefGoogle Scholar
Feller, W. (1971). An Introduction to Probability Theory and Its Applications, Vol. II, 2nd edn. John Wiley, New York.Google Scholar
Glynn, P. W. (1994). Some topics in regenerative steady-state simulation. Acta Appl. Math. 34, 225–236.CrossRefGoogle Scholar
González, M. C., Hidalgo, C. A. and Barabási, A.-L. (2008). Understanding individual human mobility patterns. Nature 453, 779–782.CrossRefGoogle ScholarPubMed
Hernández-Lerma, O. and Lasserre, J.-B. (1996). Discrete-time Markov Control Processes, Springer, New York.CrossRefGoogle Scholar
Krishnamurthy, B. and Rexford, J. (2001). Web Protocols and Practice: HTTP/1.1, Networking Protocols, Caching, and Traffic Measurement. Addison Wesley.Google Scholar
Kuznetsov, S. E. (1980). Any Markov process in a Borel space has a transition function. Theory Prob. Appl. 25, 384–388.CrossRefGoogle Scholar
Luby, M., Sinclair, A. and Zuckerman, D. (1993). Optimal speedup of Las Vegas algorithms. Inf. Process. Lett. 47, 173–180.CrossRefGoogle Scholar
Maurer, S. M. and Huberman, B. A. (2001). Restart strategies and Internet congestion. J. Econom. Dynamics Control 25, 641–654.CrossRefGoogle Scholar
Meyn, S. P. and Tweedie, R. L. (1993). Stability of Markov processes III. Forster-Lyapunov criteria for continuous-time processes. Adv. Appl. Prob. 25, 518–548.CrossRefGoogle Scholar
Ross, S. M. (1996). Stochastic Processes, 2nd edn. John Wiley, New York.Google Scholar
Stevens, W. R. (1994). TCP/IP Illustrated, Volume 1: The Protocols. Addison Wesley.Google Scholar
Walsh, P. D., Boyer, D. and Crofoot, M. C. (2010). Monkey and cell-phone-user mobilities scale similarly. Nature Phys. 6, 929–930.CrossRefGoogle Scholar