Hostname: page-component-7b7567c8c-mnklg Total loading time: 0 Render date: 2026-11-12T05:12:08.900Z Has data issue: false hasContentIssue false
  • English
  • Français

Discrete-Time Semi-Markov Random Evolutions and their Applications

Part of:Probability theory on algebraic and topological structuresLimit theoremsSpecial processes

Published online by Cambridge University Press:  04 January 2016

Nikolaos Limnios and
Anatoliy Swishchuk
Nikolaos Limnios*
Affiliation:
Université de Technologie de Compiègne
Anatoliy Swishchuk*
Affiliation:
University of Calgary
*
∗ Postal address: Laboratoire de Mathématiques Appliquées, Université de Technologie de Compiègne, BP 20529, 60205 Compiègne Cedex, France. Email address: nikolaos.limnios@utc.fr
∗∗ Postal address: Department of Mathematics and Statistics, University of Calgary, 2500 University Drive NW, Calgary, Alberta T2N 1N4, Canada.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

In this paper we introduce discrete-time semi-Markov random evolutions (DTSMREs) and study asymptotic properties, namely, averaging, diffusion approximation, and diffusion approximation with equilibrium by the martingale weak convergence method. The controlled DTSMREs are introduced and Hamilton–Jacobi–Bellman equations are derived for them. The applications here concern the additive functionals (AFs), geometric Markov renewal chains (GMRCs), and dynamical systems (DSs) in discrete time. The rates of convergence in the limit theorems for DTSMREs and AFs, GMRCs, and DSs are also presented.

Keywords

Semi-Markov chainrandom evolutionaveragingdiffusion approximationdiffusion approximation with equilibriumrates of convergenceadditive functionalgeometric Markov renewal processdynamical systemcontrolled random evolutioncontrolled additive functional and geometric Markov renewal chainoptimal controlHamilton–Jacobi–Bellman equation

MSC classification

Primary:60K15: Markov renewal processes, semi-Markov processes60B10: Convergence of probability measures90C40: Markov and semi-Markov decision processes60F17: Functional limit theorems; invariance principles
Secondary:60K37: Processes in random environments

Information

Type
General Applied Probability
Information
Advances in Applied Probability , Volume 45 , Issue 1 , March 2013, pp. 214 - 240
Copyright
© Applied Probability Trust 

References

Aase, K. K. (1988). Contingent claims valuation when the security price is a combination of an Itô process and a random point process. Stoch. Process. Appl. 28, 185–220.CrossRefGoogle Scholar
Adams, R. (1979). Sobolev Spaces. Academic Press, New York.Google Scholar
Altman, E. and Shwartz, A. (1991). Markov decision problems and state-action frequencies. SIAM J. Control Optimization 29, 786–809.CrossRefGoogle Scholar
Anisimov, V. V. (2008). Switching Processes in Queueing Models. John Wiley, Hoboken, NJ.CrossRefGoogle Scholar
Barbu, V. S. and Limnios, N. (2008). Semi-Markov Chains and Hidden Semi-Markov Models. Toward Applications (Lecture Notes Statist. 191). Springer, New York.Google Scholar
Bertsekas, D. P. and Shreve, S. E. (1996). Stochastic Optimal Control. Athena Scientific, Belmont, MA.Google Scholar
Beutler, F. J. and Ross, K. W. (1985). Optimal policies for controlled Markov chains with a constraint. J. Math. Anal. Appl. 112, 236–252.CrossRefGoogle Scholar
Borkar, V. S. (2003). Dynamic programming for ergodic control with partial observations. Stoch. Process. Appl. 103, 293–310.CrossRefGoogle Scholar
Boussement, M. and Limnios, N. (2004). Markov decision processes with asymptotic average failure rate constraint. Commun. Statist. Theory Meth. 33, 1689–1714.CrossRefGoogle Scholar
Chiquet, J., Limnios, N. and Eid, M. (2009). Piecewise deterministic Markov processes applied to fatigue crack growth modelling. J. Statist. Planning Infer. 139, 1657–1667.CrossRefGoogle Scholar
Cox, J. C., Ross, S. A. and Rubinstein, M. (1979). Option pricing: a simplified approach. J. Financial Econom. 7, 229–263.CrossRefGoogle Scholar
Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. John Wiley, New York.CrossRefGoogle Scholar
Fleming, W. H. and Rishel, R. W. (1975). Deterministic and Stochastic Optimal Control. Springer, Berlin.CrossRefGoogle Scholar
Hersh, R. (1974). Random evolutions: a survey of results and problems. Rocky Mountain J. Math. 4, 443–477.CrossRefGoogle Scholar
Jacod, J. and Shiryaev, A. N. (1987). Limit Theorems for Stochastic Processes. Springer, Berlin.CrossRefGoogle Scholar
Jaśkiewicz, A. and Nowak, A. S. (2007). Average optimality for semi-Markov control processes. Morfismos 11, 15–36.Google Scholar
Keepler, M. (1998). Random evolutions processes induced by discrete time Markov chains. Portugal. Math. 55, 391–400.Google Scholar
Koroliuk, V. S. and Limnios, N. (2005). Stochastic Systems in Merging Phase Space. World Scientific, Hackensack, NJ.CrossRefGoogle Scholar
Korolyuk, V. S. and Swishchuk, A. (1995). Evolution of Systems in Random Media. CRC Press, Boca Raton, FL.Google Scholar
Krylov, N. and Bogolyubov, N. (1947). Introduction to Non-Linear Mechanics. Princeton University Press, Princeton.Google Scholar
Kushner, H. (1971). Introduction to Stochastic Control. Holt, Rinehart and Winston, New York.Google Scholar
Ledoux, M. and Talangrand, M. (1991). Probability in Banach Spaces. Springer, Berlin.CrossRefGoogle Scholar
Limnios, N. (2011). Discrete-time semi-Markov random evolutions—average and diffusion approximation of difference equations and additive functionals. Commun. Statist. Theory Meth. 40, 3396–3406.CrossRefGoogle Scholar
Limnios, N. and Oprişan, G. (2001). Semi-Markov Processes and Reliability. Birkhäuser, Boston, MA.CrossRefGoogle Scholar
Maxwell, M. and Woodroofe, M. (2000). Central limit theorems for additive functionals of Markov chains. Ann. Prob. 28, 713–724.CrossRefGoogle Scholar
Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.CrossRefGoogle Scholar
Nummelin, E. (1984). General Irreducible Markov Chains and Nonnegative Operators. Cambridge University Press.CrossRefGoogle Scholar
Pinsky, M. A. (1991). Lectures on Random Evolution. World Scientific, River Edge, NJ.CrossRefGoogle Scholar
Pyke, R. (1961). Markov renewal processes: definitions and preliminary properties. Ann. Math. Statist. 32, 1231–1242.CrossRefGoogle Scholar
Pyke, R. (1961). Markov renewal processes with finitely many states. Ann. Math. Statist. 32, 1243–1259.CrossRefGoogle Scholar
Pyke, R. and Schaufele, R. (1964). Limit theorems for Markov renewal processes. Ann. Math. Statist. 35, 1746–1764.CrossRefGoogle Scholar
Revuz, D. (1975). Markov Chains. North-Holland, Amsterdam.Google Scholar
Rudin, W. (1991). Functional Analysis. McGraw-Hill, New York.Google Scholar
Shurenkov, V. M. (1984). On the theory of Markov renewal. Theory Prob. Appl. 19, 247–265.Google Scholar
Silvestrov, D. S. (1991). The invariance principle for accumulation processes with semi-Markov switchings in a scheme of arrays. Theory Prob. Appl. 36, 519–535.CrossRefGoogle Scholar
Silvestrov, D. S. (2004). Limit Theorems for Randomly Stopped Stochastic Processes. Springer, London.CrossRefGoogle Scholar
Skorokhod, A. V. (1989). Asymptotic Methods in the Theory of Stochastic Differential Equations. American Mathematical Society, Providence, RI.Google Scholar
Skorokhod, A. V., Hoppensteadt, F. C. and Salehi, H. (2002). Random Perturbation Methods with Applications in Science and Engineering. Springer, New York.CrossRefGoogle Scholar
Sobolev, S. L. (1991). Some Applications of Functional Analysis in Mathematical Physics. American Mathematical Society, Providence, RI.Google Scholar
Stroock, D. W. and Varadhan, S. R. S. (1979). Multidimensional Diffusion Processes. Springer, Berlin.Google Scholar
Sviridenko, M. N. (1989). Martingale approach to limit theorems for semi-Markov processes. Theory Prob. Appl. 34, 540–545.CrossRefGoogle Scholar
Swishchuk, A. V. (1995). Random Evolutions and Their Applications. Kluwer, Dordrecht.Google Scholar
Swishchuk, A. and Islam, M. S. (2010). The geometric Markov renewal processes with application to finance. Stoch. Anal. Appl. 29, 684–705.CrossRefGoogle Scholar
Swishchuk, A. and Islam, M. S. (2010). Diffusion approximations of the geometric Markov renewal processes and option price formulas. Internat. J. Stoch. Anal. 2010, 347105, 21 pp.Google Scholar
Swishchuk, A. V. and Islam, M. S. (2013). Normal deviation and Poisson approximation of GMRP. To appear in Commun. Statist. Theory Meth.Google Scholar
Swishchuk, A. V. and Limnios, N. (2011). Optimal stopping of GMRP and pricing of European and American options. In Proc. 15th Internat. Congress on Insurance: Mathematics and Economics (Trieste, Italy, June 2011).Google Scholar
Swishchuk, A. and Wu, J. (2003). Evolution of Biological Systems in Random Media: Limit Theorems and Stability. Kluwer, Dordrecht.CrossRefGoogle Scholar
Vega-Amaya, O. and Luque-Vásquez, F. (2000). Sample-path average cost optimality for semi-Markov control processes on Borel spaces: unbounded costs and mean holding times. Appl. Math. 27, 343–367.Google Scholar
Yin, G. G. and Zhang, Q. (2005). Discrete-Time Markov Chains. Springer, New York.Google Scholar