Hostname: page-component-74656f9884-npj9w Total loading time: 0 Render date: 2026-11-10T10:29:19.097Z Has data issue: false hasContentIssue false
  • English
  • Français

Countable-state-space Markov chains with two time scales and applications to queueing systems

Part of: Markov processes Asymptotic theory

Published online by Cambridge University Press:  01 July 2016

G. Yin  and
Hanqin Zhang
G. Yin*
Affiliation:
Wayne State University
Hanqin Zhang*
Affiliation:
Academia Sinica, Beijing
*
∗ Postal address: Department of Mathematics, Wayne State University, Detroit, MI 48202, USA. Email address: gyin@math.wayne.edu
∗∗ Postal address: Institute of Applied Mathematics, Academy of Mathematics and Systems Sciences, Academia Sinica, Beijing 100080, China.
Get access Rights & Permissions [Opens in a new window]

Abstract

Motivated by various applications in queueing systems, this work is devoted to continuous-time Markov chains with countable state spaces that involve both fast-time scale and slow-time scale with the aim of approximating the time-varying queueing systems by their quasistationary counterparts. Under smoothness conditions on the generators, asymptotic expansions of probability vectors and transition probability matrices are constructed. Uniform error bounds are obtained, and then sequences of occupation measures and their functionals are examined. Mean square error estimates of a sequence of occupation measures are obtained; a scaled sequence of functionals of occupation measures is shown to converge to a Gaussian process with zero mean. The representation of the variance of the limit process is also explicitly given. The results obtained are then applied to treat Mt /Mt /1 queues and Markov-modulated fluid buffer models.

Keywords

Markov chainsingular perturbationcountable state spaceasymptotic expansionoccupation measurefunctional central limit theoremMt /Mt /1 queuefluid model

MSC classification

Primary: 60J27: Continuous-time Markov processes on discrete state spaces 34E05: Asymptotic expansions

Information

Type
General Applied Probability
Information
Advances in Applied Probability , Volume 34 , Issue 3 , September 2002 , pp. 662 - 688
Copyright
Copyright © Applied Probability Trust 2002 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

[1] Asmussen, S. and Bladt, M. (1994). A sample path approach to mean busy periods for Markov-modulated queues and fluids. Adv. Appl. Prob. 26, 1117–1121.CrossRefGoogle Scholar
[2] Billingsley, P. (1968). Convergence of Probability Measures. John Wiley, New York.Google Scholar
[3] Bogoliubov, N. N. and Mitropolskii, Y. A. (1961). Asymptotic Methods in the Theory of Nonlinear Oscillator. Gordon and Breach, New York.Google Scholar
[4] Doob, J. L. (1990). Stochastic Processes. John Wiley, New York.Google Scholar
[5] Eick, S. G., Massey, W. A. and Whitt, W. (1993). The physics of the M_t/G/∞ queue. Operat. Res. 41, 731–742.CrossRefGoogle Scholar
[6] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence. John Wiley, New York.Google Scholar
[7] Harrison, J. M. (1985). Brownian Motions and Stochastic Flow Systems. John Wiley, New York.Google Scholar
[8] Hutson, V. and Pym, J. S. (1980). Applications of Functional Analysis and Operator Theory. Academic Press, London.Google Scholar
[9] Ilin, A. M. (1992). Matching of Asymptotic Expansions of Solutions of Boundary Value Problems (Trans. Math. Monogr. 102). American Mathematical Society, Providence, RI.Google Scholar
[10] Ilin, A. M., Khasminskii, R. Z. and Yin, G. (1999). Singularly perturbed switching diffusions: rapid switchings and fast diffusions. J. Optim. Theory Appl. 102, 555–591.Google Scholar
[11] Khasminskii, R. Z. (1966). On stochastic processes defined by differential equations with a small parameter. Theory Prob. Appl. 11, 211–228.Google Scholar
[12] Khasminskii, R. Z., Yin, G. and Zhang, Q. (1996). Asymptotic expansions of singularly perturbed systems involving rapidly fluctuating Markov chains. SIAM J. Appl. Math. 56, 277–293.Google Scholar
[13] Knessl, C., Matkowsky, B. J., Schuss, Z. and Tier, C. (1986). Asymptotic analysis of a state-dependent M/G/1 queueing system. SIAM J. Appl. Math. 46, 483–505.CrossRefGoogle Scholar
[14] Kushner, H. J. (1984). Approximation and Weak Convergence Methods for Random Processes, With Applications to Stochastic Systems Theory. MIT Press, Cambridge, MA.Google Scholar
[15] Ladas, G. E. and Lakshmikantham, V. (1972). Differential Equations in Abstract Spaces. Academic Press, New York.Google Scholar
[16] Massey, W. (1985). Asymptotic analysis of the time dependent M/M/1 queue. Math. Operat. Res. 10, 305–327.Google Scholar
[17] Massey, W. A. and Whitt, W. (1998). Uniform acceleration expansions for Markov chains with time-varying rates. Ann. Appl. Prob. 8, 1130–1155.CrossRefGoogle Scholar
[18] Neuts, M. F. (1981). Matrix-Geometric Solutions in Stochastic Models. The Johns Hopkins University Press, Baltimore, MD.Google Scholar
[19] O'Malley, R. E. Jr (1991). Singular Perturbation Methods for Ordinary Differential Equations. Springer, New York.Google Scholar
[20] Pan, Z. G. and Başar, T. (1995). H ∞-control of Markovian jump linear systems and solutions to associated piecewise-deterministic differential games. In New Trends in Dynamic Games and Applications, ed. Olsder, G. J.. Birkhäuser, Boston, MA, pp. 61–94.Google Scholar
[21] Pazy, A. (1983). Semigroups of Linear Operators and Applications to Partial Differential Equations. Springer, New York.Google Scholar
[22] Saaty, T. L. (1961). Elements of Queueing Theory. McGraw-Hill, New York.Google Scholar
[23] Skorohod, A. V. (1982). Studies in the Theory of Random Processes. Dover, New York.Google Scholar
[24] Vasileava, A. B. and Butuzov, V. F. (1990). Asymptotic Methods in Singular Perturbations Theory. Vysshaya Shkola, Moscow (in Russian).Google Scholar
[25] Yin, G. and Zhang, Q. (1998). Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach. Springer, New York.Google Scholar
[26] Yin, G., Zhang, Q. and Badowski, G. (2000). Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states. Ann. Appl. Prob. 10, 549–572.Google Scholar