Hostname: page-component-74656f9884-j7cmm Total loading time: 0 Render date: 2026-11-10T14:42:45.192Z Has data issue: false hasContentIssue false
  • English
  • Français

An iterative approximation scheme for repetitive Markov processes

Part of: Special processes Markov processes

Published online by Cambridge University Press:  14 July 2016

Tolga Tüfekçi  and
Refik Güllü
Tolga Tüfekçi*
Affiliation:
Middle East Technical University
Refik Güllü*
Affiliation:
Middle East Technical University
*
∗ Postal address: Industrial Engineering Department, Information Technologies and Electronics Research Institute, (BILTEN), Middle East Technical University, 06531, Ankara, Turkey.
∗ Postal address: Industrial Engineering Department, Information Technologies and Electronics Research Institute, (BILTEN), Middle East Technical University, 06531, Ankara, Turkey.
Get access Rights & Permissions [Opens in a new window]

Abstract

Repetitive Markov processes form a class of processes where the generator matrix has a particular repeating form. Many queueing models fall in this category such as M/M/1 queues, quasi-birth-and-death processes, and processes with M/G/1 or GI/M/1 generator matrices. In this paper, a new iterative scheme is proposed for computing the stationary probabilities of such processes. An infinite state process is approximated by a finite state process by lumping an infinite number of states into a super-state. What we call the feedback rate, the conditional expected rate of flow from the super-state to the remaining states, given the process is in the super-state, is approximated simultaneously with the steady state probabilities. The method is theoretically developed and numerically tested for quasi-birth-and-death processes. It turns out that the new concept of the feedback rate can be effectively used in computing the stationary probabilities.

Keywords

Quasi-birth-and-death processesapproximate solutionsnumerical algorithms

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 60J05: Discrete-time Markov processes on general state spaces 60K25: Queueing theory

Information

Type
Research Papers
Information
Journal of Applied Probability , Volume 36 , Issue 3 , September 1999 , pp. 654 - 667
Copyright
Copyright © Applied Probability Trust 1999 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Akar, N., and Sohraby, K. (1997). An invariant subspace approach in M/G/1 and G/M/1 type Markov chains. Commun. Stat.–Stochastic Models 13, 212–229.Google Scholar
Berman, A., and Pelmmons, R. (1979). Nonnegative Matrices in The Mathematical Sciences. Academic Press, New York.Google Scholar
Daigle, J. N., and Lucantoni, D. M. (1991). Queueing systems having phase-dependant arrival and service rates. In Numerical Solutions of Markov Chains, ed. Stewart, W. Marcel Dekker, New York, pp. 161–203.Google Scholar
Gail, H. R., Hantler, S., and Taylor, B. (1996). Spectral analysis of M/G/1 and G/M/1 type Markov chains. Adv. Appl. Prob. 28, 114–165.Google Scholar
Grassmann, W. K. (1990). Computational methods in probability theory. In Handbooks in OR & MS, ed. Heyman, D. and Sobel, M. J., Vol. 2. North-Holland, New York.Google Scholar
Gun, L. (1989). Experimental techniques on matrix-analytical solution techniques-extensions and comparisons. Commun. Statist–Stochastic Models 5, 669–682.Google Scholar
Horn, R. A., and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press, Cambridge.Google Scholar
Kant, K. (1992). Introduction to Computer Systems Performance Evaluation. McGraw-Hill, New York.Google Scholar
Kao, E., and Lin, C. (1990). A matrix-geometric solution of the jockeying problem. Eur. J. Operat. Res. 44, 67–74.Google Scholar
Latouche, G., and Ramaswami, V. (1993). A logarithmic reduction algorithm for quasi-birth-and-death processes. J. Appl. Prob. 30, 650–674.Google Scholar
Miller, D. R. (1981). Computation of steady-state probabilities for M/M/1 priority queues. Operat. Res. 29, 945–958.Google Scholar
Mitrani, I., and Chakka, R. (1995). Spectral expansion solution for a class of Markov models: application and comparison with the matrix-geometric method. Perf. Eval. 23, 241–260.Google Scholar
Neuts, F. M. (1981). Matrix-Geometric Solutions in Stochastic Models. John Hopkins University Press, Baltimore, MD.Google Scholar
Walrand, J. (1988). An Introduction to Queueing Networks. Prentice Hall, Englewood Cliffs, NJ.Google Scholar
Wartenhorst, P. (1995). N parallel queueing systems with server breakdown and repair. Eur. J. Operat. Res. 82, 302–322.Google Scholar