Hostname: page-component-7b7567c8c-fvjsw Total loading time: 0 Render date: 2026-11-14T22:35:29.893Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal stationary policies for denumerable Markov chains in continuous time

Published online by Cambridge University Press:  01 July 2016

John Bather
John Bather*
Affiliation:
University of Sussex
Get accessRights & Permissions [Opens in a new window]

Abstract

This paper is concerned with the problem of selecting the transition intensities for a Markov chain in continuous time so as to minimise the long-term average cost. Sufficient conditions are established for an optimal stationary policy using unbounded solutions of the optimality equation. This is a development of recent work on Markovian decision processes in discrete time. The theory is illustrated by considering a simple birth and death process with controlled immigration.

Keywords

MARKOVIAN DECISION PROCESSMINIMUM AVERAGE COSTCONTINUOUS TIMEDYNAMIC PROGRAMMINGOPTIMALITY EQUATION

Information

Type
Research Article
Information
Advances in Applied Probability , Volume 8 , Issue 1 , March 1976, pp. 144 - 158
Copyright
Copyright © Applied Probability Trust 1976 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

[1] Bather, J. (1973) Optimal decision procedures for finite Markov chains. Adv. Appl. Prob. 5, 328–540, 541–553.CrossRefGoogle Scholar
[2] Derman, C. (1966) Denumerable state Markovian decision processes—average cost criterion. Ann. Math. Statist. 37, 1545–1554.Google Scholar
[3] Feller, W. (1966) An Introduction to Probability Theory and its Applications, Vol. II. Wiley, New York.Google Scholar
[4] Hordijk, A. (1974) Dynamic programming and Markov potential theory. Mathematical Centre Tracts. No. 51. Amsterdam.Google Scholar
[5] Howard, R. A. (1960) Dynamic Programming and Markov Processes. Wiley, New York.Google Scholar
[6] Miller, B. L. (1968) Finite state continuous time Markov decision processes with a finite planning horizon. SIAM J. Control 6, 266–280.Google Scholar
[7] Miller, B. L. (1968) Finite state continuous time Markov decision processes with an infinite planning horizon. J. Math. Anal. Appl. 22, 552–569.Google Scholar
[8] Robinson, D. R. (1976) Markov decision chains with unbounded costs and applications to the control of queues. Adv. Appl. Prob. 8, 159–176.Google Scholar
[9] Ross, S. M. (1970) Average cost semi-Markov decision processes. J. Appl. Prob. 7, 649–656.Google Scholar