Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-15T14:52:32.445Z Has data issue: false hasContentIssue false

An analysis of transient Markov decision processes

Published online by Cambridge University Press:  14 July 2016

Huw W. James*
Affiliation:
University of Bristol
E. J. Collins*
Affiliation:
University of Bristol
*
∗ Current address: Commerzbank Corporates and Markets, 60 Gracechurch Street, London EC3V 0HR, UK. Email address: huw.james@commerzbank.com
∗∗ Postal address: School of Mathematics, University of Bristol, University Walk, Bristol BS8 1TW, UK.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

This paper is concerned with the analysis of Markov decision processes in which a natural form of termination ensures that the expected future costs are bounded, at least under some policies. Whereas most previous analyses have restricted attention to the case where the set of states is finite, this paper analyses the case where the set of states is not necessarily finite or even countable. It is shown that all the existence, uniqueness, and convergence results of the finite-state case hold when the set of states is a general Borel space, provided we make the additional assumption that the optimal value function is bounded below. We give a sufficient condition for the optimal value function to be bounded below which holds, in particular, if the set of states is countable.

Information

Type
Research Article
Copyright
© Applied Probability Trust 2006