Hostname: page-component-7b7567c8c-tnkh4 Total loading time: 0 Render date: 2026-11-16T07:25:49.057Z Has data issue: false hasContentIssue false

Discounted Continuous-Time Controlled Markov Chains: Convergence of Control Models

Published online by Cambridge University Press:  30 January 2018

Tomás Prieto-Rumeau*
Affiliation:
Universidad Nacional de Educación a Distancia
Onésimo Hernández-Lerma*
Affiliation:
CINVESTAV-IPN
*
∗ Postal address: Departamento de Estadística, Facultad de Ciencias, Universidad Nacional de Educación a Distancia, Calle Senda del Rey 9, 28040, Madrid, Spain. Email address: tprieto@ccia.uned.es
∗∗ Postal address: Departamento de Matemáticas, CINVESTAV-IPN, México D.F. 07000, México.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

We are interested in continuous-time, denumerable state controlled Markov chains (CMCs), with compact Borel action sets, and possibly unbounded transition and reward rates, under the discounted reward optimality criterion. For such CMCs, we propose a definition of a sequence of control models {ℳn } converging to a given control model ℳ, which ensures that the discount optimal reward and policies of ℳn converge to those of ℳ. As an application, we propose a finite-state and finite-action truncation technique of the original control model ℳ, which is illustrated by approximating numerically the optimal reward and policy of a controlled population system with catastrophes. We study the corresponding convergence rates.

Information

Type
Research Article
Copyright
© Applied Probability Trust