Hostname: page-component-7b7567c8c-2jgqx Total loading time: 0 Render date: 2026-11-19T10:21:01.135Z Has data issue: false hasContentIssue false

Perturbation analysis of inhomogeneous finite Markov chains

Published online by Cambridge University Press:  24 March 2016

Bernd Heidergott*
Affiliation:
Vrije Universiteit Amsterdam
Haralambie Leahu*
Affiliation:
University of Amsterdam
Andreas Löpker*
Affiliation:
Helmut Schmidt University
Georg Pflug*
Affiliation:
University of Vienna
*
* Postal address: Korteweg-de-Vries Institute for Mathematics, University of Amsterdam, Science Park 107, Postbus 94248, 1090 GE Amsterdam, The Netherlands. Email address: haralambie@gmail.com
** Postal address: Department of Economics and Social Sciences, Helmut Schmidt University, Hamburg, 22008, Germany. Email address: lopker@hsu-hh.de
*** Postal address: Department of Statistics and Operations Research, University of Vienna, Oskar-Morgenstern Platz 1, Vienna, 1090, Austria. Email address: georg.pflug@univie.ac.at
**** Postal address: Department of Econometrics and Operations Research and Tinbergen Institute, Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands. Email address: b.f.heidergott@vu.nl
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.

Information

Type
Research Article
Copyright
Copyright © Applied Probability Trust 2016