Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-17T15:28:41.866Z Has data issue: false hasContentIssue false

The variance of discounted Markov decision processes

Published online by Cambridge University Press:  14 July 2016

Matthew J. Sobel*
Affiliation:
Georgia Institute of Technology
*
∗ Postal address: College of Management, Georgia Institute of Technology, Atlanta, GA 30332, U.S.A.

Abstract

Formulae are presented for the variance and higher moments of the present value of single-stage rewards in a finite Markov decision process. Similar formulae are exhibited for a semi-Markov decision process. There is a short discussion of the obstacles to using the variance formula in algorithms to maximize the mean minus a multiple of the standard deviation.

Information

Type
Research Papers
Copyright
Copyright © Applied Probability Trust 1982 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable