Hostname: page-component-7b7567c8c-8p88z Total loading time: 0 Render date: 2026-11-19T18:58:16.532Z Has data issue: false hasContentIssue false

Finite-horizon dynamic optimisation when the terminal reward is a concave functional of the distribution of the final state

Published online by Cambridge University Press:  01 July 2016

E. J. Collins*
Affiliation:
Bristol University
J. M. McNamara*
Affiliation:
Bristol University
*
∗ Postal address: Department of Mathematics, University of Bristol, University Walk, Bristol BS8 1TW, UK.
∗ Postal address: Department of Mathematics, University of Bristol, University Walk, Bristol BS8 1TW, UK.

Abstract

We consider a problem similar in many respects to a finite horizon Markov decision process, except that the reward to the individual is a strictly concave functional of the distribution of the state of the individual at final time T. Reward structures such as these are of interest to biologists studying the fitness of different strategies in a fluctuating environment. The problem fails to satisfy the usual optimality equation and cannot be solved directly by dynamic programming. We establish equations characterising the optimal final distribution and an optimal policy π*. We show that in general π* will be a Markov randomised policy (or equivalently a mixture of Markov deterministic policies) and we develop an iterative, policy improvement based algorithm which converges to π*. We also consider an infinite population version of the problem, and show that the population cannot do better using a coordinated policy than by each individual independently following the individual optimal policy π*.

Information

Type
General Applied Probability
Copyright
Copyright © Applied Probability Trust 1998 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable