Hostname: page-component-74656f9884-npj9w Total loading time: 0 Render date: 2026-11-10T07:56:12.543Z Has data issue: false hasContentIssue false
  • English
  • Français

A fast algorithm for the two dimensionalHJB equation of stochastic control

Published online by Cambridge University Press:  15 August 2004

J. Frédéric Bonnans ,
Élisabeth Ottenwaelter  and
Housnaa Zidani
J. Frédéric Bonnans
Affiliation:
Projet Sydoco, Inria-Rocquencourt, Domaine de Voluceau, BP 105, 78153 Le Chesnay, France. Frederic.Bonnans@inria.fr.
Élisabeth Ottenwaelter
Affiliation:
IUT de Paris and Projet Sydoco, Inria-Rocquencourt, Domaine de Voluceau, BP 105, 78153 Le Chesnay, France. Elisabeth.Ottenwaelter@inria.fr.
Housnaa Zidani
Affiliation:
Projet Sydoco, Inria-Rocquencourt and Unité de Mathématiques Appliquées, ENSTA, 32 Boulevard Victor, 75739 Paris Cedex 15, France. zidani@ensta.fr.
Get access

Abstract

This paper analyses the implementation of the generalizedfinite differences method for the HJB equation of stochastic control, introduced by two of the authors in [Bonnans and Zidani,SIAM J. Numer. Anal.41 (2003) 1008–1021]. The computation of coefficients needs tosolve at each point of the grid (and for each control)a linear programming problem.We show here that, for two dimensional problems, this linear programming problem can be solved in O(p max) operations, where p max is the size of the stencil. The method is based on a walk on the Stern-Brocot tree,and on the related filling of the set of positive semidefinite matrices of size two.

Keywords

Stochastic controlfinite differencesviscosity solutionsconsistencyHJB equationStern-Brocot tree.

Information

Type
Research Article
Information
ESAIM: Mathematical Modelling and Numerical Analysis , Volume 38 , Issue 4 , July 2004 , pp. 723 - 735
Copyright
© EDP Sciences, SMAI, 2004

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Barles, G. and Jakobsen, E.R., On the convergence rate of approximation schemes for Hamilton-Jacobi-Bellman equations. ESAIM: M2AN 36 (2002) 33–54. CrossRef
G. Barles and E.R. Jakobsen, Error bounds for monotone approximation schemes for Hamilton-Jacobi-Bellman equations (to appear).
Barles, G. and Souganidis, P.E., Convergence of approximation schemes for fully nonlinear second order equations. Asymptotic Analysis 4 (1991) 271–283.
Bonnans, J.F. and Zidani, H., Consistency of generalized finite difference schemes for the stochastic HJB equation. SIAM J. Numer. Anal. 41 (2003) 1008–1021. CrossRef
J.F. Bonnans, E. Ottenwaelter and H. Zidani, A fast algorithm for the two dimensional HJB equation of stochastic control. Technical report, INRIA (2004). Rapport de Recherche 5078.
Camilli, F. and Falcone, M., An approximation scheme for the optimal control of diffusion processes. RAIRO Modél. Math. Anal. Numér. 29 (1995) 97–122. CrossRef
W.H. Fleming and H.M. Soner, Controlled Markov processes and viscosity solutions. Springer, New York (1993).
R.L. Graham, D.E. Knuth and O. Patashnik, Concrete Mathematics, A Foundation For Computer Science. Addison-Wesley, Reading, MA (1994). Second edition.
Jakobsen, E.R. and Karlsen, K.H., Continuous dependence estimates for viscosity solutions of fully nonlinear degenerate parabolic equations. J. Differ. Equations 183 (2002) 497–525. CrossRef
Krylov, N.V., On the rate of convergence of finite-difference approximations for Bellman's equations with variable coefficients. Probab. Theory Related Fields 117 (2000) 1–16. CrossRef
H.J. Kushner, Probability methods for approximations in stochastic control and for elliptic equations. Academic Press, New York (1977). Math. Sci. Engrg. 129.
H.J. Kushner and P.G. Dupuis, Numerical methods for stochastic control problems in continuous time. Springer, New York, Appl. Math. 24 (2001). Second edition.
Lions, P.-L., Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. Part 2: Viscosity solutions and uniqueness. Comm. Partial Differential Equations 8 (1983) 1229–1276. CrossRef
Lions, P.-L. and Mercier, B., Approximation numérique des équations de Hamilton-Jacobi-Bellman. RAIRO Anal. Numér. 14 (1980) 369–393.
Menaldi, J.-L., Some estimates for finite difference approximations. SIAM J. Control Optim. 27 (1989) 579–607. CrossRef