2.1. Notation and set-up

Let $T>0$ be a deterministic finite time horizon and, for every $t \in [0,T]$ and $s\in [t,T] $, let $\Omega^{\omega}_{t,s}$ be the set of $\mathbb{R}^M$-valued continuous functions on [t, s] taking the value 0 at t, that is,

\begin{equation*}\Omega^{\omega}_{t,s} = \{ \omega \in C ([t,s];\,\mathbb{R}^M ) \colon \omega(t)=0 \} .\end{equation*}

Let $\mathcal{G}_{t,u}^{\omega}$ be the $\sigma$-algebra generated by paths $\omega \in \Omega^{\omega}_{t,s}$ up to some time $u\in [t,s] $ with $\mathbb{G}^{\omega}_{t,s} = \{ \mathcal{G}_{t,u}^{\omega} \colon u \in [t,s]\} $ being the corresponding filtration. When endowed with the Wiener measure $\mathbb{P}^{\omega}_{t,s}$ on $\mathcal{G}_{t,s}^{\omega}$, $\Omega^{\omega}_{t,s}$ becomes a classical Wiener space. Let $B^t= \{ B^t(s) \colon s\in [t,T], B^t(t)=0 \} $ be a Brownian motion on the filtered probability space $ (\Omega^{\omega}_{t,T}, \mathcal{G}_{t,T}^{\omega},\mathbb{G}^{\omega}_{t,T}, \mathbb{P}^{\omega}_{t,T}) $.

Let us introduce the following technical assumptions.

1. (A1) U and V are compact metric spaces.

2. (A2) The maps $f \colon [0,T]\ {\times}\ \mathbb{R}^N\ {\times}\ U\ {\times}\ V\ {\rightarrow}\ \mathbb{R}^N $, $\sigma \colon [0,T]\ {\times}\ \mathbb{R}^N\ {\times}\ U\ {\times}\ V\ {\rightarrow}\ \mathbb{R}^{N \times M}$, $\Psi \colon [0,T] \times \mathbb{R}^N \rightarrow \mathbb{R}$, and $L \colon [0,T] \times \mathbb{R}^N \times U \times V \rightarrow \mathbb{R}$ are bounded, uniformly continuous with respect to all its variables, and Lipschitz-continuous with respect to $ (t,x) \in [0,T]\times {\mathbb R}^N$ uniformly in $ (u,v) \in U \times V$.

3. (A3) $\tau$ is an (absolutely) continuous random variable (with respect to the Lebesgue measure on ${\mathbb R}^+=(0,+\infty)$) defined on a probability space $ (\Omega^\tau, \mathcal{G}^\tau,\mathbb{P}^\tau)$ and has a positive, bounded, and Lipschitz-continuous probability density function defined on ${\mathbb R}^+$.

4. (A4) For each $t\in [0,T]$, the random variable $\tau$ is independent of the filtration $\mathbb{G}_{t,T}^\omega$ generated by the Brownian motion $B^t({\cdot})$.

For $\skew3\hat{t} \in (t,T)$ and $\omega \in \Omega^\omega_{t,T}$, let

\begin{equation*}\omega^{t,\skew3\hat{t}} = \omega|_{[t,\skew3\hat{t}]} \quad\text{and}\quad \omega^{\skew3\hat{t},T} = \omega - \omega|_{[\skew3\hat{t},T]},\end{equation*}

and define $\pi \colon \Omega^\omega_{t,T} \rightarrow \Omega^\omega_{t,\skew3\hat{t}} \times \Omega^\omega_{\skew3\hat{t},T}$ to be the map given by

\begin{equation*} \pi (\omega) = (\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T}) .\end{equation*}

Then, $\pi$ induces the identification

\begin{equation*}\Omega^\omega_{t,T}= \Omega^\omega_{t,\skew3\hat{t}} \times \Omega^\omega_{\skew3\hat{t},T} {,}\end{equation*}

and the inverse of $\pi$ acts on pairs of paths $ (\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T})\in \Omega^\omega_{t,\skew3\hat{t}} \times \Omega^\omega_{\skew3\hat{t},T}$ by concatenation, i.e. $\omega = \pi^{-1}(\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T})\in\Omega^{\omega}_{t,T}$. Finally, note that ${\mathbb{P}}^\omega_{t,T} = {\mathbb{P}}^\omega_{t,\skew3\hat{t}} \otimes {\mathbb{P}}^\omega_{\skew3\hat{t},T}$, where ${\mathbb{P}}^\omega_{t,\skew3\hat{t}}$ and ${\mathbb{P}}^{\omega}_{\skew3\hat{t},T}$ are the Wiener measures on $\Omega^\omega_{t,\skew3\hat{t}}$ and $\Omega^\omega_{\skew3\hat{t},T}$, respectively.

For each $t\in[0,T]$, the probability measure ${\mathbb{P}}^\tau$ of the random variable $\tau$ induces a conditional probability measure on $\Omega_t^\tau=(t,\infty)$ determined by

\begin{equation*}\mathbb{P}_t^\tau(\tau\in A) = \mathbb{P}^\tau(\tau\in A \mid \tau > t ), \quad A\in\mathcal{G}^\tau_{t},\end{equation*}

where $\mathcal{G}^\tau_{t}={\mathcal B}(\Omega_t^\tau)$ denotes the Borel $\sigma$-algebra of $\Omega_t^\tau$. Resorting to assumption (A4) regarding independence between the Brownian motion $B^t$ and the random variable $\tau$, we define $\Omega_{t}$ as the direct product

\begin{equation*}\Omega_t = \Omega_{t,T}^\omega\times\Omega_t^\tau,\end{equation*}

defining accordingly the probability measure

(1) \begin{equation}\mathbb{P}_t={\mathbb{P}}^\omega_{t,T}\otimes {\mathbb{P}}^\tau_t,\end{equation}

the $\sigma$-algebras $\mathcal{G}_{t,s}$, $s\in[t,T]$, as the completion of $\mathcal{G}^\omega_{t,s}\otimes\mathcal{G}^\tau_{t}$ with respect to the measure $\mathbb{P}_t$, and the filtration $\mathbb{G}_{t,T}$ as $\mathbb{G}_{t,T}= \{\mathcal{G}_{t,s} \colon s\in [t,T] \}$.

2.2. A stochastic differential game with a random horizon

Let us define the random horizon $\xi$ as

(2) \begin{equation}\xi = \min\{\tau,T\} {,}\end{equation}

and note that $\xi$ takes values on the interval [0, T] ${\mathbb{P}}_t$-a.s. (almost surely). The two-player zero-sum stochastic differential game with random horizon is defined on the filtered probability space $ (\Omega_t, \mathcal{G}_{t,T}, \mathbb{G}_{t,T},\mathbb{P}_t)$ and consists of the controlled stochastic differential equation

(3) \begin{equation} \begin{aligned}\mathrm{d} X(s) & = f (s, X(s), u(s),v(s)) \, \mathrm{d} s + \sigma (s, X(s),u(s),v(s))\, \mathrm{d} B^t (s), \quad s \geq t {,} \\*X(t) &= x\end{aligned}\end{equation}

and payoff functional

(4) \begin{equation} J(t, x;\, u({\cdot}),v({\cdot})) = {{\mathbb E}}_{{\mathbb{P}}_t}\bigg[ \int_t^{\xi} L (s, X_{t,x}^{u,v}(s), u(s),v(s)) \, \mathrm{d} s + \Psi(\xi, X_{t,x}^{u,v}(\xi)) \bigg],\end{equation}

where $X_{t,x}^{u,v}(s)$, $s\in[t,T]$, denotes the solution of the initial value problem (3) associated with a specific choice of $u({\cdot}),v({\cdot})$. We will refer to the functions L and $\Psi$ determining the payoff functional J as the running payoff and terminal payoff, respectively. Thus, as far as the game is concerned, the payoff functional (4) represents some payoff that a first player (Player I) is trying to minimize (and thus a second player (Player II) seeks to maximize) subject to the state variable dynamics defined by (3) and some constraints of the form $u(s) \in U$ and $v(s) \in V$ for every appropriately defined instant of time $s\geq t$.

An admissible control process $u({\cdot})$ (resp. $v({\cdot})$) for Player I (resp. II) on [t, T] is a $\mathbb{G}^{\omega}_{t,T}$-progressively measurable process taking values in U (resp. V). The set of all admissible controls for Player I (resp. II) on [t, T] is denoted by $\mathcal{U}(t,T)$ (resp. $\mathcal{V}(t,T)$). We say that two controls $u_1({\cdot}), u_2({\cdot}) \in \mathcal{U}(t,T)$ are the same on [t, s], for some $s\in[t,T]$, and denote it by $u_1({\cdot}) \approx u_2({\cdot})$, if $\mathbb{P}^{\omega}_{t,T} \{u_1({\cdot}) = u_2({\cdot}) \ \text{{a.e. (almost everywhere)}} \ \text{in} \ [t,s]\} = 1$. A similar convention is used for elements of $\mathcal{V}(t,T)$.

An admissible strategy $\alpha$ (resp. $\beta$) for Player I (resp. II) on [t, T] is a mapping $\alpha \colon \mathcal{V}(t,T) \rightarrow \mathcal{U}(t,T)$ (resp. $\beta \colon \mathcal{U}(t,T) \rightarrow \mathcal{V}(t,T)$) such that if $v({\cdot}) \approx \tilde{v}({\cdot})$ (resp. $u({\cdot}) \approx \tilde{u}({\cdot})$) on [t, s] for every $s \in [t,T]$, then $\alpha [v({\cdot})] \approx \alpha [\tilde{v}({\cdot})]$ (resp. $\beta[u({\cdot})] \approx \beta [\tilde{u}({\cdot})]$). The set of all admissible strategies for Player I (resp. II) on [t, T] is denoted by $\mathcal{A}(t,T)$ (resp. $\mathcal{B}(t,T)$).

Let $ (t,x)\in[0,T]\times{\mathbb R}^N$. The lower value function of the stochastic differential game (SDG) with random horizon (3)–(4) is given by

(5) \begin{equation}V^-(t,x)=\inf_{\beta \in \mathcal{B}(t,T)} \sup_{u({\cdot}) \in \mathcal{U}(t,T)} J(t, x;\, u({\cdot}),\beta[u({\cdot})]) {,}\end{equation}

while the corresponding upper value function is

(6) \begin{equation}V^+(t,x)=\sup_{\alpha \in \mathcal{A}(t,T)} \inf_{v({\cdot}) \in \mathcal{V}(t,T)} J(t, x;\, \alpha[v({\cdot})], v({\cdot})) .\end{equation}

We say that the SDG with random horizon (3)–(4) has a value if $V^+(t,x)=V^-(t,x)$, and call it the common value of the SDG game. We also note that this definition is consistent with the standard notion of common value of the game introduced by Elliot and Kalton [Reference Elliott and Kalton15] for differential games with a deterministic horizon.

Choosing the controls at time t, the player who moves first (the maximizing player for the lower game, and the minimizing player for the upper game) is allowed to use the past of the Brownian motion $B^t({\cdot})$ driving (3), while the player with the advantage (Player II for the lower game, Player I for the upper game), is allowed to use both the past of $B^t({\cdot})$ and the other player’s control.

2.3. Statement of main results

For all $0 \leq t \leq s$, we let $G^+(s,t)$ and $G^-(s,t)$ denote the conditional probabilities

(7) \begin{equation} \begin{aligned} G^+(s,t) &= \mathbb{P}_t^\tau (\tau > s) = \mathbb{P}^\tau (\tau > s \mid \tau > t) {,} \\ G^-(s,t) &= \mathbb{P}_t^\tau (\tau \leq s) = \mathbb{P}^\tau (\tau \leq s \mid \tau > t) . \end{aligned} \end{equation}

Moreover, note that for each fixed $t\in[0,T]$, $G^-(s,t)$ is the probability distribution function of a continuous random variable and let $g^-(s,t)$ denote the corresponding conditional density function

(8) \begin{equation}g^-(s,t) = \dfrac{{\mathrm d}}{{\mathrm d} s},\quad G^-(s,t) .\end{equation}

In general, the value functions $V^-$ and $V^+$ defined by the variational identities (5) and (6) are not smooth. Nevertheless, they can still be characterized using the language of partial differential equations, relying on the notion of viscosity solutions originally proposed by Crandall and Lions in [Reference Crandall and Lions12] for the case of first-order Hamilton–Jacobi equations. See Appendix A for the definition of viscosity solution used herein. A central step of the proof of our main result is the introduction of an auxiliary SDG, with a non-constant discount rate related to the conditional probabilities (7) and a deterministic time horizon. Indeed, the lower and upper value functions $V^-$ and $V^+$ can be characterized in terms of the value functions of said auxiliary game.

Theorem 1. Assume that assumptions (A1)–(A4) hold. The lower and upper value functions $V^-$ and $V^+$ are the unique viscosity solutions of the Hamilton–Jacobi–Bellman–Isaacs equation

(9) \begin{equation}W_t - g^-(t,t)W + \mathcal{H}^{-}(t,x,W_x,W_{xx}) = 0{,} \quad W(T,x) = \Psi(T,x)\end{equation}

and

(10) \begin{equation}W_t - g^-(t,t)W + \mathcal{H}^{+}(t,x,W_x,W_{xx}) = 0 {,} \quad W(T,x) = \Psi(T,x),\end{equation}

where, for $A \in \mathbb{S}^N$ (the set of symmetric $N \times N$ matrices), $p,x \in {\mathbb R}^N$, and $t\in[0,T]$, we have

\begin{align*}\mathcal{H}^-(t,x,p,A)&= \max_{u \in U}\; \min_{v \in V}\; H(t,x,u,v,p,A), \\*\mathcal{H}^+(t,x,p,A)&= \min_{v \in V}\; \max_{u \in U}\; H(t,x,u,v,p,A){,}\end{align*}

and

\begin{align*}H(t,x,u,v,p,A) &= \mathrm{tr}\,\Big(\dfrac{1}{2}a(t,x,u,v)A\Big)+f(t,x,u,v)p+ L(t,x,u,v)+g^-(t,t) \Psi(t,x) {,}\end{align*}

with $a=\sigma \sigma'$, $\sigma'$ denoting the transpose of $\sigma$.

Note the presence of the additional (non-standard) terms $-g^-(t,t)W(t,x)$ and $g^-(t,t)\Psi(t,x)$ related to the conditional probabilities (7)–(8) on the HJBI equations (9) and (10). Such terms reflect the randomness of the planning horizon and encapsulate the agents’ behavior planning their actions as if the horizon were fixed at T, but with subjective rate of time preferences determined by the conditional probabilities (7).

We say that the Isaacs condition holds if, for all $ (t,x,p,A) \in [0,T]\times {\mathbb R}^N \times {\mathbb R}^N \times \mathbb{S}^N$, the following relation holds:

(11) \begin{equation}\mathcal{H}^+ (t,x,p,A) = \mathcal{H}^- (t,x,p,A) .\end{equation}

The next result is then a consequence of combining Isaacs condition above with the uniqueness of the viscosity solutions to (9) and (10), guaranteed by Theorem 1. In particular, this ensures existence of value for the SDG with random horizon (3)–(4) in the sense of Elliot and Kalton [Reference Elliott and Kalton15].

The rest of the paper is devoted to the proof of Theorem 1. The analysis, in Section 3, of a related discounted SDG plays a central role in this endeavor.

3.1. Some preliminary results

Before proceeding, we need to introduce further notation and terminology that will be useful below. Let $ (t,x)\in[0,T]\times{\mathbb R}^N$ be fixed and, for any given $u({\cdot}) \in \mathcal{U}(t,T)$ and $v({\cdot}) \in \mathcal{V}(t,T)$, define

\begin{equation*}\gamma(s,\omega) = (u(s,\omega),v(s,\omega))\end{equation*}

for every $s\geq t$ and $\omega\in\Omega_{t,T}^\omega$. By definition of the control processes $u({\cdot}) \in \mathcal{U}(t,T)$ and $v({\cdot}) \in \mathcal{V}(t,T)$, it immediately follows that $\gamma({\cdot})$ is $\mathbb{G}_{t,T}^\omega$-progressively measurable. Moreover, by standard results from stochastic differential equations theory (see e.g. [Reference Karatzas and Shreve24, Reference Mao28] for further details), it is known that the SDE (3) admits a unique solution $X_{t,x}^{u,v}({\cdot})$ on the filtered probability space $ (\Omega_{t,T}^\omega,\mathcal{G}_{t,T}^\omega,\mathbb{G}_{t,T}^\omega,\mathbb{P}_{t,T}^\omega)$ for any fixed $u({\cdot})\in \mathcal{U}(t,T)$ and $v({\cdot})\in \mathcal{V}(t,T)$. Moreover, $X_{t,x}^{u,v}({\cdot})$ satisfies

(17) \begin{equation}X^{u,v}_{t,x}(s) = X^{u,v}_{t,x}(\skew3\hat{t}\kern1.5pt) + \int_{\skew3\hat{t}}^s f(r,X_{t,x}^{u,v}(r),{\gamma}(r)) \, \mathrm{d} r + \int_{\skew3\hat{t}}^s \sigma(r,X_{t,x}^{u,v}(r),{\gamma}(r)) \, \mathrm{d} B^t(r),\end{equation}

where $t\leq \skew3\hat{t}\leq s\leq T$. Further, noting that

(18) \begin{equation}B^t(s,\pi^{-1}(\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T})) - B^t(\skew3\hat{t},\pi^{-1}(\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T})) = \omega^{\skew3\hat{t},T}(s),\end{equation}

we obtain that for ${\mathbb{P}}^\omega_{t,\skew3\hat{t}}$-a.e. $\omega^{t,\skew3\hat{t}}\in\Omega^\omega_{t,\skew3\hat{t}}$ the left-hand side of (18) coincides with the standard Brownian motion $B^{\skew3\hat{t}}(s,\omega^{\skew3\hat{t},T})$ on the filtered probability space $ (\Omega^\omega_{\skew3\hat{t},T},{\mathcal G}^\omega_{\skew3\hat{t},T},\mathbb{G}^\omega_{\skew3\hat{t},T},{\mathbb{P}}^\omega_{\skew3\hat{t},T})$.

Also define

\begin{align*}\tilde{\gamma}(s,\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T}) &= \gamma (s,\pi^{-1}(\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T})), \\ \tilde{X}(s,\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T}) &= X_{t,x}^{u,v} (s,\pi^{-1}(\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T})){,}\end{align*}

and note that the relation

\begin{align*} \tilde{X}(s,\omega^{t,\skew3\hat{t}},\cdot) &= X^{u,v}_{t,x}(\skew3\hat{t}\kern1.5pt) + \int_{\skew3\hat{t}}^s f(r,\tilde{X}(r,\omega^{t,\skew3\hat{t}},\cdot),\tilde{\gamma}(r,\omega^{t,\skew3\hat{t}},\cdot)) \, \mathrm{d} r \notag \\ &\quad\, + \int_{\skew3\hat{t}}^s \sigma(r,\tilde{X}(r,\omega^{t,\skew3\hat{t}},\cdot),\tilde{\gamma}(r,\omega^{t,\skew3\hat{t}},\cdot)) \, \mathrm{d} B^{\skew3\hat{t}}(r)\end{align*}

holds ${\mathbb{P}}^\omega_{t,\skew3\hat{t}}$-a.e. $\omega^{t,\skew3\hat{t}}\in\Omega^\omega_{t,\skew3\hat{t}}$ as a consequence of (17) and the comments following it. Moreover, by uniqueness of solutions of (3), we get that the paths of $\tilde{X}(s,\omega^{t,\skew3\hat{t}},\cdot)$, $s\in[\skew3\hat{t},T]$, coincide with those of (3) with initial condition $ (\skew3\hat{t},X^{u,v}_{t,x}(\skew3\hat{t}\kern1.5pt))$ and controls $ (u(\cdot,\omega^{t,\skew3\hat{t}}),v(\cdot,\omega^{t,\skew3\hat{t}}))$ for ${\mathbb{P}}^\omega_{t,\skew3\hat{t}}$-a.e. $\omega^{t,\skew3\hat{t}}\in\Omega^\omega_{t,\skew3\hat{t}}$. From this point onwards we will also use the notation $X^{u,v}_{t,x}({\cdot})$ to refer to the stochastic process $\tilde{X}({\cdot})$ on the filtered probability space $ (\Omega^\omega_{\skew3\hat{t},T},{\mathcal G}^\omega_{\skew3\hat{t},T},\mathbb{G}^\omega_{\skew3\hat{t},T},{\mathbb{P}}^\omega_{\skew3\hat{t},T})$.

The comments above, together with the fact that

\begin{equation*} {\mathbb E}_{{\mathbb{P}}^\omega_{t,\skew3\hat{t}} \otimes {\mathbb{P}}^\omega_{\skew3\hat{t},T}} [\phi (\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T}) \mid {\mathcal G}_{t,\skew3\hat{t}}^{\omega}] = {\mathbb E}_{{\mathbb{P}}^\omega_{\skew3\hat{t},T}} [\phi (\omega^{t,\skew3\hat{t}},\omega^{\skew3\hat{t},T}) ] \quad \text{${\mathbb{P}}^\omega_{t,\skew3\hat{t}}$-a.s.}\end{equation*}

for any bounded and measurable function $\phi \colon \Omega_{t,T}^\omega\to{\mathbb R}$, yield the following technical lemma.

Lemma 1. Suppose that (A1)–(A2) hold and let $X^{u,v}_{t,x}({\cdot})$ denote the solution of (3) with initial condition $ (t,x)\in[0,T]\times{\mathbb R}^N$ and controls $ (u({\cdot}),v({\cdot}))\in{\mathcal U}(t,T)\times{\mathcal V}(t,T)$. For any bounded continuous function $\phi$ and any deterministic $s \in [\skew3\hat{t},T]$,

\[{{\mathbb{E}}_{\mathbb{P}_{t,T}^{\omega }}}[\phi (X_{t,x}^{u,v}(s),\gamma (s,\omega ))\mid \mathcal{G}_{t,\text{ }\hat{t}}^{\omega }]=\mathbb{E}_{\mathbb{P}_{\hat{t},T}^{\omega }}^{{}}[\phi (X_{\hat{t},X_{t,x}^{u,v}(\hat{t})}^{u,v}(s),\tilde{\gamma }(s,{{\omega }^{t,\hat{t}}},{{\omega }^{\hat{t},T}}))]\quad \mathbb{P}_{\text{t},\text{\hat{t}}}^{\omega }\text{-}a.s.\]

The next lemma ensures boundedness of the value functions $W^-$ and $W^+$ introduced in (13) and (14), as well as their Lipschitz continuity with respect to x and Hölder continuity with respect to t.

Proof. Let us start by proving item (i) for the discounted payoff functional

(19) \begin{equation}x \to {\mathcal J}(t,\cdot;\, \alpha[v({\cdot})],v({\cdot})),\end{equation}

where $t\in[0,T]$, $v({\cdot}) \in \mathcal{V}(t,T)$ and $\alpha \in \mathcal{A}(t,T)$. The proof for $x \to {\mathcal J}(t,\cdot;\,u({\cdot}),\beta[u({\cdot})])$, with $t\in[0,T]$, $u({\cdot}) \in \mathcal{U}(t,T)$ and $\beta \in \mathcal{B}(t,T)$, is similar.

Boundedness of (19) follows from boundedness of L and $\Psi$, guaranteed by assumption (A2), as well as boundedness of the non-constant discount factor $\Theta$, guaranteed by assumption (D1).

As for Lipschitz continuity of (19), this will follow from

(20) \begin{equation}{\mathbb E}_{{\mathbb{P}}^\omega_{t,T}}\big[|X^{u,v}_{t,x}(s)-X^{u,v}_{t,y}(s)|\big] \leq C |x-y| \quad \text{for all} \ x,y \in {\mathbb R}^N,\end{equation}

where $X^{u,v}_{t,x}(s)$ and $X^{u,v}_{t,y}(s)$, $s \in [t,T]$, are the solutions of (3) starting at t from x and y, respectively, with the same control pair (u, v). To see that (20) holds, set $Z(s)=X^{u,v}_{t,x}(s)-X^{u,v}_{t,y}(s)$, $s \in [t,T]$. From Itô’s formula, we obtain

\begin{align*}{\mathbb E}_{{\mathbb{P}}^\omega_{t,T}}\big[|X^{u,v}_{t,x}(s)-X^{u,v}_{t,y}(s)|^2\big] & = |x-y|^2+ {\mathbb E}_{{\mathbb{P}}^\omega_{t,T}}\bigg[\int_t^s \big[2Z(r)\cdot f_1(r, X^{u,v}_{t,x}(r),X^{u,v}_{t,y}(r),u(r),v(r)) \notag \\* &\quad\, + \mathrm{tr}\,(\sigma_1 \sigma_1^T)(r, X^{u,v}_{t,x}(r),X^{u,v}_{t,y}(r),u(r),v(r)) \big] {\mathrm{d}} r \bigg] \notag,\end{align*}

where $f_1$ and $\sigma_1$ are defined by

\begin{align*}f_1(t, x,y,u,v) &= f(t, x,u,v)-f(t, y,u,v) {,} \\*\sigma_1(t, x,y,u,v) &= \sigma(t, x,u,v)-\sigma(t, y,u,v)\end{align*}

for $t \in [0,T]$, $x,y \in {\mathbb R}^N$, $u \in U$, and $v \in V$. Using assumption (A2) and the Fubini–Tonelli theorem, we obtain that there exists a positive constant $C_1$ such that

\begin{equation*}{\mathbb E}_{{\mathbb{P}}^\omega_{t,T}}\big[|X^{u,v}_{t,x}(s)-X^{u,v}_{t,y}(s)|^2\big] \leq |x-y|^2 + C_1 \int_t^s {\mathbb E}_{{\mathbb{P}}^\omega_{t,T}}\big[|X^{u,v}_{t,x}(r)-X^{u,v}_{t,y}(r)|^2\big] \, \mathrm{d} r .\end{equation*}

Applying Gronwall’s inequality, we get that there exist positive constants $C_2$ and $C_3$ such that the following inequalities hold:

(21) \begin{equation}{\mathbb E}_{{\mathbb{P}}^\omega_{t,T}}\big[|X^{u,v}_{t,x}(s)-X^{u,v}_{t,y}(s)|^2\big] \leq \bigg(1+\int_t^s \,{\mathrm{e}}^{C_2 r} \, \mathrm{d} r\bigg)|x-y|^2 \leq C_3 |x-y|^2 .\end{equation}

Inequality (20) now follows from combining (21) with Hölder’s inequality, and Lipschitz continuity of (19) with respect to x follows from combining inequality (20) with Lipschitz continuity of L and $\Psi$, as guaranteed by assumption (A2).

As for the proof of item (ii), we note that boundedness of $W^-$ and $W^+$, as well as Lipschitz continuity with respect to x, follows as a consequence of the corresponding uniform properties of the discounted payoff functionals.

In the next section we will introduce a special class of restrictive strategies and the corresponding value functions, following a method originally developed by Fleming and Souganidis [Reference Fleming and Souganidis17]. These will enable us to prove certain sub- and super-optimal dynamic programming principles.

3.2. Sub-optimal and super-optimal dynamic programming principles

As noted by Fleming and Souganidis in their seminal paper [Reference Fleming and Souganidis17], serious measurability issues seem to prevent a generalization of the method for the proof of the deterministic dynamic programming principle to the stochastic set-up. To overcome these difficulties, Fleming and Souganidis have introduced the concept of restrictive strategies or, more commonly, r-strategies, that we employ here.

Before proceeding to the definition of r-strategies, we note that by definition of admissible control process, for $0\leq \bar{t}\leq t\leq T$, $u({\cdot})\in{\mathcal U}(\bar{t},T)$, and ${\mathbb{P}}^\omega_{\bar{t},t}$ a.e. $\omega^{\bar{t},t}\in\Omega^\omega_{\bar{t},t}$, the map $u(\omega_{\bar{t},t})\colon [t,T]\times\Omega^\omega_{t,T}\to U$ defined via the relation

\begin{equation*}u(\omega^{\bar{t},t})(s,\omega^{t,T}) = u (s,\omega),\end{equation*}

where $\omega=\pi^{-1}(\omega^{\bar{t},t},\omega^{t,T})$, is an admissible control for Player I, i.e. $u(\omega_{\bar{t},t})\in {\mathcal U}(t,T)$.

Given the discounted SDG determined by (3) and (12), we say that a r-strategy $\beta$ for Player II on [t, T] is an admissible strategy with the following additional property: for every $\bar{t} < t < \skew3\hat{t}$ and $u({\cdot}) \in \mathcal{U}(\bar{t},T)$ the map $ (s,\omega) \rightarrow \beta[u(\omega_{\bar{t},t})({\cdot})](s,\omega^{t,T})$ is $ ({\mathcal B}([t,\skew3\hat{t}]) \otimes {\mathcal G}_{t,\skew3\hat{t}}^{\omega},{\mathcal B}(U))$-measurable, where ${\mathcal B}(X)$ stands for the Borel $\sigma$-algebra of a set X. The set of r-strategies for Player II is denoted by $\mathcal{B}_r(t,T)$. We define r-strategies for Player I in a similar fashion and denote the set of these strategies by $\mathcal{A}_r(t,T)$.

The r-lower and r-upper value functions of the discounted SDG determined by (3) and (12) with initial data (t, x) are given by

\begin{equation*}W_r^-(t,x) = \inf_{\beta \in \mathcal{B}_r(t,T)} \sup_{u({\cdot}) \in \mathcal{U}(t,T)} {\mathcal J}(t,x;\, u({\cdot}), \beta[u({\cdot})]) \\\end{equation*}

and

\begin{equation*}W_r^+(t,x) = \sup_{\alpha \in \mathcal{A}_r(t,T)} \inf_{v({\cdot}) \in \mathcal{V}(t,T)} {\mathcal J}(t,x;\, \alpha[v({\cdot})], v({\cdot})) .\end{equation*}

The next result is a consequence of Lemma 2 as well as the definitions of admissible strategies and r-strategies. We skip its proof.

Although the r-value functions do not satisfy the full dynamic programming principle, it is nevertheless possible to obtain sub- and super-optimal dynamic programming principles for these functions. This is the content of the next result.

Proposition 1. (Sub-optimal and super-optimal dynamic programming principle.) Suppose that conditions (A1)–(A2) and (D1)–(D2) hold. For any $ (t,x) \in [0,T) \times \mathbb{R}^N$ and every $\skew3\hat{t}\in [t,T)$, we obtain

(22) \begin{align}W_r^-(t,x) &\leq \inf_{\beta\in \mathcal{B}_r(t,T)} \sup_{u({\cdot}) \in \mathcal{U}(t,T)}{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \Theta(\skew3\hat{t},t) W_r^-(\skew3\hat{t},X_{t,x}^{u,v}(\skew3\hat{t}\kern1.5pt)) \notag \\ & \quad\, + \int_t^{\skew3\hat{t}} \Theta(s,t)L(s,X_{t,x}^{u,v}(s),u(s), \beta[u({\cdot})](s)) \, \mathrm{d} s \bigg], \end{align}

where $X_{t,x}^{u,v}({\cdot})$ is the solution of (3) with $v({\cdot}) = \beta[u({\cdot})]({\cdot})\in {\mathcal V}(t,T)$ for $u({\cdot}) \in {\mathcal U}(t,T)$, and

(23) \begin{align} W_r^+(t,x) &\geq \sup_{\alpha\in \mathcal{A}_r(t,T)} \inf_{v({\cdot}) \in \mathcal{V}(t,T)}{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \Theta(\skew3\hat{t},t) W_r^+(\skew3\hat{t},X_{t,x}^{u,v}(\skew3\hat{t}\kern1.5pt)) \notag \\ & \quad\, + \int_t^{\skew3\hat{t}} \Theta(s,t)L(s,X_{t,x}^{u,v}(s),\alpha[v({\cdot})](s), v(s)) \, \mathrm{d} s \bigg],\end{align}

where $X_{t,x}^{u,v}({\cdot})$ is the solution of (3) with $u({\cdot}) = \alpha[v({\cdot})]({\cdot})\in {\mathcal U}(t,T)$ for $v({\cdot}) \in {\mathcal V}(t,T)$.

Proof. We only prove inequality (22), with the proof of (23) being analogous. For simplicity of notation, we will drop the superscripts u, v from the solution $X_{t,x}^{u,v}({\cdot})$, with the precise controls used at each instant being clear from the context.

Let $ (t,x)\in[0,T)\times{\mathbb R}^N$ be fixed, let $\skew3\hat{t}\in[t,T)$ be arbitrary, and denote the right-hand side of (22) by $\overline{W}(t,x)$. Note that for any $ \epsilon > 0$ there exists $\beta_\epsilon({\cdot}) \in \mathcal{B}_r(t,T)$ such that

(24) \begin{equation}\overline{W}(t,x) \geq {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[ \Theta(\skew3\hat{t},t)W_r^-(\skew3\hat{t},X_{t,x}(\skew3\hat{t}\kern1.5pt)) + \int_t^{\skew3\hat{t}} \Theta(s,t)L(s,X_{t,x}(s),u(s), \beta_\epsilon[u({\cdot})](s)) \, \mathrm{d} s \bigg] -\epsilon\end{equation}

for every $u({\cdot}) \in \mathcal{U}(t,T)$. Moreover, for each $y \in {\mathbb R}^N$, we have

\begin{equation*} W_r^-(\skew3\hat{t},y)=\inf_{\beta \in \mathcal{B}_r(\skew3\hat{t},T)} \sup_{u({\cdot}) \in \mathcal{U}(\skew3\hat{t},T)} {\mathcal J}(\skew3\hat{t},y;\,u({\cdot}),\beta[u({\cdot})]) .\end{equation*}

Hence, there exists $\beta_{y} \in \mathcal{B}_r(\skew3\hat{t},T)$ such that

(25) \begin{equation}W_r^-(\skew3\hat{t},y)\geq\sup_{u({\cdot}) \in \mathcal{U}(\skew3\hat{t},T)}{\mathcal J}(\skew3\hat{t},y;\,u({\cdot}),\beta_{y}[u({\cdot})])-\epsilon .\end{equation}

Let $\{D_i\}_{i\in{\mathbb{N}}}$ be a Borel partition of $\mathbb{R}^N$ with diameter $\mathrm{diam}(D_i)<\delta$ and pick $y_i \in D_i$ for each $i\in{\mathbb{N}}$. By Lemma 2(i) and Corollary 2(a), the diameter $\delta>0$ can be chosen to be sufficiently small that, for any $y \in D_i$,

(26) \begin{equation}|{\mathcal J}(\skew3\hat{t},y;\,u({\cdot}),\beta[u({\cdot})])-{\mathcal J}(\skew3\hat{t},y_i;\,u({\cdot}),\beta[u({\cdot})])| < \epsilon\end{equation}

for every $u({\cdot}) \in \mathcal{U}(\skew3\hat{t},T)$ and $\beta \in \mathcal{B}(\skew3\hat{t},T)$, and also

\begin{equation*} |W_r^-(\skew3\hat{t},y)-W_r^-(\skew3\hat{t},y_i)| <\epsilon .\end{equation*}

For each $ (\skew3\hat{t},\omega) \in [t,T]\times \Omega^\omega_{t,T}$ and $u({\cdot}) \in \mathcal{U}(t,T)$, define

\begin{equation*}\tilde{\beta}[u({\cdot})](s,\omega)=\begin{cases}\beta_\epsilon[u({\cdot})](s,\omega) &\textrm{if}\ s\in[t,\skew3\hat{t}), \\\sum_{i\in{\mathbb{N}}} \textbf{1}_{D_i}(X_{t,x}(\skew3\hat{t}\kern1.5pt))\beta_{y_i}[u(\omega^{t,\skew3\hat{t}})({\cdot})](s,\omega^{\bar{t},T}) &\textrm{if s}\in[\skew3\hat{t},T],\end{cases}\end{equation*}

where $\omega=(\omega^{t,\skew3\hat{t}},\omega^{\bar{t},T}) \in \Omega^\omega_{t,\skew3\hat{t}} \times \Omega^\omega_{\skew3\hat{t},T}$ and $u(\omega^{t,\skew3\hat{t}})({\cdot}) \in \mathcal{U}(\skew3\hat{t},T)$ is the admissible control introduced immediately before the definition of the r-value functions. Note that $\tilde{\beta}$ is an r-strategy by construction, i.e. $\tilde{\beta} \in \mathcal{B}_r(t,T)$.

Moreover, whenever $X_{t,x}(\skew3\hat{t}\kern1.5pt) \in D_i$ for some $i\in{\mathbb{N}}$ and $u({\cdot}) \in \mathcal{U}(t,T)$, relation (25) and inequality (26) yield

(27) \begin{align}W_r^-(\skew3\hat{t},y_i) &\geq {\mathcal J}(\skew3\hat{t},y_i;\,u(\omega^{t,\skew3\hat{t}})({\cdot}),\beta_{y_i}[u(\omega^{t,\skew3\hat{t}})({\cdot})])-\epsilon \notag \\ &\geq {\mathcal J}(\skew3\hat{t},X_{t,x}(\skew3\hat{t}\kern1.5pt);\,u(\omega^{t,\skew3\hat{t}})({\cdot}),\beta_{y_i}[u(\omega^{t,\skew3\hat{t}})({\cdot})])-2\epsilon\end{align}

for all $u({\cdot})\in{\mathcal U}(t,T)$ and ${\mathbb{P}}^\omega_{t,\skew3\hat{t}}$-a.e. $\omega^{t,\skew3\hat{t}}\in\Omega^\omega_{t,\skew3\hat{t}}$.

From the definition of the discounted payoff functional in (12), we get

(28) \begin{align}&{\mathcal J}(t, x;\, u({\cdot}),\tilde{\beta}[u({\cdot})]) \notag\\ & \qquad = {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \int_t^{T} \Theta(s,t)L(s, X_{t,x}(s), u(s),\tilde{\beta}[u({\cdot})](s)) \, \mathrm{d} s + \Theta(T,t) \Psi(T, X_{t,x}(T)) \bigg] \notag \\& \qquad = {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \int_t^{\skew3\hat{t}} \Theta(s,t)L(s, X_{t,x}(s), u(s),\tilde{\beta}[u({\cdot})](s)) \, \mathrm{d} s \notag \\&\qquad\quad\, + \sum_{i\in{\mathbb{N}}} {\textbf 1}_{D_i}(X_{t,x}(\skew3\hat{t}\kern1.5pt))\bigg(\int_{\skew3\hat{t}}^{T} \Theta(s,t)L(s, X_{t,x}(s), u(s),\tilde{\beta}[u({\cdot})](s))\, \mathrm{d} s \notag\\ &\qquad\quad\, + \Theta(T,t) \Psi (T, X_{t,x}(T)) \bigg) \bigg] .\end{align}

Combining assumption (D2) with (28), we get

\begin{align*} &{\mathcal J}(t, x;\, u({\cdot}),\tilde{\beta}[u({\cdot})]) ={\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \int_t^{\skew3\hat{t}} \Theta(s,t)L(s, X_{t,x}(s), u(s),\tilde{\beta}[u({\cdot})](s)) \, \mathrm{d} s \notag \\ &\qquad + \Theta(\skew3\hat{t},t)\sum_{i\in{\mathbb{N}}} {\textbf 1}_{D_i}(X_{t,x}(\skew3\hat{t}\kern1.5pt))\bigg(\int_{\skew3\hat{t}}^{T} \Theta(s,\skew3\hat{t})L(s, X_{t,x}(s), u(s),\tilde{\beta}[u({\cdot})](s)) \, \mathrm{d} s\notag \\ &\qquad + \Theta(T,\skew3\hat{t}) \Psi(T, X_{t,x}(T))\bigg) \bigg] \notag .\end{align*}

From the definition of the r-strategy $\tilde{\beta}$, we get

\begin{align*} & {\mathcal J}(t, x;\, u({\cdot}),\tilde{\beta}[u({\cdot})]) = {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \int_t^{\skew3\hat{t}} \Theta(s,t)L(s, X_{t,x}(s), u(s),\beta_\epsilon[u({\cdot})](s)) \, \mathrm{d} s \ &\qquad + \Theta(\skew3\hat{t},t)\sum_{i\in{\mathbb{N}}} {\textbf 1}_{D_i}(X_{t,x}(\skew3\hat{t}\kern1.5pt))\,{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[\int_{\skew3\hat{t}}^{T} \Theta(s,\skew3\hat{t})L(s, X_{t,x}(s), u(s),\tilde{\beta}[u({\cdot})](s)) \, \mathrm{d} s \\ &\qquad + \Theta(T,\skew3\hat{t}) \Psi (T, X_{t,x}(T)) \mid {\mathcal G}_{t,\skew3\hat{t}}\bigg] \bigg] .\end{align*}

Combining the previous relation with Lemma 1, we obtain

\begin{align*} & {\mathcal J}(t, x;\, u({\cdot}),\tilde{\beta}[u({\cdot})]) = {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \int_t^{\skew3\hat{t}} \Theta(s,t)L(s, X_{t,x}(s), u(s),\beta_\epsilon[u({\cdot})](s)) \, \mathrm{d} s \\ &\qquad + \Theta(\skew3\hat{t},t)\sum_{i\in{\mathbb{N}}} {\textbf 1}_{D_i}(X_{t,x}(\skew3\hat{t}\kern1.5pt)){\mathcal J}(\skew3\hat{t},X_{t,x}(\skew3\hat{t}\kern1.5pt);\,u(\omega^{t,\skew3\hat{t}})({\cdot}),\beta_{y_i}[u(\omega^{t,\skew3\hat{t}})({\cdot})])\bigg] .\end{align*}

Using inequalities (27) and (26), we get

\begin{align*} {\mathcal J}(t, x;\, u({\cdot}),\tilde{\beta}[u({\cdot})]) &\leq {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \int_t^{\skew3\hat{t}} \Theta(s,t)L(s, X_{t,x}(s), u(s),\beta_\epsilon[u({\cdot})](s)) \, \mathrm{d} s \\ &\quad\, + \Theta(\skew3\hat{t},t)\sum_{i\in{\mathbb{N}}} {\textbf 1}_{D_i}(X_{t,x}(\skew3\hat{t}\kern1.5pt))W_r^-(\skew3\hat{t},y_i)\bigg] +2\epsilon\\&\leq {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[ \int_t^{\skew3\hat{t}} \Theta(s,t)L(s, X_{t,x}(s), u(s),\beta_\epsilon[u({\cdot})](s)) \, \mathrm{d} s \\ &\quad\, + \Theta(\skew3\hat{t},t)W_r^-(\skew3\hat{t},X_{t,x}(\skew3\hat{t}\kern1.5pt))\bigg] +3\epsilon\notag .\end{align*}

Finally, combining the previous inequality with (24), we conclude

\begin{equation*}{\mathcal J}(t, x;\, u({\cdot}),\tilde{\beta}[u({\cdot})]) \leq \overline{W}(t, x) + 4\epsilon\end{equation*}

for every $u({\cdot})\in{\mathcal U}(t,T)$. As a consequence, we obtain

\begin{equation*}W_r^-(t,x) \leq \overline{W}(t, x) + 4\epsilon .\end{equation*}

The proof is completed by letting $\epsilon$ go to zero.

Proposition 1 can be used to guarantee Hölder continuity of $W_r^-$ and $W_r^+$ with respect to t.

Proof. We will focus on establishing Hölder continuity of $W_r^-$ with respect to t, with the corresponding argument for $W_r^+$ being similar. To simplify notation, we will drop the superscripts u,v from the solution $X_{t,x}^{u,v}({\cdot})$, with the precise controls used at each instant being clear from the context.

Without loss of generality, suppose that $t_1,t_2\in[0,T]$ are such that $t_1 < t_2$ and $|t_2 - t_1| < 1$. Using (22) and rearranging terms, we get

\begin{align*}& W_r^-(t_1,x)-W_r^-(t_2,x) \notag \\ &\qquad\leq \inf_{\beta\in \mathcal{B}_r(t_1,T)} \sup_{u({\cdot}) \in \mathcal{U}(t_1,T)}{\mathbb E}_{{\mathbb{P}}_{t_1,T}^\omega} \bigg[\int_{t_1}^{t_2} \Theta(s,t_1)L(s,X_{t_1,x}(s),u(s), \beta[u({\cdot})](s)) \, \mathrm{d} s \notag \\ & \qquad\quad\, + \Theta(t_2,t_1) ( W_r^-(t_2,X_{t_1,x}(t_2)) -W_r^-(t_2,x))\notag \\ & \qquad\quad\, + (\Theta(t_2,t_1)-\Theta(t_1,t_1))W_r^-(t_2,x) {\bigg]} .\end{align*}

Combining the inequality above with uniform Lipschitz continuity of $W_r^-(t, x)$ in x and of $\Theta(s,t)$ in s, as well as boundedness of $\Theta$, L, and $W_r^-$, we obtain that there exists a positive constant $C_1$ such that

(29) \begin{equation}W_r^-(t_1,x)-W_r^-(t_2,x) \leq C_1\big( |t_2-t_1| + {\mathbb E}_{{\mathbb{P}}_{t_1,T}^\omega} \big[|X_{t_1,x}(t_2)-x|\big]\big) .\end{equation}

A first-moment estimate for SDEs [28, Corollary 2.4.6] guarantees the existence of a positive constant $C_2$ such that

(30) \begin{equation}{\mathbb E}_{{\mathbb{P}}_{t_1,T}^\omega} \big[|X_{t_1,x}(t_2)-x|\big] \leq C_2|t_2-t_1|^{1/2} .\end{equation}

Putting together inequalities (29) and (30), we conclude that

(31) \begin{equation}W_r^-(t_1,x)-W_r^-(t_2,x) \leq K_1|t_2-t_1|^{1/2}\end{equation}

for some positive constant $K_1$.

Given $u({\cdot}) \in {\mathcal U}(t_2,T)$, define $u^*({\cdot})\in {\mathcal U}(t_1,T)$ as

\begin{equation*}u^*(s,\omega) =\begin{cases}u(t_2,\omega^{t_1,t_2}) &\textrm{if $s \in [t_1, t_2]$}, \\u(s,\omega^{t_1,t_2}) &\textrm{if $s \in (t_2, T]$},\end{cases}\end{equation*}

and given $\beta^* \in {\mathcal B}_r(t_1,T)$, define $\beta \in {\mathcal B}_r(t_2,T)$ as

\begin{equation*}\beta[u({\cdot})](s,\omega^{t_2,T}) = \beta^*[u^*({\cdot})](s,\pi^{-1}(\omega^{t_1,t_2},\omega^{t_2,T})) .\end{equation*}

We now observe that for $\beta^* \in {\mathcal B}_r(t_1,T)$ we have

\begin{align*} & {\mathcal J}(t_1, x;\, u^*({\cdot}),\beta^*[u^*({\cdot})]) \\ &\qquad = {{\mathbb E}}_{{\mathbb{P}}^\omega_{t_1,T}}\bigg[ \int_{t_1}^{T} \Theta(s,t_1)L(s, X_{t_1,x}(s), u^*(s),\beta^*[u^*({\cdot})](s)) \, \mathrm{d} s + \Theta(T,t_1)\Psi(T, X_{t_1,x}(T)) \bigg] \\&\qquad= {{\mathbb E}}_{{\mathbb{P}}^\omega_{t_1,T}}\bigg[ \int_{t_1}^{t_2} \Theta(s,t_1)L(s, X_{t_1,x}(s), u^*(s),\beta^*[u^*({\cdot})](s)) \, \mathrm{d} s \notag \\ & \qquad\quad\, + \Theta(t_2,t_1){\mathcal J}(t_2, X_{t_1,x}(t_2);\, u({\cdot}),\beta[u({\cdot})]) \bigg] \\&\qquad= {{\mathbb E}}_{{\mathbb{P}}^\omega_{t_1,T}}\bigg[ \int_{t_1}^{t_2} \Theta(s,t_1)L (s, X_{t_1,x}(s), u^*(s),\beta^*[u^*({\cdot})](s)) \, \mathrm{d} s \notag \\ &\qquad\quad\,+ \Theta(t_2,t_1) ({\mathcal J}(t_2, X_{t_1,x}(t_2);\, u({\cdot}),\beta[u({\cdot})])-{\mathcal J}(t_2, x;\, u({\cdot}),\beta[u({\cdot})])) \\ &\qquad\quad\,+ (\Theta(t_2,t_1)-\Theta(t_1,t_1)) {\mathcal J}(t_2, x;\, u({\cdot}),\beta[u({\cdot})]) + {\mathcal J}(t_2, x;\, u({\cdot}),\beta[u({\cdot})]) \bigg] {.}\end{align*}

Combining this equality with boundedness and Lipschitz continuity of $x\mapsto{\mathcal J}(t, x;\, u({\cdot}), v({\cdot}))$, as guaranteed by Corollary 2, as well as boundedness and Lipschitz continuity of $\Theta$ and boundedness of L, guaranteed by the assumptions in the statement, we obtain that there exists a positive constant C such that

\begin{equation*} {\mathcal J}(t_1, x;\, u^*({\cdot}),\beta^*[u^*({\cdot})]) \geq - C\big(|t_2-t_1| + {\mathbb E}_{{\mathbb{P}}_{t_1,T}^\omega} \big[|X_{t_1,x}(t_2)-x|\big] \big) + {\mathcal J}(t_2, x;\, u({\cdot}),\beta[u({\cdot})]){.}\end{equation*}

As a consequence, we obtain

\begin{align*}&\sup_{u({\cdot}) \in \mathcal{U}(t_1,T)}{\mathcal J}(t_1, x;\, u({\cdot}),\beta^*[u({\cdot})]) \\ &\qquad \geq - C\big(|t_2-t_1| + {\mathbb E}_{{\mathbb{P}}_{t_1,T}^\omega} \big[|X_{t_1,x}(t_2)-x|\big] \big) + \sup_{u({\cdot}) \in \mathcal{U}(t_2,T)}{\mathcal J}(t_2, x;\, u({\cdot}),\beta[u({\cdot})]) \\ &\qquad \geq - C\big(|t_2-t_1| + {\mathbb E}_{{\mathbb{P}}_{t_1,T}^\omega} \big[|X_{t_1,x}(t_2)-x|\big] \big) + W_r^-(t_2,x) .\end{align*}

Resorting once more to the first-moment estimate (30), using the previous inequality we are able to obtain

(32) \begin{equation}W_r^-(t_1, x)-W_r^-(t_2,x) \geq - K_2|t_2-t_1|^{1/2}\end{equation}

for some positive constant $K_2$. Hölder continuity of $W_r^-$ follows from combining the estimates (31) and (32).

We now observe that the r-value functions $W_r^-$ and $W_r^+$ are continuous functions of (t, x), a consequence of Corollaries 2 and 3. Indeed, the r-value functions $W_r^-$ and $W_r^+$ are, respectively, viscosity subsolutions and supersolutions of the HJBI equations (15) and (16). The proof of this fact is similar to that of [17, Proposition 1.12], with only minor adjustments being required. We skip the details here for the sake of brevity.

The next section employs an approximation procedure originally due to Fleming and Souganidis [Reference Fleming and Souganidis17, Reference Souganidis39, Reference Souganidis40]. This procedure is based on a discretization of the time variable and yields viscosity solutions for (15) and (16).

3.3. Time-discretization procedure

Let $\pi=\{0=t_0<t_1<\cdots<t_m=T\}$ be a partition of [0, T], and let

\begin{equation*}\|\pi\|=\max_{1\leq i\leq m} (t_i - t_{i-1})\end{equation*}

denote the mesh of the partition $\pi$.

A $\pi$-admissible control $u({\cdot})$ for Player I on [t, T] is an admissible control with the following additional property. If $i_0 \in \{0,\ldots,m-1\}$ is such that $t\in[t_{i_0},t_{i_0+1})$, then $u(s)=u$ for $s\in[t,t_{i_0+1})$ with $u\in U$ and $u(s)=u_{t_k}$ for $s\in[t_k,t_{k+1})$ for $k=i_0+1,\ldots,m-1$ where $u_{t_k}$ is $\mathcal{G}_{t,t_k}^\omega$-measurable. The set of $\pi$-admissible controls for Player I on [t, T] will be denoted by $\mathcal{U}_{\pi}(t,T)$. A $\pi$-admissible control $v({\cdot})$ for Player II on [t, T] is defined similarly and the set of all such controls will be denoted by $\mathcal{V}_{\pi}(t,T)$.

A $\pi$-admissible strategy $\alpha$ for Player I on [t, T] is an element of the set of admissible strategies $\mathcal{A}(t,T)$ with the additional properties that $\alpha[\mathcal{V}(t,T)] \subset \mathcal{U}_{\pi}(t,T)$, if $t \in[t_{i_0},t_{i_0+1})$ then for every $v({\cdot}) \in \mathcal{V}(t,T)$ the resulting control $\alpha[v({\cdot})]|_{[t,t_{i_0+1})}$ does not depend on $v({\cdot})$, and if $v({\cdot}) \approx \tilde{v}({\cdot})$ on $[t,t_k]$, then $\alpha[v({\cdot})](t_k) = \alpha[\tilde{v}({\cdot})](t_k)$, ${\mathbb{P}}_{t,T}^{\omega}$-a.s. for every $k\in\{i_0+1,\ldots,m\}$. The set of all $\pi$-admissible strategies for Player I on [t, T) will be denoted by $\mathcal{A}_{\pi}(t,T)$. A $\pi$-admissible strategy $\beta$ for Player II on [t, T] is defined similarly and the set of all these strategies will be denoted by $\mathcal{B}_{\pi}(t,T)$.

Let $C_b^{0,1}({\mathbb R}^N)$ denote the space of bounded, Lipschitz-continuous functions on ${\mathbb R}^N$. For every $t\in [0,T)$ and $\skew3\hat{t} \in (t,T]$, define the operator $F^-_{t,\skew3\hat{t}}\colon C_b^{0,1}({\mathbb R}^N)\to C_b^{0,1}({\mathbb R}^N)$ by

(33) \begin{equation} F^-_{t,\skew3\hat{t}}\,\phi(x) = \sup_{u\in U} \inf_{v({\cdot}) \in \mathcal{V}(t,\skew3\hat{t})} {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[\Theta(\skew3\hat{t},t)\phi(X_{t,x}^{u,v}(\skew3\hat{t}\kern1.5pt)) +\int_t^{\skew3\hat{t}}\Theta(s,t)L(s, X_{t,x}^{u,v}(s),u,v(s))\, \mathrm{d} s\bigg],\end{equation}

where $\mathcal{V}(t,\skew3\hat{t})$ denotes the set of admissible controls for Player II on $[t,\skew3\hat{t})$ and $X_{t,x}^{u,v}({\cdot})$ is the solution of (3) on $[t,\skew3\hat{t})$ associated with the choice of admissible controls $u({\cdot}) \equiv u$ and $v({\cdot})\in\mathcal{V}(t,\skew3\hat{t})$ having initial condition x at time t.

In a similar fashion, define the operator $F^+_{t,\skew3\hat{t}}\colon C_b^{0,1}({\mathbb R}^N)\to C_b^{0,1}({\mathbb R}^N)$ as

\begin{equation*}F^+_{t,\skew3\hat{t}}\,\phi(x) = \inf_{v\in V} \sup_{u({\cdot}) \in \mathcal{U}(t,\skew3\hat{t})} {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[\Theta(\skew3\hat{t},t)\phi(X_{t,x}^{u,v}(\skew3\hat{t}\kern1.5pt))+\int_t^{\skew3\hat{t}}\Theta(s,t)L(s, X_{t,x}^{u,v}(s),u(s),v)\, \mathrm{d} s\bigg],\end{equation*}

where $\mathcal{U}(t,\skew3\hat{t})$ denotes the set of admissible controls for Player I on $[t,\skew3\hat{t})$ and $X_{t,x}^{u,v}({\cdot})$ is the solution of (3) on $[t,\skew3\hat{t})$ associated with the choice of admissible controls $v({\cdot}) \equiv v$ and $u({\cdot})\in\mathcal{U}(t,\skew3\hat{t})$ having initial condition x at time t.

Let $w_{\pi}^-\colon [0,T]\times{\mathbb R}^N \to{\mathbb R}$ be such that $w_{\pi}^-(T,x)=\Psi(T,x)$ and

(34) \begin{equation}w_{\pi}^-(t,x) = F^-_{t,t_{i_0+1}} \prod_{k=i_0+2}^m F^-_{t_{k-1},t_{k}} \Psi(T,x)\end{equation}

whenever $t \in [t_{i_0},t_{i_0+1})$, and similarly, let $w_{\pi}^+\colon [0,T]\times{\mathbb R}^N \to{\mathbb R}$ be such that $w_{\pi}^+(T,x)=\Psi(T,x)$ and

(35) \begin{equation}w_{\pi}^+(t,x) = F^+_{t,t_{i_0+1}} \prod_{k=i_0+2}^m F^+_{t_{k-1},t_{k}} \Psi(T,x)\end{equation}

whenever $t \in [t_{i_0},t_{i_0+1})$. Under assumptions (A1)–(A2) and (D1)–(D2), $w_{\pi}^-$ and $w_{\pi}^+$ are both well-defined. Moreover, $w_{\pi}^-$ and $w_{\pi}^+$ admit a stochastic game characterization, as described in the next result.

Proposition 3. Suppose that conditions (A1)–(A2) and (D1)–(D2) hold. For every $ (t,x) \in [0,T] \times {\mathbb R}^N$, we have

(36) \begin{equation}w_{\pi}^-(t,x) = \inf_{\beta \in \mathcal{B}(t,T)} \sup_{u({\cdot}) \in \mathcal{U}_{\pi}(t,T)} {\mathcal J}(t,x;\,u({\cdot}),\beta[u({\cdot})])\end{equation}

and

(37) \begin{equation}w_{\pi}^+(t,x) = \sup_{\alpha \in \mathcal{A}(t,T)} \inf_{v({\cdot}) \in \mathcal{V}_{\pi}(t,T)} {\mathcal J}(t,x;\,\alpha[v({\cdot})],v({\cdot})) .\end{equation}

Proof. We prove relation (36) only, with the proof of (37) being similar. The proof of (36) relies on the following two claims.

1. (i) For every $ (t,x) \in [0,T]\times{\mathbb R}^N$ and every $\epsilon >0$, there exist $\alpha_{\epsilon} \in \mathcal{A}_{\pi}(t,T)$ and $\beta_{\epsilon} \in \mathcal{B}_{\pi}(t,T)$ such that

   (38) \begin{equation}{\mathcal J}(t,x;\,u({\cdot}),\beta_{\epsilon}[u({\cdot})]) - \epsilon \leq w_{\pi}^-(t,x) \leq {\mathcal J}(t,x;\,\alpha_{\epsilon}[v({\cdot})],v({\cdot})) + \epsilon\end{equation}
   for all $u({\cdot}) \in \mathcal{U}_{\pi}(t,T)$ and $v({\cdot}) \in \mathcal{V}_{\pi}(t,T)$.

2. (ii) For any $\beta \in \mathcal{B}(t,T)$, the pair of strategies $\alpha_{\epsilon} \in \mathcal{A}_{\pi}(t,T)$ and $\beta \in \mathcal{B}(t,T)$ define controls $u^{\epsilon}({\cdot}) \in \mathcal{U}_{\pi}(t,T)$ and $v^{\epsilon}({\cdot}) \in \mathcal{V}(t,T)$ for which

   (39) \begin{equation} {\mathcal J}(t,x;\,\alpha_{\epsilon}[v^\epsilon({\cdot})],v^{\epsilon}({\cdot})) = {\mathcal J}(t,x;\,u^{\epsilon}({\cdot}),\beta[u^\epsilon({\cdot})]) .\end{equation}

Indeed, once the two claims above are proved, the result follows from noting that the left-hand side of (38) guarantees that

\begin{equation*}w_{\pi}^-(t,x) \geq \inf_{\beta \in \mathcal{B}(t,T)} \sup_{u({\cdot}) \in \mathcal{U}_{\pi}(t,T)} {\mathcal J}(t,x;\,u({\cdot}),\beta[u({\cdot})]),\end{equation*}

while combining the right-hand side of (38) with (39) yields the reverse inequality.

The proof of claim (ii) is similar to that of the corresponding statement in [Reference Fleming and Souganidis17] and we skip it. Let us then prove claim (i). For $\varphi \in C_b^{0,1}({\mathbb R}^N)$, $x\in{\mathbb R}^N$, $u \in U$, $t\in [0,T]$, and $\skew3\hat{t} \in (t,T]$, define

\begin{align*}\psi(x,u,t,\skew3\hat{t},\varphi) = \inf_{v({\cdot}) \in \mathcal{V}(t,\skew3\hat{t})} {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[\Theta(\skew3\hat{t},t)\varphi(X_{t,x}^{u,v}(\skew3\hat{t}\kern1.5pt)) +\int_t^{\skew3\hat{t}}\Theta(s,t)L(s, X_{t,x}^{u,v}(s),u,v(s)) \, \mathrm{d} s\bigg],\end{align*}

where $X_{t,x}^{u,v}({\cdot})$ is the solution of (3) under the choice of the admissible controls $u(s) \equiv u$ and $v({\cdot})\in\mathcal{V}(t,\skew3\hat{t})$ and initial condition x at time t. Using assumptions (A1)–(A2) and (D1)–(D2), we obtain that $\psi(\cdot,\cdot,t,\skew3\hat{t},\varphi) \in C_b^{0,1}({\mathbb R}^N \times U)$ and

\begin{equation*}F^-_{t,\skew3\hat{t}}\,\varphi(x) = \sup_{u \in U} \psi(x,u,t,\skew3\hat{t},\varphi),\end{equation*}

where $F_{t,\skew3\hat{t}}$ is the operator defined in (33).

If $t \in [t_{i_0},t_{i_0+1})$ for $i_0 \in \{0,1,\ldots,m-1\}$, let

\begin{align*}\varphi_m &= \Psi(T,\cdot) {,} \\ \varphi_j &= F^-_{t_j,t_{j+1}}\varphi_{j+1}, \quad j=i_0+1,\ldots,m-1 {,}\\ \varphi_{i_0} &= F^-_{t,t_{i_0+1}}\varphi_{i_0+1} .\end{align*}

Hence, we obtain that

\begin{equation*}\varphi_{i_0}(x)=w_{\pi}^-(t,x) .\end{equation*}

Using [31, Lemma 1], we partition ${\mathbb R}^N$ and U into Borel sets of diameter less than some positive constant $\delta$, to be determined below. Denote these partitions by $\{A_k\colon k=1,2,\ldots\}$ and $\{B_\ell\colon \ell=1,2,\ldots,L\}$, respectively, and pick $x_k \in A_k$ and $u_\ell \in B_\ell$ for each $k=1,2,\ldots$ and $\ell=1,2,\ldots,L$. For any $\gamma >0$ there exists $\delta$ small enough and $u_{kj}^* = u_{\ell(k,j)} \in U$, $k=1,2,\ldots$ and $j=i_0+1,\ldots,m$, such that

\begin{equation*} \psi(x_k,u_{kj}^*,t_{j-1},t_j,\varphi_j) > F^-_{t_{j-1},t_j} \varphi_j(x_k) - \gamma .\end{equation*}

Further, we choose $v_{kj}^\ell({\cdot}) \in V(t_{j-1},t_j)$ such that, for $u({\cdot})$ identically equal to $u_\ell\in U$ on the interval $[t_{j-1},t_j)$, we obtain

\begin{align*}&{\mathbb E}_{{\mathbb{P}}_{t_{j-1},T}^\omega}\bigg[\Theta(t_j,t_{j-1})\varphi(X_{t_{j-1},x_k}^{\ell}(t_j)) +\int_{t_{j-1}}^{t_j} \Theta(s,t_{j-1})L(s, X_{t_{j-1},x_k}^{\ell}(s),u_\ell,v_{kj}^\ell(s)) \, \mathrm{d} s\bigg]\\ & \qquad < \psi (x_k,u_\ell,t_{j-1},t_j,\varphi_{j}) +\gamma,\end{align*}

with $t_{i_0}=t$ whenever $j=i_0+1$. The notation $X_{t_{j-1},x_k}^{\ell}({\cdot})$ stands for the solution of (3) with initial condition $x_k$ at time $t_{j-1}$ subject to the admissible controls $u({\cdot})\equiv u_\ell$ and $v_{kj}^\ell({\cdot})$.

We will now exhibit the strategies $\alpha_{\epsilon}$ and $\beta_{\epsilon}$ in (38). Fix $ (t,x) \in {\mathbb R}^N \times [0,T)$. For $v({\cdot}) \in \mathcal{V}(t,T)$, define

\begin{equation*}\alpha_{\epsilon}[v({\cdot})](s) = I_{[t,t_{i_0+1})}(s) \sum_{k} u_{ki_0}^* I_{A_k}(x)+ \sum_{j=i_0+1}^{m-1}I_{[t_j,t_{j+1})}(s)\sum_{k} u_{kj}^* I_{A_k}(X(t_j)),\end{equation*}

where $X({\cdot})$ is defined on each of the intervals $[t,t_{i_0+1}]$ and $[t_j,t_{j+1}]$, $j=i_0+1,\ldots,m-1$, as the solution of (3) with $u({\cdot})=\alpha_{\epsilon}[v({\cdot})]$. For $u({\cdot}) \in \mathcal{U}(t,T)$, define

\begin{align*}\beta_{\epsilon}[u({\cdot})](s) &= I_{[t,t_0+1)}(s) \sum_{k,\ell} \hat{v}^\ell_{k i_0}(s) I_{A_k}(x) I_{B_\ell}(u(s)) \\ &\quad\, + \sum_{j=i_0+1}^{m-1}\sum_{k,\ell}I_{[t_j,t_{j+1})}(s)\hat{v}^\ell_{k j}(s) I_{A_k}(X(t_j)) I_{B_\ell}(u(s)), \notag\end{align*}

where $X({\cdot})$ is now defined on each of the intervals $[t,t_{i_0+1}]$ and $[t_j,t_{j+1}]$, $j=i_0 + 1,\ldots,m-1$, as the solution of (3) with $v({\cdot})=\beta_{\epsilon}[u({\cdot})]$, and $\hat{v}^\ell_{kj}(\cdot,\omega) = v^\ell_{kj}(\cdot,\omega^{t_j,T})$ using the identification of $\Omega^\omega_{t,T}$ with $\Omega^\omega_{t,t_j} \times \Omega^\omega_{t_j,T}$ provided by $\pi(\omega) = (\omega^{t,t_j},\omega^{t_j,T})$ discussed in Section 2.1.

Let ${\mathcal J}$ stand for either ${\mathcal J}(t,x;\,\alpha_{\epsilon}[v({\cdot})],v({\cdot}))$ or ${\mathcal J}(t,x;\,u({\cdot}),\beta_{\epsilon}[u({\cdot})])$. For any $v({\cdot}) \in \mathcal{V}(t,T)$ and $u({\cdot}) = \alpha_{\epsilon}[v({\cdot})]$ or $u({\cdot}) \in \mathcal{U}_{\pi}(t,T)$ and $v({\cdot}) = \beta_{\epsilon}[u({\cdot})]$, we have

(40) \begin{align}& w_{\pi}^-(t,x) - {\mathcal J} \notag\\ &\qquad = \varphi_{i_0}(x)-{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[ \int_t^{T} \Theta(s,t) L (s, X(s), u(s),v(s)) \, \mathrm{d} s + \Theta(T,t) \varphi_m(X(T)) \bigg] \notag\\&\qquad = \sum_{j=i_0+1}^m \big\{ {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}[ \Theta(t_{j-1},t) \varphi_{j-1}(X(t_{j-1}))]- {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} [\Theta(t_{j},t)\varphi_{j}(X(t_{j}))] \big\} \notag\\ & \qquad\quad\, -{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[ \int_t^{T} \Theta(s,t) L (s, X(s), u(s),v(s)) \, \mathrm{d} s\bigg] .\end{align}

Using assumption (D2), we obtain that

(41) \begin{align}& \sum_{j=i_0+1}^m \big\{ {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}[ \Theta(t_{j-1},t) \varphi_{j-1}(X(t_{j-1}))]- {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} [\Theta(t_{j},t)\varphi_{j}(X(t_{j}))] \big\} \notag\\&\quad\, -{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[ \int_t^{T} \Theta(s,t) L (s, X(s), u(s),v(s)) \, \mathrm{d} s\bigg] \notag\\&\qquad = \sum_{j=i_0+1}^m \Theta(t_{j-1},t)\bigg\{{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}[\varphi_{j-1}(X(t_{j-1}))] \notag\\&\qquad\quad\, -{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[\Theta(t_{j},t_{j-1})\varphi_j(X(t_{j})) + \int_{t_{j-1}}^{t_j} \Theta(s,t_{j-1}) L (s, X(s), u(s),v(s)) \, \mathrm{d} s\bigg]\bigg\} \notag \\&\qquad = {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[\sum_{j=i_0+1}^m \Theta(t_{j-1},t)\bigg\{\varphi_{j-1}(X(t_{j-1}))\\&\qquad\quad\, -{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[\Theta(t_{j},t_{j-1})\varphi_j(X(t_{j})) + \int_{t_{j-1}}^{t_j} \Theta(s,t_{j-1}) L (s, X(s), u(s),v(s)) \, \mathrm{d} s \Bigm| \mathcal{G}^\omega_{t,t_{j-1}}\bigg] \bigg\} \bigg]\notag\end{align}

Combining (40) and (41), we get

\begin{align*} & w_{\pi}^-(t,x) - {\mathcal J} \\&\qquad = {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[\sum_{j=i_0+1}^m \Theta(t_{j-1},t)\bigg\{\varphi_{j-1}(X(t_{j-1}))\\&\qquad\quad\, -{\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[\Theta(t_{j},t_{j-1})\varphi_j(X(t_{j})) + \int_{t_{j-1}}^{t_j} \Theta(s,t_{j-1}) L (s, X(s), u(s),v(s)) \, \mathrm{d} s \Bigm|\mathcal{G}^\omega_{t,t_{j-1}}\bigg] \bigg\} \bigg] .\end{align*}

Inequality (38) follows from the relation above after checking that the following two statements hold ${\mathbb{P}}_{t,T}^{\omega}$-a.s.

1. (A) For any $v({\cdot}) \in \mathcal{V}(t,T)$ and $u({\cdot}) = \alpha_{\epsilon}[v({\cdot})]$, we have

   \begin{align*}\varphi_{j-1}(X(t_{j-1}))& \leq {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega} \bigg[\Theta(t_{j},t_{j-1})\varphi_j(X(t_{j}))\\&\quad\, + \int_{t_{j-1}}^{t_j} \Theta(s,t_{j-1}) L (s, X(s), u(s),v(s)) \, \mathrm{d} s \Bigm| \mathcal{G}^\omega_{t,t_{j-1}}\bigg] + \epsilon(t_j-t_{j-1}) .\end{align*}

2. (B) For any $u({\cdot}) \in \mathcal{U}_{\pi}(t,T)$ and $v({\cdot}) = \beta_{\epsilon}[u({\cdot})]$, we have

   \begin{align*}& {\mathbb E}_{{\mathbb{P}}_{t,T}^\omega}\bigg[\Theta(t_{j},t_{j-1})\varphi_j(X(t_{j})) + \int_{t_{j-1}}^{t_j} \Theta(s,t_{j-1}) L (s, X(s), u(s),v(s)) \, \mathrm{d} s \Bigm| \mathcal{G}^\omega_{t,t_{j-1}}\bigg] \\ &\qquad \leq \varphi_{j-1}(X(t_{j-1})) + \epsilon(t_j-t_{j-1}) .\end{align*}

The proofs of (A) and (B) can be obtained by performing appropriate adjustments to the proofs of analogous statements in [Reference Fleming and Souganidis17]. We skip them to keep the presentation brief.

The next lemma follows from assumptions (A1)–(A2) and (D1)–(D2), as well as the characterizations of $w_{\pi}^-$ and $w_{\pi}^+$ given above.

Lemma 3. There exists a positive constant C, depending solely on assumptions (A1)–(A2) and (D1)–(D2), such that the inequalities

\begin{equation*} |w_{\pi}^\pm(t,x)| \leq C \quad \textrm{and}\quad |w_{\pi}^\pm(t,x)-w_{\pi}^\pm(\skew3\hat{t},\hat{x})| \leq C (|x-\hat{x}|+|t-\skew3\hat{t}|^{1/2} ) \end{equation*}

hold for all $x,\hat{x} \in {\mathbb R}^N$ and $t,\skew3\hat{t} \in [0,T]$.

Resorting to Lemma 3 above and the Arzela–Ascoli theorem, we obtain that the families of functions $\{w_{\pi}^-\}$ and $\{w_{\pi}^+\}$ converge uniformly as $\|\pi\|\to 0$ along subsequences to bounded uniformly continuous functions. We will see that these uniform limits are viscosity solutions of (15) and (16). That is the content of the result below.

Proposition 4. Assume that (A1)–(A2) and (D1)–(D3) hold and let $w_{\pi}^-$ and $w_{\pi}^+$ be given by (34) and (35), respectively. Then the limits

\begin{equation*}w^-=\lim_{\|\pi\| \to 0}w_{\pi}^- \quad \textrm{and} \quad w^+=\lim_{\|\pi\| \to 0}w_{\pi}^+\end{equation*}

exist locally uniformly and are the unique viscosity solution of (15) and (16), respectively.

Proof. Existence of $w^-$ and $w^+$ follows from a comparison theorem (Theorem 5 in the appendix) and Lemma 3 as long as one guarantees that any subsequential limit of the families $\{w_{\pi}^-\}$ and $\{w_{\pi}^+\}$ as $\|\pi\|\to 0$ is a viscosity solution of (15) and (16), respectively. This can be achieved by employing the same arguments as in [17, Proposition 2.5]. We omit the details here for the sake of brevity.

3.4. $W^-$ and ${{W}^{+}}$ characterization as viscosity solutions of (15) and (16)

In what follows, we will compile the results obtained in the preceding sections to complete the characterization of the lower and upper value functions $W^-$ and $W^+$ as the unique viscosity solutions of (15) and (16), respectively. For that purpose, we start by noting that since the limit functions $w^-$ and $w^+$ of Proposition 4 are the unique viscosity solutions of (15) and (16), respectively, then Proposition 2 and a comparison theorem (Theorem 5 in the appendix) yield the following result.

Lemma 4. For every $ (t,x) \in [0,T]\times{\mathbb R}^N$ we have that

\begin{equation*}W_r^-(t,x) \leq w^-(t,x) \quad \textit{and} \quad W_r^+(t,x) \geq w^+(t,x) .\end{equation*}

We will now show that $W^-(t,x) \geq w^-(t,x)$ and $W^+(t,x) \leq w^+(t,x)$ for every $ (t,x) \in [0,T]\times{\mathbb R}^N$. As a consequence, we will obtain that the lower and upper value functions $W^-$ and $W^+$ are the unique viscosity solutions of (15) and (16), respectively.

Proof. We only prove the statement concerning the lower value function, with a similar proof holding for the corresponding statement concerning the upper value function.

Combining Corollary 2 with Lemma 4 we obtain that $W^-\leq W_r^-\leq w^-$ on $[0,T]\times{\mathbb R}^N$. On the other hand, by Proposition 3 we have that for every partition $\pi$ of [0, T], the inequality $w_{\pi}^- \leq W^-$ holds on $[0,T]\times{\mathbb R}^N$. Proposition 4 then implies that $w^- \leq W^-$ on $[0,T]\times{\mathbb R}^N$, guaranteeing that $w^- = W^-$ on $[0,T]\times{\mathbb R}^N$.

Finally, if the Isaacs condition holds, then the HJBI equations (15) and (16) coincide. Hence, uniqueness of the viscosity solutions – a consequence of Theorem 5 – ensures that $W^-$ and $W^+$ are identical.

Finally, we are able to state a dynamic programming principle for each one of the value functions $W^-$ and $W^+$, defined in (13) and (14).

Proof. We start by proving that the lower value function of the discounted SDG (3)–(12) satisfies relation (42). The corresponding proof for the upper value function is similar and we omit it here.

Let $\skew3\hat{t} \in (0,T]$ be fixed and let $\overline{W}(t,x)$ denote the right-hand side of (42). It is enough to consider in (42) controls $u({\cdot})$ and strategies $\beta$ defined in $[t,\skew3\hat{t}]$. By Theorem 2, we have that $\overline{W}$ is the viscosity solution of (15) on $[0,\skew3\hat{t}]\times{\mathbb R}^N$ with $\overline{W}(\skew3\hat{t},x)=W^-(\skew3\hat{t},x)$. Since $W^-$ is the viscosity solution of the same problem, uniqueness of viscosity solutions yields that $\overline{W} = W^-$.