2. Brownian drift control problem

Let

$$W(T) = W(0) + \int_0^T \mu (t) {\rm{d}}t + \sigma B(T),\quad \quad T \ge 0,$$

be a diffusion process with drift μ(t) ∈ {u,v}, variance σ² > 0, and initial level W(0) on some filtered space $\{\Omega,\mathcal{F},\mathbb{P};\ \mathcal{F}_t,\,t\ge0\}$ . We assume that v > u and, to avoid tedious case analysis, we also assume that neither is 0. The process W(T) describes the difference between cumulative work to have arrived by time T and cumulative work processed by time T, i.e. the netput process. The drift rate {μ(t): t ≥ 0} is adapted to the Brownian motion {B(t): t ≥ 0}, and represents the difference between the average arrival rate and the rate at which work is completed.

The controller must exert the minimal instantaneous control required to keep the process within the prescribed range ${\mathcal{R = [0,}}\Theta ]$ if Θ > 0 is finite, or ${\mathcal{R = }}{{\mathbb{R}}_ + }$ if Θ is infinite, but may also invoke those controls at any time, e.g. by idling capacity or turning away work.

We let A(T) denote the cumulative units of capacity lost to idling and let R(T) denote the cumulative amount of work turned away up to time T. The resulting controlled process is

(1) $$X(T) = X(0) + \int_0^T \mu (t) {\rm{d}}t + \sigma B(T) + A(T) - R(T),\quad \quad T \ge 0,$$

where X(0) = W(0). We assume, without loss of generality, that $W(0)\in \mathcal{R}$ and that μ(0) = u.

The controller incurs a cost of U per unit to idle capacity and a cost of M per unit to turn away customers, and must pay a fixed cost K(u, v) ≥ 0 to change the drift rate from u to v and a fixed cost K(v, u) ≥ 0 to change the drift rate from v to u. We let K = K(u, v)+K = (v, u).

When X(T) > 0, the backlog of work incurs a linear delay cost at rate h per unit time. The controller also incurs a cost per unit time for capacity that is linear in the drift rate. In particular, the cost for capacity when the drift rate is u is pu.

A policy defines the times at which to adjust the drift rate, idle capacity, and turn away work. We restrict attention to the space $\cal P$ of nonanticipating policies Φ = ({T _i : i ≥ 0}, A, R), where

• $0 = T_0 \lt T_1 \lt T_2 \lt \ldots \lt T_i\lt T_{i+1} \lt \ldots $ is a sequence of stopping times, and

• A and R are continuous, nondecreasing, adapted processes such that X as defined by (1) lies in $\cal R$ for all T ≥ 0.

Under policy Φ = ({T _i : i ≥ 0}, A, R), the drift rate μ(t) = μ _i for T _i ≤ t < T _i+1, where μ _2i = u and μ _2i+1 = v for i ≥ 0.

We consider the economic average cost Brownian control problem, which is to find a nonanticipating policy Φ = ({T_i : i ≥ 0}, A, R), that minimizes the long-run average cost:

$${\rm{AC}}(\Phi ) = \mathop {\limsup }\limits_{T \to \infty } {1 \over T}\mathbb E \left[\int_0^T \left( {p\mu (t) + hX(t)} \right){\rm{d}}t + UA(T) + MR(T) + \sum\limits_{i = 1}^{N(T)} K({u_{i - 1}},{u_i}) \right].$$

Here, for each T ≥ 0, N(T) = sup{n ≥ 0: T _n ≤ T} denotes the number of changes in the drift rate by time T.

We show that, when the economic average cost Brownian control problem admits an optimal policy, a control band policy is optimal. We characterize the conditions on the cost parameters under which there is no lower bound on the average cost, hence no optimal policy exists, and when an optimal policy exists, we determine optimal policy parameters.

In each case, an optimal policy is a control band policy.

In this context, a control band policy Φ is defined as a pair of bands, Φ = {ϕ_u, ϕ_v}, where ϕ_μ = (μ, s_μ, β_μ, S_μ, τ_μ). Under the control band policy Φ, the controller maintains the drift rate μ and refrains from intervention so long as X remains in the interval (s_μ, S_μ). When X reaches s_μ, the value β _μ ∈ {u, v} indicates the appropriate action. If β _μ = μ, the controller exerts instantaneous controls, i.e. idles capacity to keep X ≥ s _μ. Otherwise, the controller changes the drift rate. Similarly, when X reaches S_μ, τ _μ ∈ {u, v} indicates the appropriate action. If τ _μ = μ, the controller exerts instantaneous controls, i.e. turns away work to keep X ≤ S_μ. Otherwise, the controller changes the drift rate. Note that, when there is only one available drift rate μ, a control band policy is equivalent to setting bounds on the buffer size. In this case, the controller may set s_μ > 0 and/or S_μ < Θ for economic reasons, or set s_μ = 0 and S_μ = Θ to exploit the full physical capacity available.

We prove Theorem 1(a) in Lemma 1 and Lemma 2 of Section 3. We prove Theorem 1(b) in Lemma 2 of Section 3. We prove Theorem 1(c) in Lemma 4 of Section 4 and Corollary 9 of Section 4.2. We prove Theorem 1(d) in Lemma 10 and Corollary 8 of Section 4.1. We prove Theorem 1(e) in Section 4.2. In Appendix D we provide detailed performance metrics for policies of the forms (α, Ω) (α, s, Ω), and (α, s, S, Ω).

3. The one-drift rate problem: economic bounds

We first consider the case of a single drift rate μ in which the controller can only employ instantaneous controls. In the classic setting, the only possible policy is to idle capacity when the buffer is empty and turn away work when the buffer reaches the prescribed limit Θ. The economic average cost problem modifies this classic problem in two ways. First, it allows the prescribed upper bound Θ to be infinite and, second, it allows the controller to employ instantaneous controls at any time. Harrison and Taksar (Reference Harrison and Taksar1983) addressed the problem of using instantaneous controls to manage a Brownian motion within a compact state space, under a discounted cost setting with nonnegative convex holding costs, and Dai and Yao (Reference Dai and Yao2013) studied average cost Brownian control problems with instantaneous and impulse controls under nonnegative convex holding costs on the real line. We extend Dai and Yao (Reference Dai and Yao2013) by considering possibly negative (but linear holding costs) and possibly negative costs for instantaneous controls. We identify when the problem has a solution and provide closed-form expressions for an optimal policy and its average cost when such a policy exists.

Since the actions β and τ are fully determined in the one-drift rate problem, a control band in this context reduces to a pair (α, Ω), where 0 ≤ α < Ω ≤ Θ.

Observe that, for any policy $\Phi_0 = \{A_0, R_0\} \in \mathcal{P}$ , the policy Φ _a = {A _a, R _a}, where A _a (T) = A ₀ (T) + aT and R _a (T) = R ₀ (T) + aT, is also in $\cal P$ and AC(Φ _a ) = AC(Φ₀)+ (M + U)a. Thus, we see that if M + U < 0, there is no lower bound on the average cost of a policy.

The policy Φ _a involves a bit of ‘cheating’: the controller is rejecting work that has not yet arrived in order to idle additional capacity. This phenomenon highlights a strong connection between A and R that we formalize and exploit in Proposition 2.

Proposition 1

Suppose that the scalar γ and the continuous function $\delta\colon \mathcal{R}\to \mathbb{R}$ satisfy the following conditions:

(2) $$\delta (x) \;is\;continuously\;differentiable\;except\;at\;a\;finite\;set\;of\;points\;in\; {\cal R},$$

(3) $${{{\sigma ^2}} \over 2}{\delta ^{'}}(x) + \mu \delta (x) + p\mu + hx \ge \gamma \quad for\;almost\;all\; x \in {\cal R},$$

(4) $$ - U \le \delta (x) \le \min \{ 0,M\} \quad for\;all\; x \in \cal R.$$

Then γ ≤ AC(Φ) for each policy $\Phi \in \mathcal{P}$ .

Proof. The proof follows from an application of Itô’s formula. (Itô’s formula for semimartingales can be found, for example, in Theorem I.4.57 of Jacod and Shiryaev (Reference Jacod and Shiryaev2003).) Suppose that f: $\cal R \to \mathbb R$ is continuously differentiable, has a bounded derivative, and has a continuous second derivative at all but a finite number of points. Then, for each time T > 0, initial state X(0), and policy $\Phi=\{A,R\}\in\mathcal{P},$ we have

$$f(X(T)) = f(X(0)) + \int_0^T {f^{'}}(X(t)) {\rm{d}}X(t) + {1 \over 2}\int_0^T {f^{''}}(X(t)) {\rm{d}}X(t) {\rm{d}}X(t),$$

where

$${\rm{d}}X(t) = \mu {\rm{d}}t + \sigma {\rm{d}}B(t) + {\rm{d}}A(t) - {\rm{d}}R(t)\quad {\rm{and}}\quad {\rm{d}}X(t) {\rm{d}}X(t) = {\sigma ^2} {\rm{d}}t.$$

Hence,

(5) $$\eqalign{ {\mathbb{E}}[f(X(T))] & = {\mathbb{E}}[f(X(0))] + {\mathbb{E}}[\int_0^{T} ({{{\sigma ^{2}}} \over 2}{f^{''}}(X(t)) + \mu {f^{'}}(X(t))) {\rm{d}}t \cr & + \int_0^T {f^{'}}(X(t)) {\rm{d}}A(t) - \int_0^T {f^{'}}(X(t)) {\rm{d}}R(t)]. } $$

When $f(x) = \int_0^x \delta (\xi ) {\rm{d}}\xi $ so that ${f^{'}}(x) = \delta (x)$ , the inequalities (3) and (4) yield

$$\eqalign{ {\mathbb{E}}[f(X(T))] & - {\mathbb{E}}[f(X(0))] \cr & {= {\mathbb{E}}[\int_0^{T} ({{{\sigma ^{2}}} \over 2}{\delta ^{'}}(X(t)) + \mu \delta (X(t))) {\rm{d}}t + \int_0^{T} \delta (X(t))dA(t) - \int_0^{T} \delta (X(t)) {\rm{d}}R(t)]} \cr & \ge {\mathbb{E}}[\int_0^T (\gamma - p\mu - hX(t)) {\rm{d}}t - U\int_0^{T} {\rm{d}}A(t) - M\int_0^{T} {\rm{d}}R(t)].} $$

Rearranging terms, dividing both sides by T, and taking the limit superior as T goes to ∞ we see that

$$\eqalign{ & \mathop {\limsup }\limits_{T \to \infty } {1 \over T}({\mathbb{E}}[f(X(T))] - {\mathbb{E}}[f(X(0))]) \cr & + \mathop {\limsup }\limits_{T \to \infty } {1 \over T}{\mathbb{E}}[\int_0^T (p\mu + hX(t)) {\rm{d}}t + U\int_0^T {\rm{d}}A(t) + M\int_0^T {\rm{d}}R(t)] \cr & \quad \quad = \mathop {\limsup }\limits_{T \to \infty } {1 \over T}({\mathbb{E}}[f(X(T))] - {\mathbb{E}}[f(X(0))]) + {\rm{AC}}(\Phi ) \cr & \quad \quad \ge \gamma . \cr} $$

The fact that δ(x) ≤ 0 for $x\in\mathcal{R}$ implies that f (x) ≤ f (0) for $x\in \mathcal{R}$ , and so

$$\mathop {\limsup }\limits_{T \to \infty } {1 \over T}({\mathbb{E}}[f(X(T))] - {\mathbb{E}}[f(X(0))]) \le 0,$$

proving that γ ≤ AC(Φ).

Lemma 2 exploits the arguments of Proposition 1 to construct an optimal control band policy (α, Ω) or prove that no optimal policy exists when h ≤ 0 and Θ is infinite.

Proof. Case (a): h < 0. Observe that (see Lemma 13 in Appendix D)

$${\rm{AC}}(\alpha ,\alpha + 1) = {{(M + p)\mu + {{\mathbb{E}}^{ - 2\mu /{\sigma ^2}}}(U - p)\mu + h} \over {1 - {{\mathbb{E}}^{ - 2\mu /{\sigma ^2}}}}} - {{h{\sigma ^2}} \over {2\mu }} + h\alpha ,$$

and so, when h < 0 and Θ is infinite,

$$\mathop {\lim }\limits_{\alpha \to \infty } {\rm{AC}}(\alpha ,\alpha + 1) = - \infty $$

and there is no lower bound on the average cost of a policy.

In the other cases, we consider

$${\rm EA}(0,\Omega ) = {\mu \over {{{\rm e}^{2\mu \Omega /{\sigma ^2}}} - 1}}\quad {\rm{and}}\quad {\rm ER}(0,\Omega ) = {\mu \over {1 - {{\rm e}^{ - 2\mu \Omega /{\sigma ^2}}}}},$$

the average rates of instantaneous control at 0 and at Ω, respectively, under the policy (0, Ω) for Ω positive and finite. (See Lemma 13 in Appendix D.)

Case (b.1): h = 0, U ≥ 0, and μ < 0. Observe that δ(x) = –U and γ = (p – U) _μ satisfy the conditions of Proposition 1, proving that (p – U) _μ is a lower bound on the average cost of any nonanticipating policy. In this case, the fact that lim_Ω→∞ EA(0, Ω) = –μ implies that AC(0, Ω) = pμ + UEA(0, ∞) = (p – U) _μ , proving that (0, ∞) is an optimal policy.

Case (b.2) h = 0, U ≥ 0, M ≥ 0, and μ > 0. Since M ≥ 0 ≥ –U, δ(x) = 0, and γ = pμ satisfy the conditions of Proposition 1, proving that pμ is a lower bound on the average cost of any nonanticipating policy. In this case, the fact that lim_Ω→∞ EA(0, Ω) = 0 implies that lim_Ω→∞ AC(0, Ω) = pμ, proving that (0, ∞) is an optimal policy.

Case (b.3) h = 0, U ≥ 0, M < 0, and μ > 0. Observe that δ(x) = M and γ = (M + p)μ satisfy the conditions of Proposition 1, proving that (M + p)μ is a lower bound on the average cost of any nonanticipating policy. In this case, the facts that lim_Ω→∞ EA(0, Ω) = 0 and lim_Ω→∞ EA(0, Ω) = –μ lim_Ω→∞ ER(0, Ω) = μ imply that lim_Ω→∞ AC(0, Ω) = (M + p)μ, proving that, as Ω goes to ∞, the policy (0, Ω) is optimal.

Case (c). h = 0 and U < 0. Consider the policy Φ = {A, R} with R(T) = 0 and A(T) = A ₀(T)+ aT for each T ≥ 0, where A ₀ is the minimal instantaneous control required at 0 to keep the process non-negative and a > 0 is essentially a positive drift induced by additional instantaneous controls at every point. When U < 0 and h = 0,

$${\rm{AC}}(\Phi ) = \mathop {\limsup }\limits_{T \to \infty } {1 \over T}{\mathbb{E}} \left[\int_0^T p\mu {\rm{d}}t + U{A_0}(T) + UaT \right] \le p\mu + aU,$$

and, by making a large the controller can drive the average cost to –∞, proving that the problem has no lower bound.

Lemma 2 shows that when h ≤ 0 and Θ is infinite, the one-drift rate economic average cost problem either admits no best policy or a best policy is essentially for the controller to exert only the minimum effort required to keep the process nonnegative. To focus attention on the more interesting cases, in the remainder of the paper we adopt the following assumption.

Proposition 2

Under Assumptions 1 and 2, suppose that the scalar γ and the continuous function $\delta\colon \mathcal{R}\to \mathbb{R}$ satisfy (2)–(3) and

(6) $$ - U \le \delta (x) \le M\quad \;for\;all\; x \in {\cal R}.$$

Then γ ≤ AC(Φ) for each policy $\Phi \in \mathcal{P}$ .

Corollary 1 summarizes useful bounds obtained from simple applications of Proposition 2.

Corollary 1

Under Assumptions 1 and 2,

$${\rm{AC}}(\Phi ) \ge \left \{ \matrix{ {\max \{ (M + p)\mu , - (U - p)\mu \} } \hfill & {{if}\;h \ge 0,} \hfill \cr {\max \{ - (U - p)\mu + h\Theta ,(M + p)\mu + h\Theta \} } \hfill & {{if}\;h \lt 0,}}\right. $$

for each policy $\Phi\in\mathcal{P}$ .

Proof. When h ≥ 0, the function δ(x) = M and the scalar γ = (M + p)_μ satisfy the conditions of Proposition 2. Similarly, the function δ(x) = –U and the scalar γ = –(U – p)μ satisfy the conditions of Proposition 2. When h < 0, the function δ(x) = –U and the scalar γ = –(U – p)μ + hΘ satisfy the conditions of Proposition 2. Similarly, the function δ(x) = M and the scalar γ = (M + p)μ + hΘ satisfy the conditions of Proposition 2.

We employ Proposition 2 and Corollary 2 to construct an optimal control band policy (α, Ω) with the interpretation that the controller should idle capacity to keep X ≥ α and turn away work to keep X ≤ Ω. The proof of Proposition 2 is presented after Corollary 2.

Corollary 2

Under Assumptions 1 and 2, suppose that 0 ≤ α < Ω ≤ Θ, and that the scalar γ and the continuous function $\delta(x)\colon \mathcal{R}\to\mathbb{R}$ satisfy (2)–(3) and (6). If γ and δ also satisfy

(7) $${{{\sigma ^2}} \over 2}{\delta ^{'}}(x) + \mu \delta (x) + p\mu + hx = \gamma \quad for\;almost\;all\; x \in [\alpha ,\Omega ],$$

(8) $$\delta (\alpha ) = - U,$$

(9) $${\rm{and}}\;\delta (\Omega ) = M\quad if\;\Omega \;is\;finite ,$$

then AC(α, Ω) = γ, and so (α, Ω) is an optimal policy.

Proof of Proposition 2. The proof is analogous to the proof of Proposition 1. Under Assumption 2, we prove that, when

$$a = \mathop {\liminf}\limits_{T \to \infty } {1 \over T}({\mathbb{E}}[f(X(T))] - {\mathbb{E}}[f(X(0))])$$

is positive, AC(Φ) = ∞ and so AC(Φ) ≥ γ trivially.

When f (x) = x (5) becomes

(10) $${\mathbb{E}}[X(T)] = {\mathbb{E}}[X(0)] + \mu T + {\mathbb{E}}[A(T)] - {\mathbb{E}}[R(T)].$$

When $f(x) = \int_0^x \delta (\xi ) {\rm{d}}\xi $ so that $f^\prime(x)=\delta(x)$ , inequalities (3) and (6) yield

$$\eqalign{ & {\mathbb{E}}[f(X(T))] - {\mathbb{E}}[f(X(0))] \cr & \quad \quad = {\mathbb{E}}\left[\int_0^T ({{{\sigma ^2}} \over 2}{\delta ^{'}}(X(t)) + \mu \delta (X(t))) {\rm{d}}t + \int_0^{T} \delta (X(t)) {\rm{d}}A(t) - \int_0^{T} \delta (X(t)) {\rm{d}}R(t) \right] \cr & \quad \quad \ge {\mathbb{E}} \left[\int_0^{T} (\gamma - p\mu - hX(t)) {\rm{d}}t - U\int_0^{T} {\rm{d}}A(t) - M\int_0^{T} {\rm{d}}R(t) \right].} $$

Dividing both sides by T and taking the limit inferior as T goes to ∞, using the relationship lim inf (–A) = – lim sup (A) and rearranging terms, we see that

(11) $$\eqalign{ & \mathop {\liminf}\limits_{T \to \infty } {1 \over T}({\mathbb{E}}[f(X(T))] - {\mathbb{E}}[f(X(0))]) \cr & + \mathop {\limsup }\limits_{T \to \infty } {1 \over T}{\mathbb{E}} \left[\int_0^T (p\mu + hX(t)) {\rm{d}}t + U\int_0^T {\rm{d}}A(t) + M\int_0^T {\rm{d}}R(t) \right] \cr & \quad \quad \ge \gamma .} $$

Let

$$a = \mathop {\liminf}\limits_{T \to \infty } {1 \over T}({\mathbb{E}}[f(X(T))] - {\mathbb{E}}[f(X(0))]).$$

If a ≤ 0 (11) implies that AC(Φ) ≥ γ. If a > 0, observe that, since it has bounded derivative, f is Lipschitz continuous and there exists a constant r > 0 such that

$$f(X(T)) - f(X(0)) \le |f(X(T)) - f(X(0))| \le r|(X(T) - X(0)| \le r(X(T) + X(0))$$

for all T ≥ 0 and

$$0 \lt a = \mathop {\liminf}\limits_{T \to \infty } {1 \over T}{\mathbb{E}}[f(X(T))] \le \mathop {\liminf}\limits_{T \to \infty } {1 \over T}r{\mathbb{E}}[(X(T) + X(0))] = \mathop {\liminf}\limits_{T \to \infty } {r \over T}{\mathbb{E}}[X(T)].$$

Thus, there exists a constant b ≥ a/r such that

$$0 \lt b = \mathop {\liminf}\limits_{T \to \infty } {1 \over T}{\mathbb{E}}[X(T)]$$

and a constant t* > 0 such that

$${\mathbb{E}}[X(T)] \gt {b \over 2}T\quad {\rm for}\;{\rm all}\; T \gt {t^&#x002A;}.$$

The fact that . ${\mathbb{E}}[X(T)]$ has no upper limit implies that Θ must be infinite and so, by Assumption 2, h > 0. Thus, by Tonelli’s theorem,

$$\eqalign{ \mathop {\liminf}\limits_{T \to \infty } {1 \over T}{\mathbb{E}} [\int_0^T hX(t) {\rm{d}}t] & = \mathop {\liminf}\limits_{T \to \infty } {1 \over T}\int_0^T h{\mathbb{E}}\left[ {X(t)} \right] {\rm{d}}t \cr & \ge \mathop {\liminf}\limits_{T \to \infty } {1 \over T}\int_{{t^&#x002A;}}^T h{b \over 2}t {\rm{d}}t \cr & = \mathop {\liminf}\limits_{T \to \infty } {{hb} \over 4}{{{T^2} - {t^{&#x002A;2}}} \over T} \cr & = \infty .} $$

Furthermore, by (10)

$$\eqalign{{1 \over T}\left( {U{\mathbb{E}}[A(T)] + M{\mathbb{E}}[R(T)]} \right) & = {1 \over T}U({\mathbb{E}}[X(T)] - {\mathbb{E}}[X(0)]) - U\mu + {1 \over T}\left( {M + U} \right){\mathbb{E}}[R(T)] \cr & \ge {1 \over T}U({\mathbb{E}}[X(T)] - {\mathbb{E}}[X(0)]) - U\mu ,} $$

where the last inequality follows from Assumption 1. Taking the limit inferior as T → ∞ yields

$$\eqalign{ & \mathop {\liminf}\limits_{T \to \infty } {1 \over T}\left( {U{\mathbb{E}}[A(T)] + M{\mathbb{E}}[R(T)]} \right) \cr & \quad \quad \ge \mathop {\liminf}\limits_{T \to \infty } ({1 \over T}U({\mathbb{E}}[X(T)] - {\mathbb{E}}[X(0)]) - U\mu ) \cr & \quad \quad \ge \mathop {\liminf}\limits_{T \to \infty } {1 \over T}U{\mathbb{E}}[X(T)] + \mathop {\liminf}\limits_{T \to \infty } {1 \over T}U( - {\mathbb{E}}[X(0)]) + \mathop {\liminf}\limits_{T \to \infty } ( - U\mu ) \cr & \quad \quad = U(b - \mu ). \cr} $$

Thus,

$${\rm{AC}}(\Phi ) \ge p\mu + U (b - \mu ) + \mathop {\liminf}\limits_{T \to \infty } {{hb} \over 4}{{{T^2} - {t^{&#x002A;2}}} \over T} = p\mu + b\mathop {\liminf}\limits_{T \to \infty } ({h \over 4}T + U) = \infty .$$

Corollary 2 follows from the fact that, for a control band policy (α, Ω), with Ω finite, there is a unique smooth function g and a unique constant γ that satisfy (7)–(9) and AC(α, Ω) = γ. In particular (7) has the general solution

$$g(x) = - p - {{hx} \over \mu } + {{h{\sigma ^2}} \over {2{\mu ^2}}} + {\gamma \over \mu } + {{\rm e}^{ - 2\mu x/{\sigma ^2}}}{C_\mu },$$

for some constant C _μ, and (8) implies that

(12) $${C_\mu } = - {{\rm e}^{2\mu \alpha /{\sigma ^2}}}(U - p + {{h{\sigma ^2}} \over {2{\mu ^2}}} + {\gamma \over \mu } - {{h\alpha } \over \mu }).$$

When Ω is finite, (9) implies that

(13) $${C_\mu } = {{\mathbb{E}}^{2\mu \Omega /{\sigma ^2}}}(M + p - {{h{\sigma ^2}} \over {2{\mu ^2}}} - {\gamma \over \mu } + {h \over \mu }\Omega )$$

and (12)–(13) uniquely determine the average cost γ and the scalar C_μ as functions of the control parameters (α, Ω):

(14) $$\eqalign{ & {\gamma _\mu }(\alpha ,\Omega ) \cr & = {{{{\rm e}^{2\mu \Omega /{\sigma ^2}}}(h\Omega - h{\sigma ^2}/(2\mu ) + \mu (M + p)) - {{\rm e}^{2\mu \alpha /{\sigma ^2}}}(\alpha h - h{\sigma ^2}/(2\mu ) - \mu (U - p))} \over {{{\rm e}^{2\mu \Omega /{\sigma ^2}}} - {{\rm e}^{2\mu \alpha /{\sigma ^2}}}}}, \cr & \quad \quad \quad \quad \quad \quad \quad \quad \quad {C_\mu }(\alpha ,\Omega ) = {{h(\Omega - \alpha ) + (M + U)\mu } \over {({{\rm e}^{ - 2\mu \Omega /{\sigma ^2}}} - {{\rm e}^{ - 2\mu \alpha /{\sigma ^2}}})\mu }}.} $$

Lemma 3 and Lemma 4 provide explicit formulae for the parameters of an optimal control band policy under Assumptions 1 and 2. These computations rely on the Lambert-W functions W_k(⋅), where W_k(⋅) is the kth branch of the inverse relation for the function f (w) = we^w defined on the complex field (see, for example, Corless et al. (Reference Corless, Gonnet, Hare and Knuth1996), Euler (Reference Euler1921) and Lambert (Reference Lambert1758, Reference Lambert1772)). For each real value w with –1/e < w < 0, there are two possible real values for W(w), one on the branch W _–1(⋅)with W _–1(w) < –1 and the other on the branch W₀ (⋅) with W ₀(w) > –1.

Given 0 ≤ α < Ω ≤ Θ, where Ω is finite, we define

$${g_{(\alpha ,\Omega )}}(x) = - p - {{hx} \over \mu } + {{h{\sigma ^2}} \over {2{\mu ^2}}} + {{{\gamma _\mu }(\alpha ,\Omega )} \over \mu } + {{\rm e}^{ - 2\mu x/{\sigma ^2}}}{C_\mu }(\alpha ,\Omega )$$

and so

$$g_{(\alpha ,\Omega )}^{'}(x) = - {h \over \mu } - {{2\mu } \over {{\sigma ^2}}}{{\rm e}^{ - 2\mu x/{\sigma ^2}}}{C_\mu }(\alpha ,\Omega ) = - {h \over \mu } - {2 \over {{\sigma ^2}}}{{h(\Omega - \alpha ) + (M + U)\mu } \over {{{\rm e}^{ - 2\mu (\Omega - x)/{\sigma ^2}}} - {{\rm e}^{ - 2\mu (\alpha - x)/{\sigma ^2}}}}}.$$

Lemma 3

Under Assumptions 1 and 2, when h > 0, the unique nonnegative solution to the equation

(15) $$g_{(0,z)}^{'}(z) = 0$$

is

(16) $${\Omega ^&#x002A;}(\mu ) = - {{(M + U)\mu } \over h} - {{{\sigma ^2}} \over {2\mu }}(1 + {W_{k(h\mu )}}( - {{\rm e}^{ - 1 - 2(M + U){\mu ^2}/h{\sigma ^2}}})),$$

where k(x) = 0 if x < 0 and k(x) = –1 if x > 0. When h < 0, the unique solution to $g_{(z,\Theta )}^{'}(z) = 0$ satisfying z ≤ Θ is given by

(17) $${\alpha ^&#x002A;}(\mu ) = \Theta + {{(M + U)\mu } \over h} - {{{\sigma ^2}} \over {2\mu }}(1 + {W_{k(h\mu )}}( - {{\rm e}^{ - 1 + 2(M + U){\mu ^2}/h{\sigma ^2}}})).$$

The proof of Lemma 3 is given in Appendix A.

Observe that, by definition,

(18) $$\eqalign{ & {\alpha ^&#x002A;}(\mu ) = {{{\gamma _\mu }({\alpha ^&#x002A;}(\mu ),\Theta ) + (U - p)\mu } \over h}\quad {\rm when}\;h \lt 0, \cr & {\Omega ^&#x002A;}(\mu ) = {{{\gamma _\mu }(0,{\Omega ^&#x002A;}(\mu )) - (M + p)\mu } \over h}\quad {\rm when}\; h \gt 0. \cr} $$

Lemma 4 shows how to construct an optimal policy from these points.

Lemma 4

Under Assumptions 1 and 2, the control band policy (α_μ, Ω_μ), where

(19) $${\alpha _\mu } = \left\{ {\matrix{ 0 & { if\; h \ge 0,} \cr {\max \{ 0,{\alpha ^&#x002A;}(\mu )\} } & { if\; h \lt 0,} \cr } } \right.$$

and

(20) $${\Omega _\mu } = \left\{ {\matrix{ {\min \{ \Theta ,{\Omega ^&#x002A;}(\mu )\} } & { if\; h \gt 0,} \cr \Theta & { if\; h \le 0,} \cr } } \right.$$

and α* (μ) and Ω^* (μ) are defined by (17) and (16), respectively, is an optimal policy.

Observe that the fact that an optimal policy prescribes setting α = 0 when h > 0 and Ω = Θ when h < 0 follows intuitively, as, when h < 0, raising the upper limit of the process increases the savings from negative holding costs and reduces the frequency of instantaneous controls. Similarly, when h < 0, reducing the lower limit reduces both the holding costs and the frequency of instantaneous controls. We provide a formal proof below.

Proof. To simplify notation, let α = α_μ and Ω = Ω_μ. For each finite α and Ω with 0 < Ω ≤ Θ and 0 ≤ α < Ω, define

$${\delta _{(\alpha ,\Omega )}}(x) = \left\{ {\matrix{ { - U,} & {0 \le x \lt \alpha ,} \cr {{g_{(\alpha ,\Omega )}}(x),} & {\alpha \le x \le \Omega ,} \cr {M,} & {\Omega \lt x.} } } \right.$$

We argue that δ _{(α, Ω)}, where α and Ω are defined by (19) and (20), respectively, satisfies all the conditions of Corollary 2. We present the proof for the h > 0 case. The proof for the h ≤ 0 case is analogous, but relies on an assumption that Θ is finite in this case.

When h > 0, Ω is finite, and δ = δ _{(α, Ω)} and γ = γ_μ (α, Ω) satisfy (7)–(9) by construction. It remains to show that δ and γ satisfy (2) (3), and (6) as well.

To see that (6) holds, note that

$$g_{(0,{\Omega ^&#x002A;}(\mu ))}^{''}(x) = {{4{\mu ^2}} \over {{\sigma ^4}}}{C_\mu }(0,{\Omega ^&#x002A;}(\mu )){{\rm e}^{ - 2\mu x/{\sigma ^2}}},$$

and, by (18), (13) reduces to

$${C_\mu }(0,{\Omega ^&#x002A;}(\mu )) = - {{h{\sigma ^2}} \over {2{\mu ^2}}}{{\rm e}^{2\mu {\Omega ^&#x002A;}(\mu )/{\sigma ^2}}} \lt 0,$$

proving that g = g _(0,Ω*_(μ)) is concave and increasing on [0, Ω*(μ)). We consider the two cases Ω*(μ) ≤ Θ and Ω*(μ) ≤ Θ separately.

Case (i): Ω*(μ) ≤ Θ. In this case the facts that g (Ω*(μ)) = M and ${g^{'}}({\Omega ^&#x002A;}(\mu )) = 0$ ensure that δ is continuous and indeed continuously differentiable on $\mathcal{R}$ . Furthermore, since g is increasing on [0, Ω*(μ)) and satisfies g(0) = –U and g(Ω*(μ)) = M by construction, we see that δ satisfies (6).

Finally, we show that δ and γ satisfy (3). Note that, by construction, g and γ satisfy (3) with equality for 0 ≤ x ≤ Ω*(μ). By (18), (3) reduces to (M + p)μ + hx ≥ γ = (M + p)μ + Ω*(μ) for Ω*(μ) < x and, since h > 0, we see that δ and γ satisfy (3) for Ω*(μ) < x as well.

Case (ii): Ω*(μ) > Θ. In this case δ = g_{(0, Θ)} on $\mathcal{R},$ and so is continuous and satisfies (2), and δ and γ satisfy (3) by construction. It remains only to show that δ satisfies (6).

Observe that, since δ(0) = –U < M = δ(Θ), if $\delta^\prime $ has no root in $\mathcal{R}$ then δ must be increasing on $\mathcal{R}$ and so δ satisfies (6). If $\delta^\prime$ has a real root x * then

$${\delta ^{'}}({x^&#x002A;}) = - {h \over \mu } - {{2\mu } \over {{\sigma ^2}}}{{\rm e}^{ - 2\mu {x^&#x002A;}/{\sigma ^2}}}{C_\mu }(0,\Theta ) = 0,$$

and so

$${C_\mu }(0,\Theta ) = {{h\Theta + (M + U)\mu } \over {({{\rm e}^{ - 2\mu \Theta /{\sigma ^2}}} - 1)\mu }} = - {{h{\sigma ^2}} \over {2{\mu ^2}}}{{\rm e}^{2\mu {x^&#x002A;}/{\sigma ^2}}} \lt 0$$

and δ is concave. Furthermore, since (e^–2μΘ/σ2 –1)μ < 0,

(21) $$h\Theta + (M + U) \mu \gt 0.$$

We argue that x * > Θ and so δ is increasing on $\mathcal{R}$ and satisfies (6). We consider the two cases μ > 0 and μ < 0 separately.

Subcase (ii.1): Ω ^* (μ) > Θ and μ > 0. When μ > 0,

$$\mathop {\lim }\limits_{z \downarrow 0} g_{(0,z)}^{'}(z) = \mathop {\lim }\limits_{z \downarrow 0} - {h \over \mu } - {2 \over {{\sigma ^2}}}{{hz + (M + U)\mu } \over {1 - {{\rm e}^{2\mu z/{\sigma ^2}}}}} = \infty .$$

Now, the facts that $g^\prime_{(0, z)}(z)$ has a unique positive real root Ω*(μ) and that $\lim_{z\downarrow 0}g^\prime_{(0, z)} (z)\gt0$ imply that $g^\prime_{(0,z)} (z)\gt 0$ for 0 < z < Ω*(μ). Since Θ < Ω*(μ) by assumption, $\delta '(\Theta ) = g{'_{(0,\Theta )}}(\Theta ) \gt 0$ . This, together with the fact that δ is concave, implies that x* > Θ and δ is increasing on $\mathcal{R}$ .

Subcase (ii.2): Ω*(μ) > Θ and μ < 0. In this case observe that $g^\prime_{(0,z^&#x002A;)} (z^&#x002A;) = -{h}/{\mu}\gt 0$ , where z* = –(M + U)μ/h > 0. Now z* < Θ < Ω*(μ) by (21), and the facts that $g^\prime_{(0, z)}(z)$ has a unique positive real root Ω*(μ) and that $g_{(0,{z^&#x002A;})}^{'}({z^&#x002A;}) \gt 0$ imply that ${\delta ^{'}}(\Theta ) = g_{(0,\Theta )}^{'}(\Theta ) \gt 0$ , $x^&#x002A; \gt \Theta,$ and δ is increasing on $\mathcal{R}$ .

Let γ_μ denote the optimal average cost for the one-drift rate problem with rate μ.

Corollary 3

Under Assumptions 1 and 2,

$$\eqalign{ & {\alpha _\mu } = \left\{ {\matrix{ 0 & {{if}\;h \ge 0,} \cr {\max \{ 0,{{{\gamma _\mu } + (U - p)\mu } \over h}\} } & {{if}\;h \lt 0,} \cr } } \right. \cr & {\Omega _\mu } = \left \{ {\matrix{ {\min \{ \Theta ,{{{\gamma _\mu } - (M + p)\mu } \over h}\} } & {{if}\;h \gt 0,} \cr \Theta & {{if}\;h \le 0,} } } \right.} $$

and $g_{({\alpha _\mu },{\Omega _\mu })}^{'}(x) \gt 0$ for α_μ < x < Ω _μ . Furthermore, if α_μ > 0 then $g_{({\alpha _\mu },{\Omega _\mu })}^{'}(x) \lt 0$ for x < α_μ and if Ω _μ < Θ then $g_{({\alpha _\mu },{\Omega _\mu })}^{'}(x) \lt 0$ for x > Ω _μ .

4. The two-drift rate problem

In Section 3 we proved that, under Assumptions 1 and 2, an optimal policy for the one-drift rate problem is a control band policy and we provided formulae for computing γ_μ, the minimum average cost for the problem with the single drift rate μ. In this section we consider the case in which the controller has access to two drift rates u < v, show that an optimal policy can also be found among the family of control band policies, and provide tools for computing an optimal policy.

Under Assumptions 1 and 2, Proposition 3 provides a lower bound on the long-run average cost of any nonanticipating policy for the problem with two available drift rates.

Propostion 3

Under Assumptions 1 and 2, if the scalar γ and the continuous functions $\delta ( \cdot ,\mu ):{\cal R} \to {\cal R}$ for μ ∈ {u, v} satisfy

(22) $$\Delta \equiv \int_0^\Theta \left( {\delta (x,v) - \delta (x,u)} \right) {\rm{d}}x \le K,$$

(23) $$ - U \le \delta (x,u) \le \delta (x,v) \le M\quad for\;all\; x \in {\cal R},$$

and, for each μ ∈ {u, v},

(24) $$\delta ( \cdot ,\mu )\,is\,continuously\,differentiable\,except\,at\,a\,finite\,set\,of\,points\,on\,{\cal R,}$$

(25) $$and\quad {{{\sigma ^2}} \over 2}{\delta _x}(x,\mu ) + \mu \delta (x,\mu ) + p\mu + hx \ge \gamma \quad for\;almost\;all\; x \in {\cal R},$$

then γ ≤ AC(Φ) for each policy $\Phi \in {\cal P}$ .

Corollary 4 provides sufficient conditions for the control band policy Φ = {ϕ _u , ϕ _v } to be optimal. The proof of Proposition 3 is presented after Corollary 4.

Corollary 4

Under Assumptions 1 and 2, suppose that the scalar γ and the continuous functions $\delta ( \cdot ,\mu ):{\cal R} \to {\mathbb R}$ for μ ∈ {u, v} satisfy (23)–(25). If γ and δ also satisfy

(26) $$\Delta \equiv \int_0^\Theta \left( {\delta (x,v) - \delta (x,u)} \right) {\rm{d}}x = K\quad if\;\;{\tau _v}\; = u\;and\;\;{\beta _u}\; = v ,$$

and, for each μ ∈ {u, v}

(27) $${{{\sigma ^2}} \over 2}{\delta _x}(x,\mu ) + \mu \delta (x,\mu ) + p\mu + hx = \gamma \quad for\;almost\;all\; x \in [{s_\mu },{S_\mu }],$$

(28) $$\delta ({s_\mu },\mu ) = - U\quad if\; {\beta _\mu } = \mu ,$$

(29) $$ and \quad \delta ({S_\mu },\mu ) = M\quad if\; {\tau _\mu } = \mu .$$

then the control band policy Φ = {ϕ _u , ϕ _v } satisfies AC(Φ) = γ and so is an optimal policy.

Proof of Proposition 3. The proof closely follows the proofs of Proposition 1 and Proposition 2. Suppose that, for each μ ∈ {u, v}, $f( \cdot ,\mu ):{\cal R} \to {\mathbb R}$ is continuously differentiable, has a bounded derivative, and has a continuous second derivative at all but a finite number of points in $\mathcal{R}$ . Then, for each time T > 0, initial state $ (X(0),\mu ) \in {\cal R} \times \{ u,v\} ,$ and policy $\Phi = (\{ {T_i}:i \ge 0\} ,A,R) \in {\cal P},$ we have

(30) $$\eqalign{ & {\mathbb{E}}[f(X(T),\mu (T))] \cr & \quad \quad = {\mathbb{E}}[f(X(0),{\mu _0})] + {\mathbb{E}}[\int_0^T ({{{\sigma ^2}} \over 2}{f_{xx}}(X(t),\mu (t)) + \mu (t){f_x}(X(t),\mu (t))) {\rm{d}}t \cr & + \int_0^T {f_x}(X(t),\mu (t))dA(t) - \int_0^T {f_x}(X(t),\mu (t)) {\rm{d}}R(t) \cr & + \sum\limits_{i = 1}^{N(T)} (f(X({T_i}),\mu ({T_i})) - f(X({T_i} - ),\mu ({T_i} - )))]. \cr} $$

When f (x, μ) = x for each μ ∈ {u, v} (30) becomes

$${\mathbb{E}}[X(T)] = {\mathbb{E}}[X(0)] + \int_0^T \mu (t) {\rm{d}}t + {\mathbb{E}}[A(T)] - {\mathbb{E}}[R(T)],$$

and so

$$\mathop {\limsup }\limits_{T \to \infty } {1 \over T}{\mathbb{E}}[X(T)] = \overline \mu + \mathop {\limsup }\limits_{T \to \infty } {1 \over T}({\mathbb{E}}[A(T)] - {\mathbb{E}}[R(T)]),$$

where $\overline \mu = \mathop {\limsup }\nolimits_{T \to \infty } (1/T){\mathbb{E}}[\int_0^T \mu (t) {\rm{d}}t]$ is the long-run average drift rate under the policy Φ.

Letting $f(x,\mu ) = \int_0^x \delta (\xi ,\mu ) {\rm{d}}\xi $ so that f_x(x, μ) = δ(x, μ) for each μ ∈ {u, v}, inequalities (22) (23), and (25) yield

(31) $$\eqalign{ & {\mathbb{E}}[f(X(T),\mu (T))] - {\mathbb{E}}[f(X(0),{\mu _0})] \cr & \quad \quad \ge {\mathbb{E}}[\int_0^T (\gamma - p\mu (t) - hX(t)) {\rm{d}}t - U\int_0^T {\rm{d}}A(t) - M\int_0^T {\rm{d}}R(t) \cr & + \sum\limits_{i = 1}^{N(T)} \int_0^{X({T_i})} (\delta (x,{\mu _i}) - \delta (x,{\mu _{i - 1}})) {\rm{d}}x]. \cr} $$

Note that

$$\mathop {\limsup }\limits_{T \to \infty } {1 \over T}{\mathbb{E}}[\int_0^T (p\mu (t) + hX(t)) {\rm{d}}t + U\int_0^T {\rm{d}}A(t) + M\int_0^T {\rm{d}}R(t)]$$

is the long-run average cost of the policy Φ without the changeover costs. Thus, if

(32) $${\mathbb E}[\sum\limits_{i = 1}^{N(T)} K({\mu _{i - 1}},{\mu _i})] \ge - {\mathbb E}[\sum\limits_{i = 1}^{N(T)} \int_0^{X({T_i})} (\delta (x,{\mu _i}) - \delta (x,{\mu _{i - 1}})) {\rm{d}}x],$$

then dividing both sides of (31) by T, taking the limit inferior as T goes to ∞, and rearranging terms yields

$$\eqalign{\mathop {\lim inf}\limits_{T \to \infty } {1 \over T}E[f(X(T),\mu (T))] - E[f(X(0),{\mu _0})] + \cr & \mathop {\lim \sup }\limits_{T \to \infty } {1 \over T}E\left[ {\int_0^T E (p\mu (t) + hX(t)){\rm{d}}t + U\int_0^T {{\rm{d}}A(t)} + M\int_0^T {{\rm{d}}R(t)} } \right. \cr & \left. { + \sum\limits_{i = 1}^{N(T)} {K({\mu _{i - 1}},{\mu _i})} } \right] \cr & \ge \gamma . } $$

The proof that either

$$\mathop {\lim \,inf}\limits_{T \to \infty } {1 \over T}{\mathbb E}[f(X(T),\mu (T))] - {\mathbb E}[f(X(0),{\mu _0})] \le 0$$

or AC(Φ) = ∞ is analogous to the arguments used in the proof of Proposition 2.

To complete the proof, we show that (32) holds and so AC(Φ) ≥ γ. Without loss of generality, assume that μ ₀ = u, so that

\[\begin{align} & \mathbb E[\sum\limits_{i=1}^{N(T)}{}\int_{0}^{X({{T}_{i}})}{}(\delta (x,{{\mu }_{i}})-\delta (x,{{\mu }_{i-1}}))\text{d}x] \\ & \quad \quad =\mathbb E[\sum\limits_{\overset{i=1}{\mathop{iodd}}\,}^{N(T)}{}\int_{0}^{X({{T}_{i}})}{}(\delta (x,v)-\delta (x,u))\text{d}x-\sum\limits_{\overset{i=1}{\mathop{ieven}}\,}^{N(T)}{}\int_{0}^{X({{T}_{i}})}{}(\delta (x,v)-\delta (x,u))\text{d}x]. \\ \end{align}\]

Note that (22) and (23) ensure that

\[0\ge -\int_{0}^{X({{T}_{i}})}{}\left( \delta (x,v)-\delta (x,u) \right)\text{d}x\ge -\int_{0}^{\Theta }{}\left( \delta (x,v)-\delta (x,u) \right)\text{d}x\ge -K,\]

and so

\[\begin{align} & -\text{E}[\sum\limits_{i=1}^{N(T)}{}\int_{0}^{X({{T}_{i}})}{}(\delta (x,{{\mu }_{i}})-\delta (x,{{\mu }_{i-1}}))\text{d}x]\le \text{E}[\sum\limits_{\overset{i=1}{\mathop{ieven}}\,}^{N(T)}{}K] \\ & \qquad =\text{E}[\sum\limits_{i=1}^{N(T)}{}K({{\mu }_{i-1}},{{\mu }_{i}})]. \\ \end{align}\]

Corollary 4 follows from the fact that a control band policy Φ satisfying (22)–(29) has an average cost equal to γ and, since γ = AC(Φ) is a lower bound on the average cost of any nonanticipating policy, Φ is an optimal policy for the economic average cost Brownian control problem.

In the remainder of the paper we develop an approach to construct a control band policy Φ that satisfies all the conditions of Corollary 4. We first consider the case in which M > –p > –U and show how to construct a control band policy that is optimal when K = 0. We then address the general case in which K > 0, by constructing a policy Φ together with a scalar γ and functions δ satisfying (23)–(25) and (27)–(29) and adjusting the policy, γ and δ to also satisfy (22) and (26). Indeed, when M > –p > –U, this approach yields an optimal policy that is a control band policy. If –p lies outside the range (–U, M), Proposition 4 shows that a control band policy that uses only one drift rate is an optimal policy.

Assumption 3

It holds that M > –p –U.

Assumption 3 has practical implications. When M > –p, the cost to reject work exceeds the savings from not having to process it; when U > p, the cost of idling capacity exceeds the cost of operating it. When both are true, M + U > 0 and the controller has no incentive to ‘cheat’.

When considering the problem with two drift rates, we generalize the functions g of Section 3 to functions of the form

\[g(x,\mu ,\gamma )=-p-\frac{hx}{\mu }+\frac{h{{\sigma }^{2}}}{2{{\mu }^{2}}}+\frac{\gamma }{\mu }+{{\text{e}}^{-2\mu x/{{\sigma }^{2}}}}{{C}_{\mu }}(\gamma )\]

for some value C_{μ (}γ). Note that g (⋅, μ, γ) satisfies

(33) \[\frac{{{\sigma }^{2}}}{2}{{g}_{x}}(x,\mu ,\gamma )+\mu g(x,\mu ,\gamma )+p\mu +hx=\gamma \quad {\rm for}\; {\rm all}\ x\in \mathbb R,\]

and that g (z, μ, γ) = V for a given point \[z \in {\mathbb R},\] and value \[v \in {\mathbb R},\] if and only if

\[{{C}_{\mu }}(\gamma )={{\text{e}}^{2\mu z/{{\sigma }^{2}}}}(V+p-\frac{h{{\sigma }^{2}}}{2{{\mu }^{2}}}-\frac{\gamma }{\mu }+\frac{hz}{\mu }).\]

Thus, given a value for γ, we may uniquely determine the value of C_μ(γ) by specifying the value of g (⋅, μ, γ) at some point \[z\in \mathbb R\] .

Lemma 7

For each $\gamma \in \mathbb R$ ,

\[\frac{{{\sigma }^{2}}}{2}{{g}_{x}}(x,v,\gamma )+ug(x,v,\gamma )+pu+hx-\gamma =(u-v)(g(x,v,\gamma )+p)\quad for\ all\ x\in \mathbb R.\]

Furthermore, if g (z, u, γ) = g (z, v, γ) for some point \[z\in \mathbb R\] , then

\[(u-v)(g(z,v,\gamma )+p)=\frac{{{\sigma }^{2}}}{2}\left( {{g}_{x}}(z,v,\gamma )-{{g}_{x}}(z,u,\gamma ) \right).\]

We observe that Lemma 7 implies that the functions g (⋅, u, γ) and g (⋅, v, γ) will be tangent at z if g (z, u, γ) = g (z, μ, γ) = –p.

4.1. When K = 0

We first address the special case of the two-drift rate problem in which K, the cost to transition between the drift rates, is 0. In Lemma 8 we identify conditions under which an optimal policy is a particularly simple form of a control band policy defined by a pair (α, Ω) prescribing the minimum and maximum buffer levels, together with a point s with α ≤ s ≤ Ω at which to switch between the drift rates. In Lemma 10 we show that Assumptions 2 and 3 ensure that these conditions can be satisfied, and so there is an optimal policy of this simple form.

Lemma 8

Under Assumptions 2 and 3, when K = 0, suppose that s and γ satisfy

• i. g (s, μ, γ) = –p for μ ∈ {u, v},

• ii. g_x (x, u, γ) ≥ 0 for s ≤ x ≤ Ω(γ),

• iii. g_x (x, u, γ) ≥ 0 for α(γ) ≤ x ≤ s,

where

\[\begin{align} & \alpha (\gamma )=\left\{ \begin{matrix} \max \{0,\frac{\gamma +(U-p)v}{h}\} & h \lt 0, \\ 0 & h\ge 0, \\\end{matrix} \right. \\ & \Omega (\gamma )=\left\{ \begin{matrix} \min \{\Theta ,\frac{\gamma -(M+p)u}{h}\} & h>0, \\ \Theta & h\le 0, \\\end{matrix} \right. \\ \end{align}\]

and C_u(γ) and C_v(γ) are defined by g(Ω(γ), u, γ) = M and g(α (γ), v, γ) = –U, respectively. Then γ, the functions

\[\delta (x,u)=\delta (x,v)=\left\{ \begin{matrix} -U, & 0\le x\le \alpha (\gamma ), \\ g(x,v,\gamma ), & \alpha (\gamma )\le x\le s, \\ g(x,u,\gamma ), & s\le x\le \Omega (\gamma ), \\ M, & \Omega (\gamma )\le x, \\\end{matrix} \right.\]

and the policy Φ = {(v, α(γ), v, s, u) (u, s, v, Ω(γ), u} satisfy (22)–(29), proving that γ is the optimal average cost for the problem and Φ is an optimal policy.

Proof of Lemma 8. To simplify notation, let α = α(γ) and Ω = Ω(γ). Since δ(x, u) = δ(x, v) (22) and (26) are satisfied trivially. The facts that g (α, v, γ) = –U and g (Ω, u, γ) = M ensure that δ satisfies (28) and (29), and, together with (i) (ii), and (iii), ensure that δ is continuous and satisfies (23) and (24). The fact that, for μ ∈ {u, v}, g (⋅, u, γ) and γ satisfy (33) ensures that δ satisfies (27). Observe that, by (i) and (iii), g (x, v, γ) ≤ –p for α ≤ x ≤ s and so δ satisfies (25) for α ≤ x ≤ s by Lemma 7. Similiarly, g (x, u, γ) ≥ –p for s ≤ x ≤ Ω by (i) and (ii), and so, by Lemma 7, δ satisfies (25) for s ≤ x ≤ Ω. It remains to show that δ satisfies (25) for 0 ≤ x ≤ α and for Ω ≤ x ≤ Θ.

When h ≥ 0, α = 0, and so we need only show that δ(x, μ), satisfies (25) for Ω ≤ x Θ. If Ω = Θ, there is nothing to show. Otherwise, Θ > Ω = (γ –(M + p)u)/h and so

\[\frac{{{\sigma }^{2}}}{2}{{\delta }_{x}}(x,\mu )+\mu \delta (x,\mu )+p\mu +hx=\mu (M+p)+hx\ge u(M+p)+h\Omega =\gamma \]

for x ≥ Ω and μ ∈ {u, v}.

When h ≤ 0, Ω = Θ, and so we need only show that δ(x, μ) satisfies (25) for 0 ≤ x ≤ α. If α = 0, there is nothing to show. Otherwise, 0 < α = (γ + (U – p) v)/h and so

\[\frac{{{\sigma }^{2}}}{2}{{\delta }_{x}}(x,\mu )+\mu \delta (x,\mu )+p\mu +hx=-\mu (U-p)+hx\ge -v(U-p)+h\alpha =\gamma \]

for x ≤ α and μ ∈ {u, v}.

The switching point s in the policy of Lemma 8 satisfies the two conditions: g (s, μ, γ) = –p and g_x (s, μ, γ) ≥ 0. In Lemma 9 we characterize, for each value of γ above a threshold, the unique points $x_{\mu }^{&#x002A;}(\gamma )$ for μ ∈ {u, v} satisfying these conditions. In Corollary 5 we show that this threshold is in fact a lower bound on the average cost of a policy. The proofs of Lemma 9 is in Appendix A and the proof of Corollary 5 is in the Appendix C.

Lemma 9

Under Assumptions 2 and 3, let

\[\begin{align} & \alpha (\gamma )=\left\{ \begin{matrix} \max \{0,\frac{\gamma +(U-p)v}{h}\}, & h \lt 0, \\ 0, & h\ge 0, \\\end{matrix} \right.\quad \quad \Omega (\gamma )=\left\{ \begin{matrix} \min \{\Theta ,\frac{\gamma -(M+p)u}{h}\}, & h>0, \\ \Theta , & h\le 0, \\\end{matrix} \right. \\ & \begin{array}{&#x002A;{35}{l}} x_{u}^{&#x002A;}(\gamma ) & =\left\{ \begin{array}{&#x002A;{35}{l}} \Theta +\frac{{{\sigma }^{2}}}{2u}\log [\frac{\gamma -(M+p)u}{\gamma }], & h=0, \\ \frac{\gamma }{h}+\frac{{{\sigma }^{2}}}{2u}\left( 1+w(u,\gamma ) \right), & h \ne 0, \\\end{array} \right. \\ x_{v}^{&#x002A;}(\gamma ) & =\left\{ \begin{array}{&#x002A;{35}{l}} \frac{{{\sigma }^{2}}}{2v}\log [\frac{\gamma +(U-p)v}{\gamma }], & h=0, \\ \frac{\gamma }{h}+\frac{{{\sigma }^{2}}}{2v}\left( 1+w(v,\gamma ) \right), & h \ne 0, \\\end{array} \right. \\\end{array} \\ \end{align}\]

where

\[\begin{align} & w(u,\gamma )={{W}_{k(uh)}}\left[{{\text{e}}^{-1+2u(\Omega (\gamma )-\gamma /h)/{{\sigma }^{2}}}}\left(-1+\frac{2u}{{{\sigma }^{2}}}(\Omega (\gamma )-\frac{\gamma -(M+p)u}{h})\right) \right], \\ & w(v,\gamma )={{W}_{k(vh)}} \left[{{\text{e}}^{-1-2v(\gamma /h-\alpha (\gamma ))/{{\sigma }^{2}}}}\left(-1-\frac{2v}{{{\sigma }^{2}}}(\frac{\gamma +(U-p)v}{h}-\alpha (\gamma ))\right)\right] \\ & \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,and\quad k(x)=\left\{ \begin{matrix} -1, & x>0, \\ 0, & x \lt 0. \\\end{matrix} \right. \\ \end{align}\]

Then, for

(34) \[\gamma >\left\{ \begin{matrix} -(U-p)v-\frac{h{{\sigma }^{2}}}{2v}(1+{{W}_{k(vh)}}[-{{\text{e}}^{-1-2(U-p){{v}^{2}}/h{{\sigma }^{2}}}}]), & h>0, \\ \max \left\{ 0,-(U-p)v,(M+p)u \right\}, & h=0, \\ h\Theta +(M+p)u-\frac{h{{\sigma }^{2}}}{2u}(1+{{W}_{k(uh)}}[-{{\text{e}}^{-1+2(M+p){{u}^{2}}/h{{\sigma }^{2}}}}]), & h \lt 0, \\\end{matrix} \right.\]

\[x_{u}^{&#x002A;}(\gamma )\] is the unique real value for z ≤ Ω (γ) satisfying

(35) \[\begin{align} & \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,g(z,u,\gamma )=-p, \\ & {{g}_{x}}(x,u,\gamma )>0\quad for\ all\ z\le x \lt \Omega (\gamma ), \\ \end{align}\]

where C_u(γ) is defined by g(Ω(γ), u, γ) = –U, and \[x_{v}^{&#x002A;}(\gamma )\] is the unique real value for z ≥ α (γ) satisfying

(36) \[\begin{align} & \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,g(z,v,\gamma )=-p, \\ & {{g}_{x}}(x,v,\gamma )>0\quad for\ all\ \alpha (\gamma ) \lt x\le z, \\ \end{align}\]

where C_v(γ) is defined by g(α(γ), v, γ) = M.

In Corollary 5 we show that conditions (34) of Lemma 9 strengthen the lower bounds of Corollary 1 for the problem with two drift rates.

Corollary 5

Under Assumptions 2 and 3, when K ≥ 0

\[\text{AC}(\Phi )\ge \left\{ \begin{matrix} \max \{(M+p)u, & {} \\ \quad -(U-p)v-\frac{h{{\sigma }^{2}}}{2v}(1+{{W}_{k(vh)}}[-{{\text{e}}^{-1-2(U-p){{v}^{2}}/h{{\sigma }^{2}}}}])\}, & h>0, \\ \max \left\{ 0,-(U-p)v,(M+p)u \right\}, & h=0, \\ \max \{h\Theta -(U-p)v, & {} \\ \quad h\Theta +(M+p)u-\frac{h{{\sigma }^{2}}}{2u}(1+{{W}_{k(uh)}}[-{{\text{e}}^{-1+2(M+p){{u}^{2}}/h{{\sigma }^{2}}}}])\}, & h \lt 0, \\\end{matrix} \right.\]

for each policy $\Phi \in \cal P$ .

We conclude this section by using the characterizations of Lemma 9 to prove the existence of the point s and value γ satisfying the conditions of Lemma 8.

Lemma 10

Under Assumptions 2 and 3, there exist s and γ satisfying the conditions of Lemma 8.

Proof. We show that these values exist and how to compute them in the three cases h = 0, h > 0, and h < 0.

We first show that, when γ = min{γ_u, γ_v}, either \[x_{u}^{&#x002A;}(\gamma )=x_{v}^{&#x002A;}(\gamma )\] and the point \[s=x_{u}^{&#x002A;}(\gamma )\] and the scalar γ satisfy the conditions of Lemma 8 or \[x_{u}^{&#x002A;}(\gamma )>x_{v}^{&#x002A;}(\gamma )\] . In the latter case, we show that, as γ approaches the lower bound in (34), \[x_{u}^{&#x002A;}(\gamma )\le x_{v}^{&#x002A;}(\gamma )\] and, since \[x_{\mu }^{&#x002A;}(\cdot )\] for μ ∈ {u, v} is continuous on this domain, there is a value of γ such that \[x_{u}^{&#x002A;}(\gamma )=x_{v}^{&#x002A;}(\gamma )\] . This value of γ and the point \[s=x_{u}^{&#x002A;}(\gamma )=x_{v}^{&#x002A;}(\gamma )\] satisfy the conditions of Lemma 8.

Let γ = min{γ_u, γ_v}, and consider the case in which γ = γ_u ≤ γ_v. The arguments for the case in which γ = γ _{v <} γ_u are analogous.

When γ = γ_u ≤ γ_v, we have, by Corollary 3, α_u ≤ Ω_u = Ω(γ), g_x(x, u γ) > 0 for α_u < x < Ω_u, and if 0 < α_u then g_x(x, u, γ) < 0 for x < α_u. Furthermore, since γ = γ_u = γ _u (α_u, Ω_u), g(α_u, u, γ) = –U. Finally, if h ≥ 0, then α(γ) = α_u = 0 and if h < 0 then α(γ) = max{0 (γ + (U – p)v)/h ≤ max {0 (γ + (U – p)u)/h} = α_u. Let Ω = Ω_u = Ω(γ), and consider the two cases α(γ) = α_u and α(γ) < α_u.

Case 1: α(γ) = α_u. In this case let α = α_u α(γ), and observe that g(α, v, γ) = g(α, u, γ) = –U and, by Lemma 7,

\[\frac{{{\sigma }^{2}}}{2}({{g}_{x}}(\alpha ,v,\gamma )-{{g}_{x}}(\alpha ,u,\gamma ))=(v-u)(U-p)>0.\]

Furthermore, by Lemma 6, g(Ω, v, γ) ≤ M = g(Ω, u, γ). Thus, there is a point S ∈ (α, Ω] such that g(S, u, γ) = g(S, v, γ), g(x, v, γ) = g(x, u, γ) for α < x < S, and g_x(S, v, γ) ≤ g_x(S, u, γ).

If g_x(S, u, γ) = g_x(S, v, γ) then, by Lemma 4, g(S, u, γ) = –p and, by Corollary 3, g_x(x, u, γ) > 0 for S ≤ x < Ω _u , so \[S=x_{u}^{&#x002A;}(\gamma )\] . Since g_x(α, v, γ) > g_x(α, u, γ) > 0 and g_x(S, v, γ) = g_x(S, v, γ) g_x(S, u, γ), and g_x(x, μ, γ) is either convex or concave by Lemma 5, g_x(x, v, γ) > 0 for α ≤ x ≤ S, and so \[S=x_{v}^{&#x002A;}(\gamma )\] as well. But then S and γ satisfy the conditions of Lemma 8.

If g_x(S, u, γ) > g_x(S, v, γ) then, again by Lemma 7, g_x(S, u, γ) > –p and, since g_x(x, u, γ) > 0 for α < x < Ω, we see that \[\alpha \lt x_{u}^{&#x002A;}(\gamma ) \lt S\] and \[g(x_{u}^{&#x002A;}(\gamma ),v,\gamma )>g(x_{u}^{&#x002A;}(\gamma ),u,\gamma )=-p\] , from which it follows that \[\alpha \lt x_{v}^{&#x002A;}(\gamma ) \lt x_{u}^{&#x002A;}(\gamma )\] .

Case 2: α (γ) < α _u. In this case, h < 0 and we argue that g_x(x, v, γ) is positive for x > α(γ). To see this, observe that either α (γ) = 0 > (γ + U – p) v)/h, in which case

\[{{C}_{v}}(\gamma )=-\frac{h}{v}(\frac{\gamma +(U-p)v}{h}+\frac{{{\sigma }^{2}}}{2v}),\]

and so

\[{{g}_{x}}(x,v,\gamma )=-h(\frac{1-{{\text{e}}^{-2vx/{{\sigma }^{2}}}}}{v}-{{\text{e}}^{-2vx/{{\sigma }^{2}}}}\frac{2}{{{\sigma }^{2}}}\frac{\gamma +(U-p)v}{h}),\]

which is positive for all x ≥ 0, or α(γ) = (γ + U – p)v)/h ≥ 0, in which case

\[{{C}_{v}}(\gamma )=-\frac{h}{v}{{\text{e}}^{2v\alpha (\gamma )/{{\sigma }^{2}}}}(\frac{\gamma +(U-p)v}{h}+\frac{{{\sigma }^{2}}}{2v}-\alpha (\gamma ))=-\frac{h{{\sigma }^{2}}}{2{{v}^{2}}}{{\text{e}}^{2v\alpha (\gamma )/{{\sigma }^{2}}}},\]

and so

\[{{g}_{x}}(x,v,\gamma )=-\frac{h}{v}(1-{{\text{e}}^{-2v(x-\alpha (\gamma ))/{{\sigma }^{2}}}}),\]

which is positive for all x > α (γ).

Since g(α(γ), v, γ) = g(α(u), u, γ) = –U and g(⋅, v, γ) is increasing and, by Corollary 3, g(⋅, u, γ) is decreasing on (α(γ), α_u), there must be a unique point s ∈ (α(γ), α_u) such that g(s, v, γ) = g(s, u, γ) and, by Lemma 7, g(s, v, γ) < –p. Furthermore, by Lemma 6, g(Ω, v, γ) ≤ M = g(Ω, u, γ), and so there must be a point S ∈ (α_u, Ω) such that g(S, v, γ) = g(S, u, γ), g(x, v, γ) > g(x, u, γ) for s < x < S and g_x(S, v, γ) ≤ g_x(S, u, γ).

If g_x(S, v, γ) = g_x(S, v, γ) then, by Lemma 7, g(S, v, γ) = –p, and so S and γ satisfy the conditions of Lemma 8.

If g_x(S, v, γ) < g_x(S, u, γ) then, by Lemma 7, g(S, v, γ) > –p. Since g(s, u, γ) = g(s, v, γ) < –p, _{g(x,v,y) > g(x,u,y)} for s < x < S and g(s,v,y) = g(S,u,γ) > –p, it follows that $x_v^&#x002A;(\gamma ) \lt x_u^&#x002A;(\gamma )$ .

We now argue that, as γ reaches the lower bound in (34), $x_u^&#x002A;(\gamma ) \le x_v^&#x002A;(\gamma )$ .

Case (a): h = 0. We consider the case in which u < v < 0. The arguments for the other cases are analogous. In this case, min{0,–(U–p)v (M + p)u} = –(U–p)v > (M + p)u and

$$\mathop {\lim }\limits_{\gamma \to - (U - p)v} x_v^&#x002A;(\gamma ) = {{{\sigma ^2}} \over {2v}}\mathop {\lim }\limits_{\gamma \to - (U - p)v} \log [{{\gamma + (U - p)v} \over \gamma }] = \infty ,$$

while

$$\mathop {\lim }\limits_{\gamma \to - (U - p)v} x_u^&#x002A;(\gamma ) = \Theta + {{{\sigma ^2}} \over {2u}}\log [1 + {{(M + p)u} \over {(U - p)v}}] \lt \Theta .$$

Case (b): h > 0. In this case the lower bound in (34) is

$$\gamma = - (U - p)v - {{h{\sigma ^2}} \over {2v}}(1 + {W_{k(vh)}}[ - {{\rm{e}}^{ - 1 - 2(U - p){v^2}/h{\sigma ^2}}}]),$$

and we see from (43) that w(v,γ) = –1, and so $x_v^&#x002A;(\gamma ) = \gamma /h$ . In this case α = 0 and Ω(γ) = min{Θ (γ – (M + p)u)/h} = (γ – (M + p)u)/h – ε for some ε ≥ 0, and so, if u > 0,

$$w(u,\gamma ) = {W_{ - 1}}[{{\rm{e}}^{ - 2(M + p){u^2}/h{\sigma ^2}}}{{\rm{e}}^{ - 1 - 2u\varepsilon /{\sigma ^2}}}( - 1 - {{2u} \over {{\sigma ^2}}}\varepsilon )] \lt - 1 - {{2u} \over {{\sigma ^2}}}\varepsilon ,$$

where the last inequality follows from the fact that W _–1 is a decreasing function, and

$$x_u^&#x002A;(\gamma ) = {\gamma \over h} + {{{\sigma ^2}} \over {2u}}(1 + w(u,\gamma )) \lt {\gamma \over h} - \varepsilon \le x_v^&#x002A;(\gamma ).$$

If u < 0,

$$w(u,\gamma ) = {W_0}[{{\rm{e}}^{ - 2(M + p){u^2}/h{\sigma ^2}}}{{\rm{e}}^{ - 1 - 2u\varepsilon /{\sigma ^2}}}( - 1 - {{2u} \over {{\sigma ^2}}}\varepsilon )],$$

and either –1–2uε/σ ² ≥ 0, in which case w(u,γ) ≥ 0 and $$x_u^&#x002A;(\gamma ) \lt \gamma /h = x_v^&#x002A;(\gamma ),$$ or –1–2uɛ/σ ² < 0, in which case, since W ₀ is an increasing function, w(u,γ) > –1 –2uɛ/σ ² and

$$x_u^&#x002A;(\gamma ) = {\gamma \over h} + {{{\sigma ^2}} \over {2u}}(1 + w(u,\gamma )) < {\gamma \over h} - \varepsilon \le x_v^&#x002A;(\gamma ).$$

The arguments for the h < 0 case are analogous.

In Figure 1 we illustrate an optimal policy and the associated functions g(·, u, γ) and g(·, v, γ) for a case in which h > 0, Θ is finite, and K = 0.

Figure 1.

Optimal values for s, γ, α(γ), and Ω(γ), and the associated functions g(·, u, γ) (thin line) and g(·, v, γ) (thick line) for a problem with two drift rates where h > 0, Θ is finite, and K = 0. To explore other cases, download a Wolfram computable document format application from https://www2.isye.gatech.edu/faculty/John_VandeVate/EconomicAvgCost.html.

Let $\underline \gamma $ denote the the optimal average cost for the economic average cost Brownian control problem when K = 0.

Corollary 6

Under Assumptions 2 and 3,

$$\eqalign{\alpha (\gamma ) & \lt x_v^&#x002A;(\gamma ) \lt x_u^&#x002A;(\gamma ) \lt \Omega (\gamma ), \cr g(x_v^&#x002A;(\gamma ),u,\gamma ) & \lt - p\quad { and}\quad g(x_u^&#x002A;(\gamma ),v,\gamma ) \gt - p } $$

for $\underline \gamma \lt \gamma \le \min \{ {\gamma _u},{\gamma _v}\} $ .

In Corollary 7 we provide a closed-form expression for the average cost and parameters defining an optimal policy when h > 0 and the upper limit on the buffer capacity is not constraining. Its proof is given in Appendix C.

Corollary 7

Under Assumption 3, when h > 0, let

$$\eqalign{ w & = {W_{k(u)}}[ - {{\rm{e}}^{ - 1 - 2(M + p){u^2}/h{\sigma ^2}}}], \cr s = & \underline \gamma {1 \over h} + {{{\sigma ^2}} \over {2u}}(w + 1), \cr \underline \gamma & = - (U - p)v - {{h{\sigma ^2}} \over {2v}}\left(1 + {W_{k(v)}}({{\rm{e}}^{ - 1 - 2(U - p){v^2}/h{\sigma ^2} + v(w + 1)/u}}({v \over u}(w + 1) - 1))\right),} $$

where k(μ) = 0 if μ < 0 and –1 otherwise. If $$\underline \gamma \le h\Theta + (M + p)u$$ then $\underline \gamma $ and s satisfy the conditions of Lemma 8, proving that $\Phi = \{ (u,s,v,\Omega (\underline \gamma ),u),(v,0,v,s,u)\} $ is an optimal policy and ${\rm{AC}}(\Phi ) = \underline \gamma $ .

Corollary 8 extends Lemma 10 to the problem with more than two drift rates.

Corollary 8

Under Assumptions 2 and 3, when K = 0, an optimal policy for the problem with the two available rates u < v is optimal for the problem with the set of available rates $\{\mu :u \le \mu \le v \} $ .

4.2 When K > 0

In Section 4.1 we addressed the problem in which the cost K to change the drift rate is 0. In this section we address the problem when K > 0. Note that $\overline \gamma = \min \{ {\gamma _u},{\gamma _v}\} $ provides an upper bound and $\underline \gamma $ provides a lower bound on the average cost of a policy in this case. In Lemma 11 we show how, given an average cost γ between these bounds, to construct functions δ and a policy Φ(γ) that satisfy (23)–(25) and (27)–(29), and so prove that Φ(γ) is an optimal policy when $K = \Delta (\gamma ) \equiv \int_0^\Theta (\delta (x,v,\gamma ) - \delta (x,u,\gamma )) {\rm{d}}x$ . In Lemma 12 we argue that Δ(·) is a continuous, increasing function on $ (\underline \gamma ,\overline \gamma ) $ and so conclude that, for $0 \lt K \lt \Delta (\overline \gamma )$ , we can use binary search to find the value $\underline \gamma < \gamma < \overline \gamma$ such that Δ(γ) = K and so Φ(γ) is an optimal policy.

Lemma 11

Under Assumptions 2 and 3, given $\overline \gamma \ge \gamma \gt \underline \gamma ,$ let

$$\eqalign{ & \alpha (\gamma ) = \left\{ {\matrix{ {\max \{ 0,{{\gamma + (U - p)v} \over h}\} } & { \;if\; h \lt 0,} \cr 0 & { \;if\; h \ge 0,} \cr } } \right. \cr & \Omega (\gamma ) = \left\{ {\matrix{ {\min \{ \Theta ,{{\gamma - (M + p)u} \over h}\} } & { \;if\; h \gt 0,} \cr \Theta & { \;if\; h \le 0,} } } \right.} $$

where C_u(γ) is determined by g(Ω(γ), u, γ) = M and C_v(γ) is determined by g(α(γ), v, γ) = –U. Then there exists a unique point $$s(\gamma ) \in [\alpha (\gamma ),x_v^&#x002A;(\gamma ))$$ such that g(s, u, γ) = g(s, v,γ) and a unique point $S(\gamma ) \in (x_u^&#x002A;(\gamma ),\Omega (\gamma )]$ such that g(S, u, γ) = g(S, v,γ) and γ and the functions

(37) $$\delta (x,u,\gamma ) = \left\{ {\matrix{ { - U,} & {0 \le x \lt \alpha (\gamma ),} \cr {g(x,v,\gamma ),} & {\alpha (\gamma ) \le x \lt s(\gamma ),} \cr {g(x,u,\gamma ),} & {s(\gamma ) \le x \lt \Omega (\gamma ),} \cr {M,} & {\Omega (\gamma ) \lt x \le \Theta ,} \cr } } \right.$$

and

(38) $$\delta (x,v,\gamma ) = \left\{ {\matrix{ { - U,} & {0 \le x \lt \alpha (\gamma ),} \cr {g(x,v,\gamma ),} & {\alpha (\gamma ) \le x \lt S(\gamma ),} \cr {g(x,u,\gamma ),} & {S(\gamma ) \le x \lt \Omega (\gamma ),} \cr {M,} & {\Omega (\gamma ) \lt x \le \Theta ,} \cr } } \right.$$

satisfy (23)–(25) and (27)–(29) with respect to the policy Φ( γ) = {(u, s(γ), v, Ω(γ), u) (v, α(γ), v, S(γ), u)}, proving that Φ(γ) is an optimal policy for the problem with changeover cost $K = \Delta (\gamma ) = \int_0^\Theta \left( {\delta (x,v,\gamma ) - \delta (x,u,\gamma )} \right) {\rm{d}}x$ .

Proof. We first argue that there is a point $$S \in (x_u^&#x002A;(\gamma ),\Omega (\gamma )]$$ such that g(S, v, γ) = g(S, u, γ). To see this, observe that, by Corollary 6, $g(x_u^&#x002A;(\gamma ),v,\gamma ) \gt g(x_u^&#x002A;(\gamma ),u,\gamma ) = - p$ and, by Lemma 6, g(Ω(γ), v, γ) ≤ g(Ω(γ), u, γ) = M. Thus, there is a point $x_u^&#x002A;(\gamma ) \lt S \le \Omega (\gamma )$ such that g(S, v, γ) = g(S, u, γ). Similarly, since $g(x_v^&#x002A;(\gamma ),u,\gamma ) \lt g(x_v^&#x002A;(\gamma ),v,\gamma ) = - p$ , by Corollary 6 and g(α(γ), u, γ) ≥ g(α(γ), u, γ) = –U by Lemma 6, there is a point $\alpha (\gamma ) \le s \lt x_v^&#x002A;(\gamma )$ such that g(s, v, γ) = g(s, u, γ).

We next observe that, since g(·, u, γ) is increasing on $[x_u^&#x002A;(\gamma ),\Omega (\gamma ))$ by Lemma 9, g(S, u, γ) > –p and so, by Lemma 7, g_x(S, v, γ) < g _x (S, u, γ). Similarly, since g(·, v, γ) is increasing in $$(\alpha (\gamma ),x_v^&#x002A;(\gamma )]$$ by Lemma 9, g(s, v, γ) < –p and so, by Lemma 7, g_x(s, v, γ) > g_x(s, v, γ).

To see that s and S are unique, suppose that there are roots S < S ₁ < S ₂ < … ≤ Ω(γ) and s > s ₁ > s ₂ > … ≥ α(γ) of g(·, v, γ) – g(·, u, γ). Since g_x(S, v, γ) < g_x(S, u, γ), we must have g_x(S ₁, v, γ) ≥ g_x(S ₁, u, γ) and so, by Lemma 7, g(S ₁, u γ) ≤ –p, which contradicts the fact that g(·, u, γ) is increasing on $[x_u^&#x002A;(\gamma ),\Omega (\gamma ))$ . Similarly, since g_x(x, v, γ) > g_x(s, u, γ), we must have g_x(s ₁, v, γ) ≤ g_x(s ₁, u, γ) and so, by Lemma 7, g(s ₁, v, γ) ≥ –p, which contradicts the fact the g(·, v, γ) is increasing on $ (\alpha (\gamma ),x_v^&#x002A;(\gamma )] $ .

We next argue that the functions δ satisfy (25). The functions g(·, u, γ) and g(·, v, γ) satisfy (25) with equality for all real x. Since g(·, v, γ) is increasing on $ (\alpha (\gamma ),x_v^&#x002A;(\gamma )] $ , the fact that $\alpha (\gamma ) \le s < x_v^&#x002A;(\gamma )$ ensures that g(x, v, γ) < –p for all x ∈ [α(γ), s] and so, by Lemma 7, δ(x, u, γ) satisfies (25) for x ∈ [α(γ), s]. Similarly, we see that g(·, u, γ) > –p for x ∈ [S, Ω(γ)] and so δ(x, v, γ) satisfies (25) for x ∈ [S, Ω(γ)]. If Ω(γ) < Θ then h > 0, Ω(γ) = (γ –(M + p)u)/h < Θ, and

$${{{\sigma ^2}} \over 2}{\delta _x}(x,\mu ,\gamma ) + \mu \delta (x,\mu ,\gamma ) + p\mu + hx = \mu (M + p) + hx \ge u(M + p) + h\Omega (\gamma ) = \gamma ,$$

proving that δ satisfies (25) on (Ω)(γ), Θ]. Similarly, if α (γ) > 0 then h < 0, α (γ) = (γ + (U – p)v)/h , and

$${{{\sigma ^2}} \over 2}{\delta _x}(x,\mu ,\gamma ) + \mu \delta (x,\mu ,\gamma ) + p\mu + hx = \mu (M + p) + hx \ge u(M + p) + h\Omega (\gamma ) = \gamma ,$$

proving that δ satisfies (25) on [0, α (γ)).

The facts that s and S are roots of g(·, v, γ) – g(·, u, γ), g(α(γ), v, γ) = – U, and g(Ω(γ), u, γ) = M ensure that δ(·, μ, γ) is continuous for each μ ∈ {u, v} and satisfies (24).

To see that δ satisfies (23), observe that the choices of s(γ) and S(γ)ensure that δ(x, v, γ) ≥ δ(x, u, γ) for all $x \in {\cal R}$ .

Note that, by Lemma 6, g(x, v, γ) ≤ M and g(x, u, γ) ≥ –U for all x ∈ (α(γ), Ω(γ)). The facts that g(α(γ), v, γ) = –U, g(x, v, γ) is increasing on $ (\alpha (\gamma ),x_v^&#x002A;(\gamma )],$ and g(x, v, γ) ≥ g(x, u, γ) on (s(γ), S(γ)) ensure that g(x, v, γ) ≥ –U for x ∈ [α(γ), S(γ)]. Similarly, the facts that g(Ω(γ), u, γ) = M, g(⋅, u, γ) is increasing in $[x_u^&#x002A;(\gamma ),\Omega (\gamma )],$ and g(x, u, γ) ≤ g(x, v, γ) for x ∈ (s(γ), S(γ)) ensure that g(x, u, γ) ≤ M for x ∈ [s(γ), Ω(γ)].

Finally, by assumption, K = Δ(γ) and so δ and γ satisfy (22). Thus, we conclude that δ, γ, and Φ(γ) satisfy all the conditions of Proposition 3 and Corollary 4, proving that γ is a lower bound on the cost of any nonanticipating policy and, since AC(Φ(γ)) = γ, Φ(γ) is an optimal policy.

In Figure 2 we illustrate the policy and the functions $g( \cdot ,u,\overline \gamma )$ and $g( \cdot ,v,\overline \gamma )$ constructed in Lemma 11 for a case in which h > 0, Θ is finite, $$\overline \gamma = {\gamma _v} \lt {\gamma _u},$$ and $K = \overline K $ .

Figure 2.

The policy and the functions $g( \cdot ,u,\overline \gamma )$ and $g( \cdot ,v,\overline \gamma )$ constructed in Lemma 11 for a case in which h > 0, Θ is finite, $\overline \gamma = {\gamma _v} \lt {\gamma _u},$ and $K = \overline K $ . Here $\Delta (\overline \gamma )$ is the highlighted area between $g( \cdot ,v,\overline \gamma )$ and $g( \cdot ,u,\overline \gamma )$ . To explore other cases, download a Wolfram computable document format application from https://www2.isye.gatech.edu/faculty/John_VandeVate/EconomicAvgCost.html.

Corollary 9

Under Assumptions 2 and 3, if

$K \gt \overline K \equiv \Delta (\overline \gamma ) = \int_0^\Theta (\delta (x,v,\overline \gamma ) - \delta (x,u,\overline \gamma )) {\rm{d}}x,$

where δ is defined by (37) and (38), then $$\overline \gamma $$ is a lower bound on the average cost of a policy and the better of the two single-rate policies is an optimal policy.

Observe that Δ(⋅) is a continuous function of γ with $\Delta (\overline \gamma ) = \overline K $ and $\Delta (\underline \gamma ) = 0$ . Thus, for each $\overline K \gt K \gt 0,$ there exists $\overline \gamma \gt \gamma \gt \underline \gamma $ such that Δ(γ) = K, and so γ, δ(⋅, μ, γ) for μ ∈ {u, v}, and Φ(γ) satisfy (22)–(29), proving that γ is a lower bound on the average cost of a policy and, since AC(Φ(γ)) = γ, Φ(γ) is an optimal policy. In Lemma 12 we show that Δ is increasing and so we can find γ using binary search.

Lemma 12

The function

$$\Delta (\gamma ) = \int_0^\Theta (\delta (x,v,\gamma ) - \delta (x,u,\gamma )) {\rm{d}}x,$$

where δ is defined by (37) and (38), is continuous and increasing on $ (\underline \gamma ,\overline \gamma ) $ .

Proof. Since g(s(γ), u, γ)) = g(s(γ), v, γ) and g(S(γ), u, γ) = g(S(γ), v, γ),

$$\eqalign{{\Delta ^{'}}(\gamma ) & = \int_{s(\gamma )}^{S(\gamma )} ({\partial _\gamma }g(x,v,\gamma ) - {\partial _\gamma }g(x,u,\gamma )) {\rm{d}}x \cr & = \int_{s(\gamma )}^{S(\gamma )} ({1 \over v} + {{\rm{e}}^{ - 2vx/{\sigma ^2}}}C_v^{'}(\gamma ) - {1 \over u} - {{\rm{e}}^{ - 2ux/{\sigma ^2}}}C_u^{'}(\gamma )) {\rm{d}}x \cr & = \int_{s(\gamma )}^{S(\gamma )} ({1 \over v}(1 - {{\rm{e}}^{2v(\alpha (\gamma ) - x)/{\sigma ^2}}}(1 + {{2v} \over {{\sigma ^2}}}(\gamma + (U - p)v - h\alpha (\gamma )){\alpha ^{'}}(\gamma ))) \cr & \quad - {1 \over u}(1 - {{\rm{e}}^{2u(\Omega (\gamma ) - x)/{\sigma ^2}}}(1 + {{2u} \over {{\sigma ^2}}}(\gamma - (M + p)u - h\Omega (\gamma )){\Omega ^{'}}(\gamma )))) {\rm{d}}x,} $$

where

$$\eqalign{{\alpha ^{'}}(\gamma ) & = \left\{ {\matrix{ 0 & {{\rm{if}}\;h \ge 0\;{\rm{or}}(\gamma + (U - p)v)/h \lt 0,} \cr {{1 \over h}} & {{\rm{if}}\;h \lt 0\;{\rm{and}}(\gamma + (U - p)v)/h \ge 0,} \cr } } \right. \cr & {\Omega ^{'}}(\gamma ) = \left\{ {\matrix{ 0 & {{\rm{if}}\;h \le 0\;{\rm{or}}(\gamma - (M + p)u)/h \gt \Theta ,} \cr {{1 \over h}} & {{\rm{if}}\;h \gt 0\;{\rm{and}}(\gamma - (M + p)u)/h \le \Theta ,} \cr } } \right. \cr} $$

and so

$${\Delta ^{'}}(\gamma ) = \int_{s(\gamma )}^{S(\gamma )} ({1 \over v}(1 - {{\rm{e}}^{2v(\alpha (\gamma ) - x)/{\sigma ^2}}}) - {1 \over u}(1 - {{\rm{e}}^{2u(\Omega (\gamma ) - x)/{\sigma ^2}}})) {\rm{d}}x \gt 0$$

so long as s(γ) < S(γ), i.e. so long as $\gamma \gt \underline \gamma $ . □

5. Conclusion

In this paper we extended a classical problem of controlling a Brownian motion by allowing the controller to employ instantaneous controls at any time. We characterized the conditions under which the resulting economic average cost Brownian control problem admits an optimal policy and showed that, when it does, a control band policy is optimal. We developed a simple method to calculate optimal policy parameters, namely economic bounds on the buffer and bands within which each drift rate should be used. This type of policy is straightforward to understand and implement, and could provide significant savings over ad-hoc methods. Furthermore, we provided explicit formulae for critical performance metrics, such as the frequencies with which the drift rate changes, work is rejected, and capacity is idled, for any control band policy, thereby helping the controller to better evaluate the impact of a chosen policy.

Our approach provides a clean and analytical solution which lends itself to further generalizations in terms of the form of the cost function. A future direction worth pursuing is to extend the method to handle the cases with more than two available rates.