2. ASYMPTOTIC EXPRESSIONS FOR MSPEP_h(k) AND MSPED_h(k) IN STATIONARY AR MODEL

In the discussion that follows, we shall frequently use the following condition on the distribution function of ε₁,F(·). For some positive numbers α, η, and M,

provided |x − y| < η. Condition (8) is fulfilled by any distribution function with bounded density. Some distribution functions with unbounded density also satisfy (8). For more details on this condition, see Findley and Wei (2002, Sect. 4). When data are generated from model (1), Theorems 1 and 2 deal with asymptotic expressions for MSPEP_h(k) and MSPED_h(k) when k ≥ p, respectively. Their proofs are given in Appendix A.

THEOREM 1. Assume that observations x₁,…,x_n are generated from model (1) with ε₁ satisfying (8) and

where θ_h = max{8,2(h + 1)} + δ for some δ > 0. Then, for k ≥ p and h ≥ 1,

Remark 1. Condition (8) is used to ensure that for any q > 0,

holds for n sufficiently large, where

and ∥G∥ = sup_∥z∥≤1∥Gz∥ for matrix G (see Lemma A.1 in Appendix A for more details). Equality (11) was first proven by Fuller and Hasza (1981) for stationary Gaussian processes. For various generalizations of their result, see Bhansali and Papangelou (1991), Papangelou (1994), Ing and Wei (2002), and Findley and Wei (2002). By directly assuming that (11) holds, Kunitomo and Yamamoto (1985, Corollary 6) have established (10) with θ_h in (9) replaced by max{32,16h}. Obviously, their assumptions are more stringent than those used in Theorem 1.

THEOREM 2. Let the assumptions of Theorem 1 hold with θ_h in (9) replaced by 8 + δ, for some δ > 0. Then, for k ≥ p and h ≥ 1,

and for random vector z, cov(z) = E[(z − E(z))(z − E(z))′].

A comparison between the first terms on the right-hand side of (10) and (12), f_1,h(k) and f_2,h(k), respectively, is given as Proposition 3.2 by Bhansali (1997), which shows that for k ≥ p ≥ 1 and h ≥ 2,

This result and Theorems 1 and 2 together imply that when the model is correctly specified, the plug-in predictor has an advantage over the direct predictor, provided the terms of order O(n^−3/2) are ignored. For example, assume h = 2 and k ≥ p ≥ 1. Then,

and

(Recall that a_k = 0 for k > p and that |a_p| < 1 in the stationary case.) Hence, for k ≥ p,

Moreover, let k ≥ p ≥ 1 be fixed. Then, it can be shown that f_2,h(k) − f_1,h(k) is a nondecreasing function of h. Therefore, the advantage of the plug-in predictor with respect to the direct predictor becomes more significant as h increases. To see this, observe that

where the third equality follows from A′(k)R(k)A(k) = R(k) − σ²e_ke_k′, with k-dimensional vector e_k = (1,0_k−1′)′. In addition, one also has

As a result,

which gives the asserted property.

The following result establishes that the second-order terms of MSPEP_h(k) and MSPED_h(k) are strictly increasing functions of k in the correctly specified case.

THEOREM 3.

where with the convention that

is a (k + 1)-dimensional vector.

Remark 2. When 1 ≤ h ≤ 5, it can be shown that either (19) or (20) holds for all k ≥ p. In addition, if h − 1 ≤ k, then the (k + 2 − h)th entry of l* is b₀² = 1. In these cases, (18) holds without extra constraints on the parameter space. In general, (19) (with h ≥ 2) and (20) only exclude at most a zero-volume subset in the parameter space.

Under the assumptions of Theorem 1 and (20) (or (19)), we conclude from (10) and (18) that for h ≥ 1 and k ≥ p,

Similarly, when the assumptions of Theorem 2 hold, (12) and (21) also yield

for h ≥ 1 and k ≥ p.

Kunitomo and Yamamoto (1985, Theorem 4) obtain that f_1,h(k + 1) ≥ f_1,h(k) when k ≥ p and h ≥ 2 (see also (A.26)). However, from the order selection point of view, Kunitomo and Yamamoto's result is not sharp enough. To see this, first consider the following multistep generalizations of accumulated prediction errors (APEs) based on sequential plug-in and direct predictors:

respectively, where m is the smallest positive number such that

are well defined for all i ≥ m. Note that the APE with h = 1 was first proposed by Rissanen (1986) for the purpose of use for determining p. For h = 1, Hemerly and Davis (1989), under certain conditions, prove that

is a strongly consistent estimator of p in the sense that

Here, K is a prescribed upper bound of p. Under the conditions similar to those used in Hemerly and Davis (1989), Ing (2002) further shows that for h > 1

and

as k ≥ p. As is clear from (24) (or (25)), without the strict inequality, (18) (or (21)), it does not seem possible for

with h ≥ 2 to be strongly consistent. For more details on the statistical properties of APEP_n,h(k) and APED_n,h(k), see Ing (2002).

The following theorem compares MSPEP_h(k) and MSPED_h(k) in the situation where k < p.

THEOREM 4. Assume that observations x₁,…,x_n are generated from model (1) with 1 ≤ p ≤ ∞. (Here, p = ∞ means that {x_t} is truly an AR(∞) process. In this case,

is assumed instead of a_p ≠ 0.) Also assume that ε₁ satisfies (8) and E(|ε₁|^4h+2+δ) < ∞, for some δ > 0. Then, for h ≥ 2 and 1 ≤ k < p,

where a_D(h,k) = R⁻¹(k)(γ_h,…,γ_h+k−1)′ and a(h,k) is defined in Section 1. Moreover, if

for at least one positive integer s satisfying h ≤ s ≤ k + h − 1, then for h ≥ 2 and 1 ≤ k < p,

where γ_j* = γ_j for 1 ≤ j ≤ k and γ_j* = a_D(1,k)′(γ_j−1*,…,γ_j−k*)′ for j ≥ k + 1.

Remark 3. Because p is allowed to be infinite, Theorem 4 covers the causal and invertible autoregressive moving average (ARMA) process as a special case. When (27) is violated, γ_max{h,k+1},…,γ_k+h−1 must be functions of γ₀,…,γ_k. Therefore, (27) only excludes at most a zero-volume subset in the parameter space. In particular, (27) holds automatically in the situation where p ≤ k + h − 1 and h ≤ k + 1 (e.g., h = 2 and k = p − 1), because if (27) is violated in this situation, then a_k+1 = ··· = a_p = 0, which contradicts the assumption that a_p ≠ 0.

Theorem 4 and Remark 3 assert that in the misspecified case, the direct predictor is superior to the plug-in predictor for almost all points in the parameter space. On the other hand, we have found examples showing that when (27) is violated, the plug-in predictor can be better than the direct predictor in the sense of (7). However, to obtain a more conclusive result for the situation where (27) does not hold, it is necessary to compare MSPEP_h(k) and MSPED_h(k), up to terms of order 1/n. Because this comparison is quite complicated, we do not pursue these details in this paper. For a related result on an asymptotic expression for MSPEP_h(k), up to terms of order 1/n, in the misspecified case, see Kunitomo and Yamamoto (1985, Theorem 3).

3. EXTENSIONS TO STATIONARY AR MODELS AROUND DETERMINISTIC TIME TRENDS

This section investigates performances of the plug-in and direct predictors in stationary AR models around deterministic time trends. Assume that observations x₁,…,x_n come from the following model:

where q₀ is a nonnegative integer, β_i's are unknown real numbers, and p, a_i's and ε_t are defined in model (1). We also assume that q₀ in model (29) is known to simplify the discussion. With an argument similar to that used in Theorem 3.1.1 of Brockwell and Davis (1987) and some algebraic manipulations, x_t can be expressed as

where β_i*'s are functions of β_i's and a_i's and

with b_i's defined in Section 1, is a stationary AR(p) process satisfying model (1). For example, if q₀ = 1 (linear time trend), then

Under model (29), the plug-in predictor of

, has the following representation:

where

is a (q₀ + 1 + k)-dimensional vector,

,

is a (q₀ + 1 + k) × (q₀ + 1 + k) matrix. Here,

is a (q₀ + 1) × (q₀ + 1) upper triangular matrix with C_l^r = r!/(l!(r − l)!), 0 ≤ l ≤ r,

is a (q₀ + 1) × k matrix, 0_k×(q0+1) is a k × (q₀ + 1) matrix of zeros, and

is the matrix

(see Section 1) with

replaced by

. The corresponding direct predictor,

, is

Corollary 1 establishes an asymptotic expression for the MSPE of

, k ≥ p, under non-Gaussian settings.

COROLLARY 1. Assume that in model (29) ε₁ satisfies (8) and for any q > 0,

Then, for k ≥ p and h ≥ 1,

where

with 1_q0+1 denoting a vector of 1's of dimensional q₀ + 1 and

being a (q₀ + 1) × (q₀ + 1) matrix.

Remark 4. As observed in (34), the (second-order) MSPE of the plug-in predictor is the sum of two terms. By comparing (34) with (10), the first term, C_q0,h, can be viewed as the loss due to forecasting the deterministic time trend,

, in (30), and the second term, f_1,h(k), can be viewed as the loss due to forecasting the stationary process, y_t, in (30). Equality (34) considerably extends (2.5) of Fuller and Hasza (1981), which provides an asymptotic expression for

in the situation where q₀ = 0 (constant time trend) and ε_i's are independent Gaussian random variables, each with mean 0 and variance σ².

Corollary 2 provides an asymptotic expression for the MSPE of

with k ≥ p.

COROLLARY 2. Let the assumptions of Corollary 1 hold. Then, for k ≥ p and h ≥ 1,

The (second-order) MSPE of the direct predictor is also the sum of two different losses due to forecasting two different components in model (30), and the loss due to forecasting the deterministic time trend is exactly the same as that of the plug-in predictor. To illustrate the preceding theoretical results, assume that q₀ = 1 and h = 2. In this case, C_q0,h = 4(1 + a₁)²σ². Hence, for k ≥ p,

In general, one has for k ≥ p and h ≥ 2,

Therefore, the plug-in predictor is still better than the direct predictor in the sense of (36). In addition, by Corollaries 1 and 2 and Theorem 3, when k ≥ p and h ≥ 1, the strict parsimony principle still holds for the plug-in and direct predictors in stationary AR models with deterministic time trends.

The following corollary (Corollary 3) compares the performances of

and

with k < p. It shows that if (27) holds, then for h > 1 and k < p, the direct predictor is better than the plug-in predictor, a conclusion that coincides with that of Theorem 4 for the zero mean process. Like Theorem 4, Corollary 3 also allows p in model (29) to equal ∞. In this case,

is assumed instead of a_p ≠ 0.

COROLLARY 3. Assume that in model (29) 1 ≤ p ≤ ∞ and ε₁ satisfies (8) and (33). Then, for h ≥ 2 and 1 ≤ k < p,

Moreover, if (27) holds, then for h ≥ 2 and 1 ≤ k < p,

4. SOME AMBIGUITIES ASSOCIATED WITH PLUG-IN AND DIRECT PREDICTORS IN NONSTATIONARY AR MODELS

Although the advantage of the plug-in predictor with respect to the direct predictor is guaranteed in the situation where the underlying process is stationary and k ≥ p, this is not the case for the random walk model,

even if the assumed model is correctly specified. Throughout this section, we assume that {ε_t} is a sequence of independent Gaussian random variables, each with mean 0 and variance σ².

PROPOSITION 1. Let observations x₁,…,x_n come from model (37) with the initial condition x₀ = 0. Then for h ≥ 1,

Consequently,

The proof of Proposition 1 is similar to that of Corollary 1 of Ing (2001). We omit the details. In the rest of this section, we consider the following nonstationary AR(2) model:

Proposition 2, which follows, shows that under model (38) the plug-in two-step-ahead predictor obtained from the AR(2) model and that obtained from the AR(3) model have the same prediction efficiency. (Recall that in the stationary AR(2) model, Remark 2 and (22) show that the plug-in two-step-ahead predictor obtained from the AR(2) model is strictly better than that obtained from the AR(3) model.)

PROPOSITION 2. Let observations x₁,…,x_n come from model (38) with the initial condition x_i = 0 for i = 0,−1. Then,

Remark 5. A complete verification of Proposition 2, which requires a rather subtle argument, is not presented in this paper, but it is available from the author. We also note that the ambiguity encountered with plug-in predictors stated in Proposition 2 does not exist for their corresponding direct predictors because it can be shown that

The main purpose of this section is to show that some stationary case results on MSPEs of the plug-in and direct predictors can not be generalized to nonstationary cases. However, to gain a deeper understanding of these predictors' performances in nonstationary cases, extensions of Proposition 1 (or 2) to more general nonstationary AR models are needed. Further investigation into this question would be interesting.