2. TREND REGRESSION OF UNTRENDED TIME SERIES

The following development concentrates on the scalar case. Only minor modifications are required to extend the results to the vector case and matrix HAC estimation. Accordingly, let u_t be a weakly dependent time series satisfying

where ε_t = iid(0,σ²) and E(|ε_t|^v) < ∞, for some v > 2.

The time series u_t is stationary with variance

, autocovariance function

, finite vth absolute moment

, spectrum f_u(λ) = (σ²/2π)|C(e^iλ)|², and long-run variance ω² = 2πf_u(0) = σ²C(1)². The summability condition in L ensures that

which is helpful in some technical derivations that follow and means that f_u(λ) has continuous second derivative

. Allowance for heterogeneity in ε_t and u_t can be made in the usual way with minor modifications to L (cf. Phillips and Solo, 1992) without affecting the procedures or the properties discussed subsequently in an essential way.

Under L, partial sums

satisfy the functional law (e.g., Phillips and Solo, 1992)

where [lfloor ]a[rfloor ] signifies the integer part of a, ⇒ is weak convergence, and B(·) is Brownian motion with variance ω².

Let {φ_k}_k=1^∞ be a complete orthonormal system in L₂ [0,1]. Later, we will work with the explicit sequence (8), but it is sufficient to assume that the functions φ_k are twice continuously differentiable on [0,1] . We propose a regression of u_t on a collection of K deterministic regressors {φ_k(t/n)}_k=1^K formed by taking the first K members of this orthonormal sequence evaluated over t = 1,…,n. Write this regression in the form

where φ_Kt = (φ₁(t/n),…,φ_K(t/n))′. In observation format (3) can be written as

, with

.

Let P_K = Φ_K(Φ_K′Φ_K)⁻¹Φ_K′ and construct the estimate

As shown in Lemma A in Section 6,

. Standard functional limit arguments and Wiener integration reveal that for fixed K as n → ∞ we have

It follows immediately that

where χ_K² is chi-squared with K degrees of freedom. For fixed K, the asymptotic mean and variance of (5) are

as n → ∞. These results motivate the long-run variance estimator

which, in view of (6), can be expected to be consistent for ω² when K → ∞ as n → ∞.

The estimate

is simply the sample variance of u_PK, the data projected onto the space spanned by the regressors Φ_K. Thus,

is that part of the sample variance of u_t explained by the regression of u_t onto a deterministic trend basis. This explained sum of squares may be regarded as another way of thinking about a long-run variance—the contribution to the variation of u_t that comes from long-run (or trendlike) behavior in the series. Thus, there would seem to be a strong heuristic motivation for considering estimates like (7).

Note that

is a nonnegative definite quadratic form in the data, whose matrix is the projection P_K. Thus,

belongs to the general class of quadratic estimators of ω². General quadratic estimators were considered in the early spectral analysis literature but have received little subsequent attention relative to kernel estimates. One reason is that, for every such quadratic estimator, one can find a corresponding lag kernel estimator with smaller mean squared error (Grenander and Rosenblatt, 1957, p. 129). Interestingly, as we will show later,

turns out itself to be asymptotically equivalent to a lag kernel estimator and to have nice asymptotic properties analogous to those of quadratic kernel estimates. Thus, there is no need to adapt

into kernel form, and, of course,

is positive by construction.

To develop a consistent estimation technique, we need to allow for the number of regressors K to pass to infinity with the sample size n in such a way that the regression (3) remains feasible. Accordingly, we impose the following rate condition on K:

which requires K to go to infinity faster than

but more slowly than n. To establish a central limit theorem for

, we need the further condition K = o(n^4/5), which controls the expansion rate so that there is no bias in the limit.

For an explicit limit theory, including an explicit expression for the limiting bias of

, it is convenient to use the orthonormal sequence

The functions (8) are the eigenvectors of the covariance kernel of Brownian motion (cf., Phillips, 1998) and form an orthonormal system for L₂ [0,1]. Of course, other orthonormal sequences can be used. However, it turns out that the asymptotic results given subsequently are invariant to the choice of the orthonormal sequence within the class of trigonometric polynomials because the estimates are asymptotically equivalent to the same lag kernel estimator. This point is discussed in Section 4.

Under this setup, we give formulas for the limiting bias, variance, and mean squared error and a limit distribution theory for

. The details of the proofs are different from those of the conventional literature on HAC estimation and are of some independent interest, so they are provided here. But the final results end up being qualitatively similar, as the following result shows.

THEOREM 1. Under conditions L and R

Part (i) shows that

has bias of order K²/n² of the form

From (ii), the variance of

is of O(K⁻¹). So, increases in the number of regressors K increase bias and reduce variance. The situation is analogous to bandwidth choice in kernel estimation.

The mean squared error of

has the form

Optimization of this quantity with respect to K leads to the first-order condition (4K³/n⁴)D² − (2ω⁴/K²) = 0, which gives the following formula for the optimal value of K:

Of course, this is analogous to conventional mean squared error optimization formulas for bandwidth choice in kernel estimation (e.g., Grenander and Rosenblatt, 1957).

Formula (9) can be used to implement a data-determined choice of K in a conventional way. One approach is to use nonparametric estimates of D² and ω⁴ in (9) as, for example, in Newey and West (1994). The most common and convenient method in practice is a simple plug-in estimator based on the use of a parametric model for preliminary estimation of ω⁴ and D² in (9). In the case of a first-order autoregression with fitted coefficient

and error variance s², the standard formulas give

. Some modifications to these formulas may be desirable in cases where

is close to unity. In the context of prewhitening, for example, Andrews and Monahan (1992) propose a 0.97 rule in which

is replaced by 0.97 whenever

exceeds this value. It is known that this particular rule seriously interferes with power in some cases, especially stationarity testing (cf. Lee, 1996). An alternative boundary restriction that seems to improve the size and power properties of procedures based on HAC estimates is the sample-size-dependent rule given in Sul, Phillips, and Choi (2003), where

is replaced by

whenever it exceeds that value.

3. ASYMPTOTIC FORM AS A KERNEL ESTIMATOR

Lemma B(iii) in Section 6 shows that the orthogonal sequence φ_k(r) in (8) satisfies the following summation formula:

Using this formula and (27), (31), and (36) from the proof of part (ii) of Theorem 1, we find that the HAC estimate

has the following asymptotic form:

where

may be regarded as a lag kernel function and

is the sample autocovariance. The dominant term in (11) has the usual form of a kernel estimate of ω² and is dependent on K, which serves the role of a smoothing parameter. Thus,

behaves asymptotically like a kernel estimate.

Let n = KM. For h/n small we can write the lag kernel k_Kn(h/n) as a function of h/M in approximate form as follows:

with which we may associate the function

which is the lag kernel for the Daniell estimate (e.g., Priestley, 1981, p. 441). This lag kernel is a smoothed periodogram estimate with a rectangular spectral window. Here, 2π/M is the width of the frequency band over which the periodogram is being smoothed. The regression-based estimate

is therefore closely related to this well-known kernel estimate of ω² and has the same bias, variance, and mean squared error as the Daniell estimate. Evidently, therefore, the asymptotic mean squared error of

is dominated by that of the Bartlett–Priestley quadratic spectral window (Priestley, 1981; Andrews, 1991).

4. THE EFFECT OF DIFFERENT TREND BASES

We may choose to use other sequences of orthogonal functions on [0,1] in the regression (3), and it is interesting to explore the effects of such alternate choices on the asymptotic kernel form of estimate

. In the following discussion, we confine our attention to sequences of trigonometric polynomials.

Parts (i) and (ii) of Lemma C in Section 6 give the following summation formulas for the orthonormal sine and cosine trigonometric polynomials

, respectively:

As in the calculation leading to (11), the second component of (13) turns out to be of smaller order as K,n → ∞, and (11) holds with

which is asymptotically equivalent to the lag kernel (12). Thus, these different orthonormal trigonometric sequences all lead to HAC estimates that are asymptotically equivalent.

If the complex orthonormal sequence φ_k(r) = e^−2πikr is used, then the regression coefficients in (3) are themselves complex and have the form

, where Φ_K is the matrix of observations of the K regressors {φ_k(t/n) : k = 1,…,K} and * signifies complex conjugation and transposition. In place of (3) we can write

which may be regarded as a fitted empirical version of the Cramér representation of the stationary process u_t, which was noticed earlier in Phillips (1996, Remark 5.2a). Note that λ_k = 2πk/n → 0 for all k = 1,…,K because K/n → 0. Thus, the regression (14) focuses attention on the zero frequency (or long-run) component of the Cramér representation of u_t.

The matrix Φ_K satisfies Φ_K*Φ_K = nI_K and is a scaled unitary matrix. So the kth element of

is simply the (standardized) discrete Fourier transform,

, which is well known to satisfy a central limit theorem (e.g., Hannan, 1970, p. 224) upon rescaling. Additionally, as shown in Phillips (1999, Theorem 3.2), the (asymptotically infinite) collection of K such elements have the following limit as n → ∞ and are asymptotically independent provided K → ∞ but not too fast relative to n. In particular

As is easily seen, the limit variates

are independent complex Gaussian N_c(0,ω²). Letting P_K = Φ_K(Φ_K*Φ_K)⁻¹Φ_K*, we then have

As such, the estimate

may be interpreted as the sample variance of the empirical estimates (obtained from the fitted regression (14)) of the orthogonal process (at the zero frequency) that appears in the Cramér representation of u_t.

Observe that |ζ_nk|² is 2πI_u(λ_k), where I_u(λ) is the periodogram of u_t and λ_k = 2πk/n are the fundamental frequencies. Thus we can write

which corresponds to a smoothed periodogram estimate of the spectrum at the zero frequency (given that K/n → 0). The spectral window in the estimate

is clearly rectangular, just as that of the Daniell window (e.g., Hannan, 1970, p. 279). Again the results are asymptotically equivalent to those of the estimate (7) based on the sinusoidal sequence (8).

Of course, this setup easily permits the use of other spectral windows. Let W_K = diag{W(λ₁),…,W(λ_K)} be a diagonal matrix prescribing a particular weighting sequence based on the window function W(λ). Then the HAC estimate

has the usual form of a smoothed periodogram estimate with spectral window W(λ). In this way, the present approach accommodates all conventional kernel-based HAC estimates.

6. ADDITIONAL LEMMAS AND PROOFS

LEMMA A. Under R,

, as n → ∞.

Proof. We first provide a direct calculation when the elements of φ_Kt are given by the trigonometric functions (8). In this case, the diagonal elements of

are

and the off-diagonals are

It follows that

.

In the general case, if φ_k(s) is twice continuously differentiable on [0,1] by Euler summation we have

because φ_k(t/n) and φ_k′(t/n) are uniformly bounded on [0,1]. Similarly,

uniformly for k ≠ [ell ]. █

LEMMA B. For

, we have

where Δ is the differencing operator Δφ_k(t/n) = φ_k(t/n) − φ_k(t − 1/n).

Proof of Lemma B. For part (i)

For part (ii)

For part (iii)

For part (iv), we use Fejér's integral, giving

as required. █

LEMMA C.

(i) If

,

(ii) If

,

Proof of Lemma C. For part (i)

For part (ii)

which differs from part (i) only in the sign of the second term. The stated result therefore follows by the same calculations. █

Proof of Theorem 1.

Part (i).

In view of Lemma A

and, because

,

for a > 3 because

by condition L. Also for any positive integer L_n < n we have

and because the elements φ_k(t/n) are bounded uniformly in t we have

Now choose L_n such that

as n → ∞. Then, 1/L_n³ = o(K²/n²) and

For the first term of (18) we note that

uniformly in |h| ≤ L_n. It therefore follows from (17)–(21) that

Next consider

Taking the first term of (23), averaging over k, and using the fact that |h| ≤ L_n and L_n satisfies (19) so that L_n K/n = o(1), we get

For h ≥ 0 we can write the second term of (23) as

for |h| ≤ L_n and L_n satisfying (19). Averaging over k we find

uniformly in 0 ≤ h ≤ L_n. Without going through the calculation, the same rate result holds when −L_n ≤ h < 0.

We deduce from (23)–(26) that

Thus,

because 1/n = o(K²/n) for K satisfying R. Thus

as stated.

Part (ii).

From Lemma A

Using the device in Phillips and Solo (1992) we have the decomposition

where

under L. Then, (28), partial summation, and Lemma B(i) yield

uniformly in k ≤ K. Thus,

From Lemma B(ii),

, and so

by Lemma B(iv). Hence,

and it follows from (29) and (30) that

Thus,

Write

In view of Lemma A, we have

Thus, for K = o(n^4/5) we have from (32)–(34)

We show that the second term in the final expression (35) is negligible. Note that

Hence

and so

Next note from Lemma B(iv) that

Finally, using a martingale central limit argument along the same lines as that in Phillips, Sun, and Jin (2003), we may establish that

giving the stated central limit theorem for

.

Part (iii).

Part (i) shows that the bias of

is given by

whereas arguments as in part (ii) show the variance of

to be

It follows that

and, if K⁵/n⁴ → 1, we get

as stated.