2. COVARIANCE MATRIX ESTIMATION FOR I(0) SERIES

Consider a p × 1 vector-valued sequence x_t, t = 0,±1,…. For the purpose of a concise discussion we take the elements of x_t to be jointly covariance stationary, later mentioning possible departures. We assume x_t has zero mean and absolutely continuous spectral distribution matrix. Defining the autocovariance matrices

the spectral density matrix f (λ), λ ∈ (−π,π], is given by

and is Hermitian nonnegative definite.

For n ≥ 1 define the arithmetic mean

The covariance matrix of x is

If f (λ) is continuous at λ = 0, Fejér's theorem indicates that

Under a variety of additional conditions, n^1/2x satisfies a central limit theorem, so that, if f (0) is also positive definite,

Note that

Large sample inference based on x thus requires consistent estimates of f (0). These could result from an assumed parametric model for γ(j), j = 0,±1,…, or equally for f (λ), λ ∈ (−π,π], an obvious example being a stationary and invertible autoregressive moving average process of prescribed orders. However, if either of the orders is underspecified, or both are overspecified, f (0) will be inconsistently estimated.

As the Weierstrass approximation theorem hints, this theoretical drawback can be overcome if the autoregressive or moving average orders are regarded as increasing, slowly, with sample size n. In particular Berk (1974) justified the consistency of autoregression-based spectral density estimates. The autoregressive order can here be thought of as a smoothing number. Such estimates have been employed and modified in the HAC econometric literature, but this has been more influenced by spectral density estimates developed still earlier in the statistical literature, entailing an alternative form of smoothing and based on quadratic functions of the data, in particular weighted autocovariance spectral estimates (see, e.g., Grenander and Rosenblatt, 1957; Parzen, 1957). We shall discuss instead a closely related class of quadratic estimate, stressed by Brillinger (1975), that is not much used by econometricians in the HAC context but yields more conveniently to the necessary modifications required in the following section than weighted autocovariance forms, which have already been extensively discussed in the econometric literature.

Define the periodogram matrix

For an integer m ∈ [1,n/2], introduce a sequence of nonnegative weights w_jm, j = 0,…,m, such that

. Define

for λ_j = 2πj/n. The simplest version of (2.5) takes equal weights, w_jm = 1/(2m + 1). Under suitable conditions on {x_t}, on the w_jm, and on m (such that m increases with n but more slowly), we have

Various rules have been suggested for choosing the bandwidth m, possibly to satisfy some optimality criterion, such as cross-validation, and also rules of thumb. Optimality theory for choice of the w_jm is also available. The estimates considered by Brillinger (1975) are more general than (2.5), allowing weighted summation over all Fourier frequencies λ_j. However, the weights must again concentrate around zero to an extent that increases slowly with sample size, and the form (2.5) fits in conveniently with that of narrow-band estimates of memory parameters, which have predominated in the semiparametric memory estimation literature relevant to the following section. For each choice of weights {w_jm} one can effectively find a choice of lag weights, approximately related to the w_jm by Fourier transformation, that can be employed in a corresponding weighted autocovariance spectral estimate of f (0), which typically has very similar asymptotic properties to those of its weighted periodogram twin (2.5). Note that the stated conditions on the w_jm guarantee that

is nonnegative definite. It is possible to refine (2.5) by employing different bandwidths and weights across the elements, though the nonnegative definite property is less easy to enforce. Refinements such as prewhitening and tapering are also available to reduce bias in

due to “leakage” from remote frequencies.

The description HAC appears to stress “heteroskedasticity” at least as much as “autocorrelation,” but whereas there is explicit allowance for the latter in

and rival estimates, there is none for the former, and the robustness to heteroskedasticity essentially just appeals to long-standing limit theorems for nonidentically distributed variates. For example, in the special case of serially uncorrelated x_t, such that Ex_t x_t′ = Ω_t, suppose

as n → ∞. Then from (2.5),

. Even the usual covariance matrix estimate motivated by uncorrelated, homoskedastic variates,

, satisfies

and so can also be called heteroskedasticity-consistent. The econometric HAC literature has stressed mixing conditions, and extensions thereof, that are designed mainly to describe dependence but also allow a degree of heterogeneity. It would be possible to allow for such heterogeneity in the discussion of the following section, but because again no explicit correction for heteroskedasticity is involved we prefer the simplicity of presentation gained by maintaining the covariance stationarity assumption.

3. COVARIANCE MATRIX ESTIMATION FOR NON-I(0) SERIES

Of crucial importance in the preceding discussion was the I(0) assumption, that f (λ) be continuous and positive definite at λ = 0. To relax this requirement, suppose that

where G is a finite, positive definite matrix with (a,b)th element g_ab

for d_j ∈ (−½,½), j = 1,…,p, the overbar means complex conjugation, and ∼ means that the ratio of real parts, and of imaginary parts, of corresponding elements of the matrices on the left- and right-hand sides of (3.1) tends to 1. If d₁ = ··· = d_p = 0, (3.1) holds with G = f (0) under the I(0) assumption. Slightly more generally, if d_a + d_b = 0, for some a,b, the (a,b)th element of f (λ), f_ab(λ), satisfies f_ab(0) = g_ab cos(π/2)(d_a − d_b), which can again be consistent with f_ab(λ) being continuous at λ = 0. For d_a + d_b > 0, on the other hand, f_ab(λ) diverges as λ → 0+, whereas for d_a + d_b < 0, f_ab(0) = 0. Of course when a ≠ b, f_ab(0) = 0 also occurs, for any d_a,d_b, if g_ab = 0. The d_j are called “memory parameters.”

To motivate (3.1) and (3.2), Theorem III-1 of Yong (1974) gives

The left-hand side is the frequency response function of the fractional difference operator. An important special case of f (λ) satisfying (3.1) is the spectral density matrix of a stationary, noncointegrated and invertible fractionally integrated autoregressive moving average system, with possibly distinct memory parameters d₁,…,d_p.

Although, in a nonparametric setting, we do not want to impose such a parametric model, nevertheless we need to supplement (3.1), when at least one d_j is nonzero, by an assumption that is easily satisfied in that parametric model. We have to approximate the right side of (2.1) for large n, and this can be achieved by approximating γ(j) + γ(−j) for large j. For some a,b such that d_a + d_b ≠ 0, denote by γ_ab(j) the (a,b)th element of γ(j). For λ close to zero, f_ab(λ) has real part

On the other hand, from (3.1) it follows that

An important topic in the trigonometric series literature concerns the asymptotic behavior of Fourier coefficients that provides the power law behavior found in (3.4), a detailed reference being Yong (1974). Consider a function

for λ close to zero. Yong (1974, Theorems III-1, III-10, III-12, and III-17) gives conditions on the s(j) such that, for some β ≠ 0,

is equivalent to

when α ∈ (0,1). Yong (1974, Theorem III-27) shows that if

(3.5) implies (3.6) for α ∈ (1,3).

We apply these properties with s(j) = γ_ab(j) + γ_ab(−j), r(λ) = 2πRe{f_ab(λ)}, and α = 1 − d_a − d_b. We deduce from (3.4) and (3.6), reflection formula for the gamma function, and trigonometric identities that

to give three alternative expressions. Note that if d_a + d_b < 0, (3.4) implies f_ab(0) = 0, so (3.7) is indeed relevant. On the other hand, for d_a + d_b > 0, the conditions of Yong (1974) can be checked in the case of plausible autocovariance sequences.

We can now deduce that

as n → ∞, where

in which we arbitrarily employ the first and the three equivalent expressions for β from before. This follows by approximating sums by integrals, though in the case d_a + d_b < 0 (implying

one first writes the left side of (3.8) as

We deduce (cf. (2.2))

where

Q(d₁,…,d_p) is the p × p matrix with (a,b)th element q(d_a,d_b), and ○ denotes Hadamard product. Because the right side of (3.10) is the limit of a sequence of nonnegative definite matrices, it also is nonnegative definite. If it is positive definite we can deduce under suitable additional conditions (e.g., if x_t is a linear process in stationary, conditionally homoskedastic martingale differences)

(cf. (2.3)).

The preceding discussion strictly covers only elements such that d_a + d_b ≠ 0, but (3.8) applies when d_a + d_b = 0 for some a,b, because from the second equality of (3.9) it agrees with f_ab(0) = g_ab cos(π/2)(d_a − d_b) (which follows from (3.4)). This includes the case d_a = d_b = 0, and in the full I(0) case d₁ = ··· = d_p = 0, (3.10) reduces to (2.2). Thus (3.10) generalizes (2.2); indeed q(u,v) is continuous at u + v = 0 (and all u,v ∈ (−½,½)).

Given estimates

such that

we can replace (3.11) by the useful result

where

is the p × p identity matrix. The rate requirement in (3.12) is due to the need to approximate the norming factors n^(1/2)−d_j by the

.

An acronym has become almost obligatory. The best that emerged to describe our robustified variance estimate of x,

was MAC: memory autocorrelation consistent. Partly for reasons given at the end of the previous section, and partly for the sake of an acronym that slips easily off the tongue, reference to “H” for “heteroskedasticity” is suppressed.

Robinson (1998) proposed HAC estimates for parameter estimates in regression models in which regressors and disturbances can satisfy a condition like (3.1), and thus need not be I(0), though the product of regressors and disturbances is I(0). His HAC estimates are nonparametric but do not require a user-chosen bandwidth.

Estimates of the

satisfying (3.12) under suitable conditions are readily available, such as log periodogram, local Whittle, and averaged periodogram estimates, all of which are “semiparametric” in character, being based principally on the local–to-zero model (3.1). Like (2.5), they involve functions of the periodogram I(λ_j) at low frequencies such that j = 1 − m,…,m − 1, with m satisfying rather similar conditions to those that would be required for (2.5) in Section 2. As a result, the

estimates converge more slowly than the n^1/2 parametric rate, but nevertheless the slow rate in (3.12) is easily justified.

With respect to

, in view of (2.5), (3.1), and (3.2), it is natural to consider

where

Under conditions familiar from the semiparametric memory parameter estimation literature, (3.14) will satisfy (3.12). To make the procedure more fully automatic, rules for choice of m in (3.14), and of bandwidths in the

estimates, are required. These issues have been discussed in the literature (see, e.g., Hurvich and Beltrao, 1994; Robinson, 1994a; Hurvich, Deo, and Brodsky, 1998; Henry and Robinson, 1996; Hurvich and Deo, 1999).

The MAC estimate (3.13), with

given by (3.14), is guaranteed nonnegative definite. To see this, note first that (3.14) is nonnegative definite. Thus it follows from Schur (1911, p. 14) that it suffices to show that Q(d₁,…,d_p) is nonnegative definite for all d_j ∈ (−½,½), j = 1,…,p. But from the previous development it is clear that

where

, surely that f*(λ) satisfies (3.1) with g_ab = 1, all a,b (so G is now taken to be nonnegative definite, with rank 1), and γ*(j) has the same asymptotic behavior as γ(j) with g_ab = 1, all a,b. But the term in square brackets in (3.15) is

, which is clearly nonnegative definite, for all n, because f*(λ) can be chosen nonnegative definite for all λ.

Even in the expectation that all d_j are zero, MAC estimates might be useful rivals to long autoregressive and weighted autocovariance (or periodogram) HAC estimates, these latter having the reputation of being appropriate in the presence of, respectively, (finite) peaks and (nonzero) troughs in f (λ) at λ = 0.

The present topic was also discussed in the case of scalar x_t by Beran (1989) and Robinson (1994b). The first author, however, employed parametric memory parameter estimates, whereas the second author employed semiparametric averaged periodogram memory parameter estimates and noted the need for the rate condition in the first part of (3.12). The estimate (3.14) extends one of Robinson (1995) in the scalar case, in which w_jm ≡ 1/(2m + 1) and was employed in a different context by Robinson and Yajima (2002). Robinson (1994b) considered covariance matrix estimates in the case x_t′ = (1,t,…,t^p−1)e_t, where e_t is a scalar long memory process, this being relevant to inference on least squares estimates in polynomial-in-time linear regression with long memory or antipersistent disturbances. For the same kind of disturbances, Robinson (1997) considered covariance matrix estimates for a vector of scalar fixed-design nonparametric regression estimates at finitely many fixed points. The rates of convergence in these situations, and the forms of the limiting covariance matrices of the normalized statistics, differ from those found in the present paper, which is motivated by other situations in econometrics. In a fractional cointegration context, in which two distinct memory parameters are involved, Kim and Phillips (1999) proposed estimates of the long-run covariance matrix in which stationary fractional sequences are filtered in the time domain and the resulting I(0) long-run covariance matrix estimate rescaled.

Many econometric statistics are functionals of partial sums of vector variates, which themselves can be products of other nonlinear functions of underlying variates, for example, generalized-method-of-moments estimates, including least squares estimates for linear regression models with stochastic regressors. Consider an estimate

of a vector-valued parameter θ of dimension no greater than p. Typically we can consider a linearization

, where, when

is only implicitly defined, this requires an initial consistency proof (which should itself allow for possible long memory or antipersistence) and T_n is a matrix-valued statistic. If, for some matrix E_n, E_nT_n D_n converges in probability to a finite limit U of full row rank, we would deduce from (3.11) that

, whence the previous discussion is relevant. However, in case the d_j are not all identical, lack of commutativity can obstruct this argument; the analysis could be preceded by a test of equality of the d_j, employing known limit distribution theory for semiparametric memory estimates. In this kind of setting, moreover, when a typical element of x_t is a nonlinear function of underlying variates such as a product of an explanatory variable and a disturbance, it is important to bear the following in mind. If some or all of the underlying variates have long memory, it is still possible at one extreme that x_t can be I(0) and at the other that x has a nonnormal limit distribution (see Robinson, 1994c). Note also that disturbances will have to be replaced by residuals to produce proxies for x_t that can be used in the estimation of the d_j.