1. INTRODUCTION
Let
be a real stationary process with mean 0 and finite covariances
,
. Assume that the covariances are absolutely summable. Then the spectrum or spectral density function
exists and is continuous and finite. A fundamental problem in time series analysis is to estimate f. Given the observations X1,…, Xn, let the periodogram
Here Sn(λ) is the Fourier transform of the sample X1,…, Xn and
is the imaginary unit. It is well known that In(λ) is an asymptotic unbiased but inconsistent estimate of f (λ) (see, e.g., Anderson, 1971; Brillinger, 1975; Brockwell and Davis, 1991; Hannan, 1970; Priestley, 1981). Denote by λj = λj,n = 2πj/n,
, the Fourier frequencies. Under suitable conditions on the underlying process (Xk), In(λj) are asymptotically independent at different Fourier frequencies. To obtain a consistent estimate of the spectral density at a given frequency θ ∈ [0,π], one can naturally smooth the periodograms over Fourier frequencies near θ. Namely, the following weighted periodogram estimate can be used:
where k0 = [lfloor ]nθ/(2π)[rfloor ], K is a nonnegative kernel function, and
. Here [lfloor ]x[rfloor ] denotes the integer part of the real number x, namely,
. Generically (3) is called the smoothed periodogram spectral density estimate. See Robinson (1983) for a review of various approaches to spectral density estimation. The smoothing parameter m = mn is chosen such that
Under appropriate conditions on (Xk), (4) ensures that
is a consistent estimate of f. As a significant merit,
can be quickly computed via the fast Fourier transforms. On the other hand, however, it has been a notoriously difficult problem to establish a central limit theorem of the estimate under the natural bandwidth condition (4). Such asymptotic results are certainly needed in the related statistical inference in the frequency domain, such as hypothesis testing and the construction of confidence intervals. In previous results the process (Xk) is assumed to have very special structures. For example, Lomnicki and Zaremba (1959) and Hannan (1970) deal with linear processes; Brillinger (1969) assumes that Xk has finite moment of all order and the joint cumulants of the process are summable of any order; also see Dahlhaus (1985). Rosenblatt (1984) considers strong mixing processes. See also Bentkus and Rudzkis (1982) for Gaussian processes and Henry (2001) for linear processes with martingale difference innovations.
An important problem in econometrics is to estimate the long-run variance, which basically corresponds to the spectral density evaluated at zero frequency. A closely related problem in the multivariate case is to study the so-called heteroskedasticity and autocorrelation consistent (HAC) covariance matrix estimate. Asymptotic results have been obtained by Andrews (1991), Hansen (1992), de Jong and Davidson (2000), and Jansson (2002) among others. The HAC estimation plays an important role in various econometrics problems, such as unit root tests, robust hypothesis testing, and cointegration estimation. For recent developments see Phillips, Sun, and Jin (2006a, 2006b).
In this paper we obtain a central limit theorem of
for a very general class of stationary processes under conditions stronger than (4) (see expressions (38) and (39) in Section 3.2). In particular, our results allow linear processes and nonlinear processes including threshold, bilinear, and exponential autoregressive processes among others. The imposed condition on (Xk) is directly related to the data-generating mechanism of the process and hence is easily verifiable. Additionally, we do not need the summability conditions on joint cumulants and/or strong mixing conditions. Conditions of the latter two types do not seem to be tractable in many applications. We shall study asymptotic properties of the quadratic form
where the weights an(t,t′), 1 ≤ t,t′ ≤ n, n = 1,2,…, are (deterministic) real coefficients. The form of Qn is very general. An important special case of (5) is the smoothed periodogram spectral density estimate (3). Elementary calculations show that (3) is of the form (5) with an(t,t′) = an(t − t′), where
It would certainly be impossible to obtain limit theorems for Qn without imposing suitable conditions on an(t,t′) and (Xt). In Section 2 we assume that the process (Xt) is a stationary martingale difference sequence. Using the idea of martingale approximation, we show in Section 3 that, for stationary processes (Xt) that may not necessarily be martingale differences, the smoothed periodogram spectral density estimation (3) can be approximated by the quadratic form (5) of martingale differences under mild conditions.
The asymptotic problem of Qn has a long history. See Whittle (1964), de Wet and Venter (1973), ten Vregelaar (1990), Varberg (1966), Mikosch (1991), Basalykas (1994), and Götze and Tikhomirov (1999), and references cited therein. Results of this sort have many applications in statistics. In all those works Xk are assumed to be independent and identically distributed (i.i.d.), and/or the weights an(t,t′) do not depend on n. Both assumptions are violated in our setting. In particular, in our problem the independence assumption is too restrictive, and we have to resort to other powerful and more versatile methods.
In a series of recent papers (see, e.g., Wu and Mielniczuk, 2002; Wu, 2005a, 2005b; Wu and Woodroofe, 2004; Hsing and Wu, 2004), we argue that the method of martingale approximation is quite useful in a variety of asymptotic problems. In particular, Hsing and Wu (2004) established an asymptotic theory for the weighted U-statistics
for a quite general class of stationary processes including linear processes and many widely used nonlinear time series. However, even though the bivariate function G(u,v) in (7) can be specifically chosen to be the product uv, the result in Hsing and Wu (2004) is not directly applicable to Qn. The major difficulty is that the weights an(t,t′) in (5) depend on n. The dependence of an(t,t′) on n makes the asymptotic problem of Qn considerably more challenging. Nonetheless the method of martingale approximation used in Hsing and Wu (2004) sheds new light on asymptotic properties of Qn. In this paper we shall apply a modified version of the method in Hsing and Wu (2004).
Throughout the paper let ⇒ denote convergence in distribution and N(μ,σ2) the Gaussian distribution with mean μ and variance σ2. The notation C stands for a generic constant that may vary from line to line. For a random variable ξ write
(p > 0) if
and ∥ξ∥ = ∥ξ∥2. Denote the real part of a complex number a by
.
Suitable structural assumptions on the process (Xk) are certainly needed. Here we assume that (Xk) is a stationary causal process of the form
where
, are i.i.d. random variables and g is a measurable function for which Xk is a properly defined random variable. The class of processes that (8) represents is huge; see Rosenblatt (1971), Kallianpur (1981), and Tong (1990, p. 204).
Let the shift process
. For
define the projection operator
The projection operator plays an important role in the study of Qn. Clearly
, are martingale differences, and hence they are orthogonal in
if
.
The rest of the paper is structured as follows. Section 2 presents a central limit theorem for Qn for martingale differences. Applications to the smoothed periodogram spectral density estimate (3) are given in Section 3. Some proofs are collected in the Appendix.
2. QUADRATIC FORMS FOR MARTINGALE DIFFERENCES
In this section we assume that
, are martingale differences, namely,
almost surely. We are interested in the asymptotic distribution of the quadratic form
where, for each n, the weights an(j,j′), 1 ≤ j < j′ ≤ n, are real numbers. A particularly interesting special case is that the bivariate function an(·,·) can be written in the univariate way, namely, an(j,j′) = an(j − j′). For smoothed periodogram spectral density estimates, we present in Section 3 a martingale approximation scheme and show that (3) can be reduced to the form (10) even though the process (Xk) itself may not be a martingale difference sequence. Martingale approximations of this type act as a bridge that connects general stationary processes and martingales (Wu and Woodroofe, 2004).
Let
Theorem 1, which follows, concerns the asymptotic distribution of Tn /σn under easily verifiable and mild conditions on an(j,j′), 1 ≤ j < j′ ≤ n.
THEOREM 1. Let the process Xk of (8) be a martingale difference sequence and
for some q > 4. Assume
(i)
(ii) for any fixed
,
(iii)
(iv)
Then we have the asymptotic normality
We now briefly discuss conditions (12)–(15) of Theorem 1. Condition (12) is basically the Lindeberg-type condition. In the important special case an(t′,t) = an(t′ − t), we have At,n ≤ An,n for 2 ≤ t ≤ n, and then (12) indicates that An,n does not dominate σn2. In this case, (12) is equivalent to the following claim: there exists a δ ∈ (0,1) such that A[lfloor ](1−δ)n[rfloor ],n > δAn,n for all sufficiently large n. Condition (13) means that the contribution of an(j,j′)Xj Xj′ to Tn is negligible if j′ − j is small. Note that
; by the Cauchy–Schwarz inequality,
. So (14) indicates that the magnitudes of
and σn4 are comparable.
Condition (15) is needed to exclude sequences an(·,·) that may lead to non-Gaussian limiting distributions. For example, if an(j,j′) ≡ 1, then At,n = t − 1,
, and (12)–(14) are fulfilled, whereas (15) is violated. In this case, if Xk are i.i.d. standard normal random variables, then Tn /σn has the non-Gaussian asymptotic distribution
, where W is the standard Gaussian random variable. For a more general case for which (15) is violated, we let an(k,j) = K [(k − j)/n], where K is a kernel function with support [−1,1]. This case is closely related to the spectral density estimation without truncation where the bandwidth is equal to the sample size. Kiefer and Vogelsang (2002) considered estimators of such type and obtained a non-Gaussian limiting distribution. See also Phillips et al. (2006a, 2006b) for more discussion. We are unclear as to what conditions on an(·,·) are needed such that Tn /σn has a non-Gaussian limiting distribution.
With trigonometric identities, it is easy to verify (12)–(15) for an(j) with form (6); see Section 3.2.
Proof of Theorem 1. Without loss of generality we assume ∥X0∥ = 1. Note that Xt Zt−1,n,
, are martingale differences with respect to the filter
; we shall apply the martingale central limit theorem (MCLT) (Chow and Teicher, 1988). By Lemma 1, which follows, ∥Zt−1,n∥q2 ≤ Cq2At,n∥X0∥q2. By the Cauchy–Schwarz inequality, the Lindeberg condition is satisfied because
The last step is due to (12) and q > 4. Applying Lemmas 2 and 3, which are given later in this section, with
, it is easily seen that the convergence of conditional variance
in probability. By the MCLT, (16) holds. █
LEMMA 1. Let D1,…, Dn be a martingale difference sequence for which
, p > 1. Let p′ = min(2,p). Then
where Cp = 18p3/2/(p − 1)1/2.
Lemma 1 is a simple consequence of the Minkowski and the Burkholder inequalities (Chow and Teicher, 1988). The details of the proof are omitted.
LEMMA 2. Assume that the process
for some q > 4. Further assume that an(j,j′) satisfies (12) and (13). Then
Proof. Without loss of generality let
. For integer I ≥ 1 introduce the truncated process
. Then Zt−1,n,I = 0 if t ≤ I and
. For 1 ≤ t ≤ n, by Lemma 1,
Let
, where [ell ]n(J) is the quantity on the left-hand side of (13). Let
By the Cauchy–Schwarz inequality, (12), and (13),
as n → ∞. For 0 ≤ j ≤ I − 1 let
Then
. Note that the sequence
, t = 2,…, n, forms martingale differences. By Lemma 1 and (12), because
,
Because |Vn(1)| ≤ |Vn(1) − Vn(I)| + |Vn(I) − Tn(I)| + |Tn(I)|, by (19),
Therefore (18) follows because
as I → ∞. █
LEMMA 3. Assume that (Xk) defined in (8) is a martingale difference sequence with
. Then under (14) and (15), we have
Proof. For notational convenience we omit the subscript n in an2(t′,t), Bj,n, and Zt−1,n. Write
; then
. Recall that
, 1 ≤ t′ ≤ n − 1, and let
. To show (20), we shall apply the martingale decomposition method. By the orthogonality of
, k = …,n − 2,n − 1, and because
if 1 ≤ t ≤ k,
Then it remains to show that both terms in the preceding display are of the order o(σn4). For the first one, let k be a nonpositive integer. Because (Xj) are martingale differences,
which by the triangle inequality implies
Hence by the Cauchy–Schwarz inequality and (14),
It is slightly more complicated to deal with the second term on the right-hand side of (21). Let 1 ≤ k ≤ n − 1; then
if t ≤ k and
if t > k. By the Cauchy–Schwarz inequality, Lemma 1, and (15),
For the term Ik(t), we have
Let [ell ] = [lfloor ]n1/4[rfloor ]. Then
By (24),
Combining (23) and (26), we have (20). █
3. SMOOTHED PERIODOGRAM ESTIMATES
The central limit theorem presented in Section 2 is only for martingale differences. To obtain asymptotic distribution of the smoothed periodogram spectral density estimate (3) for processes with general forms, we shall approximate
by martingales so that Theorem 1 is applicable. Such a martingale approximation scheme has been proposed in Wu (2005a). An explicit construction of approximating martingales is given in Section 3.1. Section 3.2 shows the asymptotic normality of the estimate
in (3).
3.1. A Martingale Approximation Scheme
LEMMA 4. Assume that the process (Xk) defined in (8) satisfies
for some q ≥ 2. Then for every
, the process
exists and is in
. Let
and
Then there exists a constant C > 0, independent of n and λ, such that
Proof. Let
and t ≥ 0. Note that
. By Lemma 1, we have
, which in conjunction with (27) implies that
Hence
, and additionally
. For notational convenience write
,
, and
. Then
Because
, by the triangle inequality,
Note that
Then (30) follows from (31) with
. █
In Wu and Min (2005) the condition (27) implies that the process (Xk) is
weakly dependent with order 1. It also implies
; thus the spectral density function is continuously differentiable; see Lemma A.1 in the companion paper in this issue (Shao and Wu, 2007b). Note that
is a martingale difference sequence and it is related to the spectral density in an interesting way. By Wu (2005a), ∥Dk(θ)∥2 = 2πf (θ). The latter identity gives a probabilistic representation of the spectral density.
3.2. Asymptotic Normality
Recall that λj = 2πj/n,
, are Fourier frequencies. Let n > 4 and m be a positive integer with m < n/2 − 1. For a real sequence s = (sj)j=−mm, let
,
,
, and
. Crudely speaking the quantity ωm measures the oscillation of the sequence s. For θ ∈ [0,π] let
Let
be an i.i.d. copy of
. For k ≥ 0 define
Then Xk′ is a coupled version of Xk with ε0 replaced by ε0′. Our weak dependence condition (cf. expression (34) in Theorem 2, which follows) is expressed in terms of the distance between Xk and Xk′.
THEOREM 2. Assume that, for some q > 4,
Further assume that m = o(n2/3) and the sequence (sj)j=−mm satisfies
(i) If θ = 0 and sj = 0 for j = −m,…,−1, then as n → ∞,
where
satisfies ∥D0(θ)∥2 = 2πf (θ).
(ii) If θ ∈ (0,π), then (36) holds with Δn(0) and D0(0) replaced by Δn(θ) and D0(θ), respectively.
Remark 1. In case (i), sj are assumed to be 0 for −m ≤ j ≤ −1. If otherwise, noting that In(λ) = In(−λ), we can let sj′ = sj + s−j, s−j′ = 0 for j = 1,…, m and s0′ = s0. Then the central limit theorem (36) is still applicable.
As in Wiener (1958) and Priestley (1988), the causal process (8) can be interpreted as a physical system with …,εk−1,εk being the inputs, Xk the output, and g a transform or a filter. If g is a linear function, then (Xk) is a linear process. Otherwise (Xk) is a nonlinear process. The condition (34) has the following interesting interpretation. Note that Xk′ is a coupled version of Xk by replacing ε0 in Xk by an i.i.d. copy ε0′. If the function g(…,εk−1,εk) does not depend on ε0, then Xk − Xk′ = 0. Hence the quantity ∥Xk − Xk′∥q measures the contribution of ε0 to Xk, in other words, the degree of dependence of Xk on ε0. In this sense (34) means that the weighted cumulative contribution of ε0 to all future values Xk, k > 0, is finite, and hence (34) ensures short-range dependence. See Wu (2005b) for a more detailed discussion on the dependence of stationary causal processes from the nonlinear system theory point of view. Conditions based on the quantity ∥Xk − Xk′∥q are often easily verifiable because they are directly related to the data-generating mechanism of the process (Xk).
Our dependence condition (34) is very different from the classical strong mixing conditions, which may be too restrictive in certain applications (Andrews, 1984). On the other hand, we avoid summability conditions on joint cumulants that are commonly imposed in the large-sample spectral theory (Brillinger, 1975; Rosenblatt, 1984). The verification of the latter generally is not easy for processes that are non-Gaussian and nonlinear. In the companion paper in this issue (Shao and Wu, 2007b), we find an easily verifiable sufficient condition for the absolute summability of qth (
, q ≥ 2) joint cumulants, which is
; see Theorem 4.1 and Remark 4.1 therein. Unfortunately the latter sufficient condition appears to be overly strong. It is an open problem whether weaker versions such as
suffice.
Clearly (34) implies (27) by the identity
and the Jensen inequality. Theorem 2 has the following two immediate corollaries. The proofs are straightforward and hence the details are omitted. The first corollary gives a central limit theorem for
, and the second one has an application in Shao and Wu (2007b). The latter paper studies the local Whittle estimation of long memory index for fractionally integrated nonlinear processes.
COROLLARY 1. Assume that (34) holds for some q > 4. Further assume that the kernel K is a nonnegative function with support [−1,1], K has bounded variation, and
as m → ∞. Additionally assume that the m = mn satisfies
Then
.
Using the Cramér–Wold device, we can show that, for different frequencies ω1,…, ωJ ∈ (0,π),
, 1 ≤ j ≤ J, are asymptotically i.i.d. normal N(0,κ). Because the proof of the latter claim is routine and it involves very lengthy and tedious calculations, we omit the details.
We conjecture that the central limit theorem in Corollary 1 still holds if (38) is weakened to m−1 log n + n−1m → 0. A key difficulty toward this result is that our argument requires m = o(n2/3) to ensure a satisfactory martingale approximation of Δn(θ) by Λn(θ); see the proof of Theorem 2 in the Appendix. If Corollary 1 is valid under m−1 log n + n−1m → 0, then we can allow m to be a multiple of n4/5 that minimizes the mean squares error
because the bias
is of the order (m/n)2 under suitable conditions on K and f.
COROLLARY 2. Let sj = 1 + log(j/m) for 1 ≤ j ≤ m and sj = 0 for −m ≤ j ≤ 0. Assume (34) for some q > 4 and
Then (36) holds.
Example 1 (Nonlinear time series)
Let
, be i.i.d. random variables and define Xn recursively by
where R is a measurable function. Many popular nonlinear time series models, such as threshold autoregressive (TAR) models, bilinear autoregressive models, and autoregressive models with conditional heteroskedascity (ARCH) are of the form (40). Let
Assume that
for some α > 0 and x0. Then (40) has a stationary distribution (Diaconis and Freedman, 1999; Wu and Shao, 2004), and iterates of (40) give rise to (8). Additionally, Wu and Woodroofe (2000) show that (41) implies that there exist β > 0 and r ∈ (0,1) for which
The preceding property is called geometric-moment contraction (GMC) in Hsing and Wu (2004). Furthermore, if
for some q > 4, then (42) holds for all β ∈ (0,q) (cf. Wu and Min, 2005, Lem. 2). It is easily seen that (42) implies ∥Xn − Xn′∥β = O(rn). Hence (34) holds. Recently, Shao and Wu (2007a) showed that GMC holds for various generalized autoregressive conditional heteroskedasticity (GARCH) models, including general asymmetric GARCH(r,s) and nonlinear GARCH(1,1) models. For the GMC property of an exponential GARCH (EGARCH) model, see Min (2004).
Example 2 (Nonlinear transforms of linear processes)
Let εk be i.i.d. random variables with
for some q > 4; let a0,a1,… be a square summable real sequence and
be a linear process. Consider the process
Let
be an i.i.d. copy of
. Then | |Ut| − |Ut′| | ≤ |at| |ε0 − ε0′|, and consequently ∥Xt − Xt′∥q ≤ ∥|Ut| − |Ut′|∥q ≤ |at|∥ε0 − ε0′∥q. Under the simple sufficient condition
because ∥ε0∥q < ∞, (34) holds. It is easily seen that, for any Lipschitz continuous function G, (43) implies (34) for
. The classical central limit theorems on spectral density estimates are not applicable here because they require strong mixing conditions and summability conditions on joint cumulants. As pointed out in Andrews (1984), the process Xt is not strong mixing if εk are i.i.d. with the distribution
and aj = 2−j,
. On the other hand, if G is a nonlinear function, it seems very difficult to verify summability conditions on joint cumulants of Xt, because of the nonlinear nature. The central limit theorem in Hannan (1970, Thm. 5.11) is only for linear processes and hence is not applicable to our Xt. The argument in Hannan (1970) does not work either because it depends heavily on the linearity structure.
APPENDIX
Proof of Theorem 2. Recall the definitions of χm(s), τm(s), ϖm(s), and ωm(s) in Section 3.2. For simplicity we abbreviate them as χ, τ,ϖ, and ω, respectively. By Lemma 4, condition (34) implies that the martingale Mn(θ,λ) defined in (29) exists in
. Recall that
. Let
Let Rn(θ,λ) = Sn(θ + λ) − Mn(θ,λ). By (30) of Lemma 4, for −m ≤ j ≤ m,
. Because m = o(n2/3) and
,
It then remains to show (36) with Δ(θ) replaced by Λn(θ).
(i) Let θ = 0. Then (A.1) becomes
where
Note that
. The second term τΩn in Λn(0) is of order
because
as a consequence of (34). We now shall apply Theorem 1 to Tn by verifying conditions (12)–(15). By Lemma 5, which is given later in this Appendix,
and
So (12) follows because τ2 = O(mχ) and m = o(n2/3). Note that for any l, an2(l) = O(mχ) = o(n2/3χ), (13) easily follows. Because
, we have (14). It is slightly more complicated to verify (15). To this end, for δ ∈ (0,1/4) let
and
. By the Cauchy–Schwarz inequality, |cn(k,t)| ≤ An+1,n. For l ≥ 1, let
, v ≥ −m, and h−m−1 = 0. Then
. Using the Abelian summation technique,
Hence |an(l)| = O(ω) uniformly over δn ≤ l ≤ (1 − δ)n. By (35),
as n → ∞. Therefore,
which completes the proof of (15) because δ > 0 can be made arbitrarily small. Here #(Γ0 − Γδ) denotes the number of elements in the set Γ0 − Γδ.
(ii) Outline of the proof in the case θ ∈ (0,π). The proof for this case can be done in an analogous way as in the case θ = 0, and it does not involve additional methodological difficulties. However, it does involve quite lengthy manipulations. Here we only provide an outline of the arguments. Let
, where both (Ak) and (Bk) are real, stationary martingale difference sequences with
. Write
where an(l) and bn(l) are real numbers. The quantity Λn(θ) in (A.1) can be rewritten as
where
,
and
Similar to (A.2), we have
, and consequently
. Because Z1,n(k′ − 1)Ak′ + Z2,n(k′ − 1)Bk′, 2 ≤ k′ ≤ n, form martingale differences, we can again apply the martingale central limit theorem. Following the proof of Theorem 1, it is easily seen that the Lindeberg condition holds. It remains to verify the convergence of the sum of conditional variance
By (ii) of Lemma 5,
Therefore, because
and ∥D0(θ)∥2 = ∥A0∥2 + ∥B0∥2, we have
An analogue of Lemma 2 indicates that (A.3) follows from
The preceding convergence is a consequence of (A.4) and a similar version of Lemma 3. █
LEMMA 5. Let m be a positive integer with m < n/2 − 1; let sj,−m ≤ j ≤ m, be real numbers,
, and
.
(i) Assume that sj = 0 for −m ≤ j ≤ −1. For
let
Then
and
(ii) For a fixed θ ∈ (0,π) and
let
Assume that m = o(n). Then
and
Proof. (i) For −m ≤ j,j′ ≤ m let
Because λj = jλ1, basic trigonometric manipulations imply that μn(j,j′) = n if j = j′ = 0; μn(j,j′) = 0 if j ≠ j′; μn(j,j′) = n/2 if j = j′ ≠ 0; νn(j,j′) = n(n + 1)/2 if j = j′ = 0; νn(j,j′) = n/2 if j ≠ j′; and νn(j,j′) = n(n + 2)/4 if j = j′ ≠ 0. Using these trigonometric identities, (A.6) and (A.7) easily follow.
(ii) We only prove (A.9) because (A.10) can be proved in a similar way. As in (i), let
Let β ∈ (0,2π). Then
If j = j′, because for sufficiently large n, θ/2 < λj + θ < θ + (π − θ)/2 uniformly over j = −m,…, m, (A.11) implies that μn(j,j;θ) = n/2 + O(1). On the other hand, if j ≠ j′, we similarly have μn(j,j′;θ) = O(1) uniformly over j and j′. Therefore,
because ϖm2(s) ≤ (2m + 1)χm(s) = o[nχm(s)]. █
