No CrossRef data available.
EXPANSIONS FOR THE DISTRIBUTION OF THE MAXIMUM LIKELIHOOD ESTIMATOR OF THE FRACTIONAL DIFFERENCE PARAMETER
Published online by Cambridge University Press: 08 June 2004
Abstract
The maximum likelihood estimator (MLE) of the fractional difference parameter in the Gaussian ARFIMA(0,d,0) model is well known to be asymptotically N(0,6/π2). This paper develops asymptotic expansions to the distribution of this statistic under the assumption of a known unit variance. The correction term for the density is shown to be independent of d, so that the MLE is second-order pivotal for d. This feature of the MLE is unusual, at least in time series contexts. Simulations show that the normal approximation is poor and that the expansions can make a significant improvement in accuracy provided the correction terms are computed without further asymptotic approximation.This paper was commenced and revised while Lieberman was visiting the Cowles Foundation during 2000–2002. Lieberman thanks the Cowles Foundation for support and hospitality during this visit. Phillips thanks the NSF for support under grants SBR 97-30295 and SES 0092509.
Information
- Type
- Research Article
- Information
- Copyright
- © 2004 Cambridge University Press
1. INTRODUCTION
The simplest long-memory model is the ARFIMA(0,d,0) model. In this model the short-memory component has a flat spectrum and the long-memory component depends on the fractional difference parameter d. If 0 < d < ½ the process is stationary long-memory with hyperbolically decaying autocorrelations, for d > ½ the process is nonstationary long-memory, for −½ < d < 0 the process is antipersistent with autocovariances that sum to zero (producing a zero spectrum at the origin), and for d = 0 the process is independent and identically distributed (i.i.d.). Given the widely differing characteristics of the process for different d-values it is hardly surprising that its estimation has attracted such a great deal of interest over recent years. The literature is now vast and covers many different approaches allowing for parametric structures such as autoregressive fractionally integrated moving average (ARFIMA) systems and semiparametric structures where the short-memory component is specified in terms of the behavior of its spectrum in a neighborhood of zero.
The present paper focuses on the maximum likelihood estimator (MLE) of d in a simple ARFIMA(0,d,0) model with Gaussian innovations and known unit variance. Let
be the MLE of d and d0 be the true value of d and set
. It is well known that
is asymptotically distributed as N(0,6/π2). It is apparent that, unlike the AR(1) model,
is asymptotically pivotal in the sense that its asymptotic distribution is independent of unknown parameters. We are motivated to explore higher order theory for
for two reasons. The first is to see whether the pivotal character of
extends to higher orders. The second is to assess the adequacy of the asymptotic distribution in small samples and see how well higher order asymptotic terms correct the discrepancy.
The paper derives Edgeworth expansions for the distribution of
. Although our model is simple and involves a known variance assumption, it is the leading canonical case, and an explicit form of the Edgeworth approximation is derived. In this sense it continues in the tradition of Phillips (1977), who has developed explicit forms of the Edgeworth expansions of the MLE of the autoregressive coefficient and t-ratio in the canonical first-order autoregression. Taniguchi (1984) derives similar expansions for estimators in stationary autoregressive moving average (ARMA) models. Using results of Fox and Taqqu (1987), Dahlhaus (1989) and Lieberman, Rousseau, and Zucker (2003), we extend Taniguchi's work to the long memory ARFIMA(0,d,0) case with unit variance. We show that to order o(n−1/2) the distribution expansion is independent of d, so that
is second-order pivotal. This feature of the MLE seems to be rare in time series contexts (although note the independent work of Giraitis and Robinson, 2003, on the local Whittle estimate described subsequently), and so the present case is quite unusual. Simulations indicate that the normal approximation is poor but that the expansions can provide significant improvements in accuracy. The distribution expansions are valid uniform asymptotic series. Andrews and Lieberman (2002a) show that an implication of the second-order pivot result is that the parametric bootstrap yields high-order improvements beyond those that are already proved for nonpivot models.
We now outline the method of derivation and the connection of this paper to the existing literature. Lieberman et al. (2003) prove the validity of the Edgeworth expansion to the distribution of the MLE of a parameter vector of a zero-mean, stationary long-memory process. Their results cover models more general than ARFIMA under mild assumptions on the behavior of spectral density derivatives in the neighborhood of the origin. All these assumptions hold for the Gaussian ARFIMA(0,d,0) model considered here. The error rate of the MLE distribution expansion is valid uniformly in any compact set of the parameter set and in the Borel σ-field. In connection with the parametric bootstrap for Gaussian long-memory processes, Andrews and Lieberman (2002a) extend the result of Lieberman et al. (2003) to the unknown mean case, and in Andrews and Lieberman (2002b) validity is established for the Edgeworth expansion for the distribution of the Whittle MLE. In the aforementioned references, validity results for the MLE and the Whittle MLE expansions are established, but no explicit expansions are extracted. To our knowledge, the expansions presented in this paper are the first in the literature in the context of parametric long-memory processes. In the semiparametric framework, Giraitis and Robinson (2003) derive an Edgeworth expansion for the local Whittle estimator of the long-memory parameter. The error rate in their semiparametric expansion is slower than the parametric rate because it depends on the number of frequencies (m) used in estimation and the order of magnitude of the bias effect in local Whittle estimation. In the case where m → ∞ so slowly that the bias effect is smaller than m−1/2, Giraitis and Robinson find the Whittle estimator independent of d to order m−1/2.
The method of derivation is standard and proceeds as follows. First, the score is expanded in a Taylor series about the true parameter value. The series is then inverted to obtain an expression for the normalized MLE in terms of the log-likelihood derivatives (LLDs). Second, Theorem 3 of Lieberman et al. (2003) implies that the formal Edgeworth expansion for the joint density of the LLDs is a valid asymptotic series to any order. We obtain a formal Edgeworth expansion for the joint density of the first- and second-order LLDs, which in view of the result of Lieberman et al. has an error rate of o(n−1/2) uniformly in any compact set of the parameter set and in the Borel σ-field. Third, we transform from the LLDs to the MLE and integrate out one of the LLDs to obtain a formal expansion to the density of the MLE. Further integration yields an Edgeworth expansion to the distribution function. By Theorem 4 of Lieberman et al. (2003), the expansion to the distribution function is a valid uniform asymptotic expansion. Finally, the O(n−1/2) correction term in the Edgeworth expansion to the distribution function involves traces of products of Toeplitz matrices for which certain integral limits exist. Therefore, to o(n−1/2), the distribution expansion with the integral limits replacing the exact correction terms is still a valid uniform asymptotic expansion. It turns out that the expansion with the integral limits is independent of d.
The results of this paper can be extended in at least three areas: higher order ARFIMA models, models with unknown variance, and the d < 0 case. The d = 0 case corresponds to short memory, which was covered in the work of Taniguchi (1984). As for the d < 0 case, the first-order asymptotic normality result of Dahlhaus (1989) strictly precludes this case. Our developments rely on the results of Dahlhaus (1989), and unless new tools are found to handle the first-order d < 0 case first, the extension of our high-order results is not going to be trivial. On the other hand, it seems to be possible to extend our work to the multiparameter case, including higher order ARFIMA models and models with unknown variance, but the algebraic task becomes unmanageable. In addition, although the nice second-order pivot feature of
is expected to be retained when σ2 is estimated, it is not clear what the properties are of this estimator when the ARMA parameters need estimation.
The plan for the rest of the paper is as follows. Section 2 gives preliminaries, Section 3 develops Edgeworth expansions for the LLDs, and the main results are contained in Section 4. Section 5 reports some simulations evaluating the accuracy of the normal approximation and the expansions. Proofs are contained in the Appendix.
2. PRELIMINARIES
Denote by D the interval (0,½). Formally, the assumption on the model under investigation is as follows.
A1. {Xt,t ∈ Z} is a zero-mean, Gaussian ARFIMA(0,d,0) process, with spectral density fd(λ) = |1 − eiλ|−2d/2π and d ∈ D.
The log likelihood of the model is given by
where xn = (X1,…,Xn)′ is the sample, and Σn(fd) is the covariance matrix. This model satisfies assumptions A0–A9 of Dahlhaus (1989), and we have the following result from Theorem 2.1 of Dahlhaus (1989).
LEMMA 1. Under Assumption A1,
tends in distribution to a normal variable with zero mean and variance 6/π2.
The convergence rate is
, and the model satisfies assumptions I–VIII of Lieberman et al. (2003). Therefore, a refinement of the result of Dahlhaus (1989) is achievable by means of Edgeworth expansions. To fix ideas and notation in what follows, let Sn be a random p-vector with cumulants of order O(n), mean E(Sn), and positive definite covariance matrix Kn. Denote by φp(x; Kn) the multivariate normal density with zero mean and covariance matrix Kn. Then, the (s − 2)-order formal Edgeworth expansion for the density of
is given by
where Pnj(x) is a generalized Edgeworth polynomial of order j as defined, for example, by Durbin (1980, Eq. (31)). The putative error rate of the expansion is o(n−(1/2)s+1). The construction of the Pnj's is described by Chambers (1967).
The following result is an immediate consequence of Theorem 4 of Lieberman et al. (2003).
LEMMA 2. Let
be the (s − 2)-order formal Edgeworth expansion for the density of
, and let D* be any compact subset of D. Then, under Assumption A1,
a. There exist a sequence of statistics
and a constant c0 = c0(D*) such that
b. Any estimator
satisfying the statement in (a) admits the Edgeworth expansion
uniformly over D* and every class
of Borel sets satisfying the condition
where (∂C)ε denotes the ε-neighborhood of the boundary of C and φ(u;6/π2) is the univariate normal density with mean zero and variance 6/π2.
Lemma 2 states that the formal Edgeworth expansion is a valid asymptotic expansion. The purpose of this paper is to find the explicit form of the expansion when s = 3. The class
is rich because it includes all convex subsets of
(see Bhattacharya and Rao, 1976, Corollary 3.2).
3. EXPANSIONS FOR LLDs
Let Ln,i(d) = ∂iLn(d)/∂di,i = 1,2,3,4,
and Zn = (Zn,1,Zn,2)′. The covariance matrix of Zn is denoted by
LEMMA 3.
where ζ(·) is the Riemann-zeta function.
Remark. The matrix κ is independent of d. Because ζ(3) ≃ 1.202 and ζ(4) ≃ 1.082, κ is also positive definite.
The MLE is a solution of
. A Taylor series about d0 yields
with
. Lemma 4, which follows, shows that
has the same Edgeworth expansion to o(n−1/2) as a simpler statistic based on a rearrangement of (5).
LEMMA 4. Let
and let
Then
where εn satisfies Pd0(|εn| > c(log n)1/2) = o(n−1/2) for some c > 0. Moreover, the Edgeworth expansion to the distribution of
to o(n−1/2) is equivalent to the Edgeworth expansion to the distribution of
to the same order.
To derive the Edgeworth expansion to the distribution of
, we first need a valid Edgeworth expansion to the joint density of Zn. Denote by gZn(x;d) the true density of Zn and by
its formal Edgeworth expansion corresponding to s = 3. Following Barndorff-Nielsen and Cox (1989, pp. 173–176), the expansion is
where
x = (x1,x2)′ is the argument of the density, κn,i,j,k(d) is the third-order cross cumulant of the components of Zn, the third-order covariant Hermite tensor is given by
and ∂i = ∂/∂xi,i = 1,2.
The following lemma follows from Theorem 3 of Lieberman et al. (2003).
LEMMA 5. Let D* be any compact subset of D. Then, under Assumption A1,
We proceed from here taking the following steps. (i) We transform from Zn to Wn = (Wn,1,Wn,2)′ with
. (ii) We integrate out Wn,2 to obtain the marginal density expansion of Wn,1. (iii) We transform from
. (iv) We integrate the density expansion of
to obtain a cumulative distribution function (c.d.f.) expansion. The result in the last step is a valid Edgeworth expansion to the distribution of
to order o(n−1/2).
Some light is shed on the complexity of the problem by considering the terms involving hijk(x;κn(d)). It is clear that the correction term in the density expansion (8) includes eight covariant Hermite tensors of degree 3. In analogy with (6), an attempt to extract the Edgeworth expansion to the distribution of the MLE of a two-parameter vector, such as σ2 and d, involves in general a vector Zn of five LLDs. This implies a third-order Hermite tensor that is comprised of 125 terms. In addition, following the procedure described previously necessitates a 5 by 5 transformation from Zn to a suitable vector containing the MLE and integrating out three superfluous variables. Further integration is required to obtain the marginal density expansion of the MLE of the parameter of interest. This task involves heavy algebra that is substantially in excess of what is presented here, and this paper concentrates solely on the uniparameter case.
4. MAIN RESULTS
When there is no confusion in what follows, we shall omit the dependence of various quantities on n and on d and use the following notation (see Lieberman, Rousseau, and Zucker, 2000):
and so forth, where
Let
and
In the Appendix, we show that
THEOREM 6. Under Assumption A1, the formal Edgeworth expansion for the density of
is
Let δn* =
. In view of (13), the density expansion for δn* is
Although the last expansion depends on d, the expressions (10)–(12) that appear in it can be approximated to yield a density expansion that does not depend on d and whose terms are of the same order.
COROLLARY 7. Under Assumption A1, the approximate Edgeworth expansion for the density of δn* is
The c.d.f. expansions and error rates are provided in the following theorem.
THEOREM 8. Under Assumption A1:
a. The formal Edgeworth expansion for the distribution of
is
b.
c. Let
Then
d.
We refer to (13) and (15) as the “exact” Edgeworth expansions and to (14) and (16) as the “approximate Edgeworth expansions.” Parts (b) and (d) of Theorem 8 state that the “exact” and “approximate” distribution expansions are valid uniform asymptotic expansions to o(n−1/2). To our knowledge, these explicit density and c.d.f. asymptotic expansions are the first for parametric models in the long-memory context. In related work in the short-memory (circular ARMA) case, Taniguchi (1984, pp. 49–50) obtains expansions whose coefficients are given in terms of similar integrals of spectral functionals and that agree with the approximate expansions given earlier.
4. DISCUSSION AND NUMERICAL EVALUATION
Figures 1, 2, 3, 4, 5, and 6 display Edgeworth density and c.d.f. expansions based on the formulas (13)–(16). The density was evaluated in the n = 20, d = 0.2 and 0.4 cases, and for the c.d.f. we also include results for n = 40. The MLEs were calculated by a simple grid search on the stationarity restricted interval [−0.49, 0.49] to ensure problem-free evaluation of the covariance matrix. Note that the interval also includes the short- and intermediate-memory cases to avoid a pile-up at the origin. The number of replications was 8,000. The cases considered here were also investigated by Lieberman et al. (2003), but because their paper does not include explicit expansions, their Edgeworth expansions are all based on simulated empirical cumulants. See Section 6 of Lieberman et al. (2003). In contrast, our experiment is based on the explicit Edgeworth expansions (13)–(16).
Density of , d = 0.2, n = 20.
Density of , d = 0.4, n = 20.
PP plots for the distribution of , d = 0.2, n = 20.
PP plots for the distribution of , d = 0.4, n = 20.
PP plots for the distribution of , d = 0.2, n = 40.
PP plots for the distribution of , d = 0.4, n = 40.
The densities were computed using a normal kernel and a plug-in bandwidth based on the rule of Silverman (1986) and are displayed in Figures 1 and 2. It is obvious that the densities are not symmetric. The asymmetry is partly caused by the fact that the maximization is restricted to the [−0.49, 0.49] interval and is therefore more enhanced for the d = 0.4 case. This asymmetry is captured, at least in part, by the correction factor in (14). The breakdown of the normal approximation and the “approximate” Edgeworth expansion are particularly vivid in the n = 20, d = 0.4 case (Figure 2). Here, the breakdown occurs not only at the tails but also at the center of the distribution. The “exact” Edgeworth expansion is more satisfactory in capturing the overall shape of the distribution in this case and makes a huge improvement at the center of the density, but it does so at the expense of some fluctuation, including negative density, in the tails.
We move on to evaluate the expansions to the c.d.f. of
. PP plots for the various approximations are provided in Figures 3, 4, 5, and 6, corresponding to n = 20, 40 and d = 0.2, 0.4. The simulated c.d.f. is taken to be the benchmark, and the closeness of the approximation to the 45° line indicates the accuracy of the approximation. We conducted simulation experiments for other cases, and our conclusions were not altered. It is apparent that the normal approximation is poor, especially in the upper tail and in the d = 0.4 case. The expansion (16) improves significantly over the normal approximation and overall behaves quite well in the d = 0.2 case. However, in the d = 0.4 case this expansion is poor in the upper tail. The exact Edgeworth expansion (15), on the other hand, is surprisingly accurate and is decidedly superior to the other approximations in both the n = 20 and d = 0.4 cases. The curve corresponding to it traces the 45° line very closely in the cases considered.
APPENDIX
Before we prove Lemma 3, we will state a proposition due to Lieberman et al. (2003). This is a uniform version of Theorem 5.1 of Dahlhaus (1989), who in turn extended an earlier result by Fox and Taqqu (1987) on the limiting behavior of a product of Toeplitz matrices.
LEMMA 9. Denote by gd,νj(λ) a spectral density derivative with respect to d of order 1 or 2 and let r be any positive integer. Under Assumption A1,
for any compact subset D* of D.
Proof of Lemma 3. We first obtain the elements of κn(d). Because the Zn,j's are centered quadratic forms in normal variables,
(see, e.g., Searle, 1971, p. 55). Because
, we have
Equations (A.2)–(A.5) imply that
Lemma 9, (A.6), and the definition of κ imply that uniformly in D*,
where
are the first- and second-order spectral density derivatives with respect to d, respectively. Because
. Now,
(see, e.g., Gradshteyn and Ryzhik, 1980). Hence,
and the lemma is proved. █
Proof of Lemma 4. Rearrangement of (5) gives
where
. Using the second Bartlett identity (e.g., Barndorff-Nielsen and Cox, 1989, p. 147),
where εn involves the Zn,j's. From Chibisov (1972, p. 629), the Edgeworth expansion to the distribution of the MLE to o(n−1/2) is equivalent to the Edgeworth expansion to the distribution of
to the same order, provided that
, for some sequence ρn such that
. Thus, we need to show that there are constants c1,j > 0, c2,j > 0, a > 0, and a small δ > 0, such that
where we have set
. The result follows on choosing a = (1 + 2δ)/4 and applying Chebyshev's inequality to the left-hand side of (A.8). █
LEMMA 10. Let
Then both bn,1 and bn,2 are O(1) uniformly in D*.
Proof of Lemma 10. Using
, we get
Thus
By Lemma 9, (A.2), and (A.10), μn,3′ and κn,1,1 have finite limits, uniformly in D*. █
LEMMA 11.
if at least one of the indices i,j,k is equal to 2. Moreover,
Proof of Lemma 11. By the definition of the covariant Hermite tensors,
which vanishes if at least one of the indices equals 2. When i = j = k = 1,
where h3(wn;κn,1,1(d)) is the third-order univariate Hermite polynomial. From Barndorff-Nielsen and Cox (1989, pp. 149–150)
and the proof is completed. █
Proof of Theorem 6. Let Wn = (Wn,1,Wn,2)′, with
. It follows from (6) and (A.9) that
The jacobian of the transformation from Zn to Wn is
Using Lemma 10,
Further,
The last two equalities together with (8) show that the formal density expansion for the MLE to o(n−1/2) is
The first term in (A.12) is
To solve the integral, we note that
and
With Lemma 11, (A.14), and (A.15), (A.13) becomes
where κn,1,1,1 is n−1 times the third-order cumulant of xn′An,1(d)xn, given by
The second term in (A.12) is, using (A.14)–(A.15),
Combining (A.16) and (A.18), (A.12) simplifies to
where
and
Now,
. So the density of
is
where
Finally, (A.9) and (A.19)–(A.21) imply
Substituting (A.10), (A.6), and (A.17) into (A.22), we obtain
which establishes (10). Similarly,
which establishes (11). The proof of the theorem is completed. █
Proof of Corollary 7. Using Lemma 9, (A.23), and (A.24), Cn,1* and Cn,3* converge uniformly in D* to 0 and
respectively. The proof of the corollary is completed on noting (A.7) and using the fact that ζ(2) = π2/6. █
Proof of Theorem 8.
a. We use the standard results
where φ(·),Φ(·) are the standard normal probability density function (p.d.f.) and c.d.f., respectively. This yields
c. This part of the theorem follows immediately from Theorem 4(b) of Lieberman et al. (2003).
Using the proof of Corollary 7, (A.7), the facts that (Cn,1* + 2Cn,3*/κn,1,1) converges uniformly in D* to −12ζ(3)/π2 and Cn,3* to −ζ(3), we get
as required.
d. Part (d) of the theorem is a consequence of parts (b) and (c) and the triangle inequality. █
References
REFERENCES
Andrews, D.W.K. &
O. Lieberman
(2002a)
Higher-Order Improvement of the Parametric Bootstrap for Long-Memory
Gaussian Processes.
Cowles Foundation discussion paper 1378,
Yale University.
Andrews, D.W.K. &
O. Lieberman
(2002b)
Valid Edgeworth Expansions for the Whittle Maximum Likelihood Estimator
for Stationary Long-Memory Gaussian Time Series.
Cowles Foundation discussion paper 1361,
Yale University.
Barndorff-Nielsen, O.E. &
D.R. Cox
(1989)
Asymptotic Expansions for Use in Statistics.
Chapman and Hall.
Bhattacharya, R.N. &
R.R. Rao
(1976)
Normal Approximation and Asymptotic Expansions.
Wiley.
Chambers, J.M.
(1967)
On methods of asymptotic approximation for multivariate
distributions.
Biometrika
54,
367–383.Google Scholar
Chibisov, D.M.
(1972)
An asymptotic expansion for the distribution of a statistic admitting an
asymptotic expansion.
Theory of Probability and Its Applications
17,
620–630.Google Scholar
Dahlhaus, R.
(1989)
Efficient parameter estimation for self-similar processes.
Annals of Statistics
17,
1749–1766.Google Scholar
Durbin, J.
(1980)
Approximation for densities of sufficient estimators.
Biometrika
67,
311–333.Google Scholar
Fox, R. &
M.S. Taqqu
(1987)
Central limit theorems for quadratic forms in random variables
having long-range dependence.
Probability Theory and Related Fields
74,
213–240.Google Scholar
Giraitis, L. &
P.M. Robinson
(2003)
Edgeworth expansions for the semiparametric Whittle estimator of
long memory.
Annals of Statistics
31,
1325–1375.Google Scholar
Gradshteyn, I.S. &
I.M. Ryzhik
(1980)
Table of Integrals, Series, and Products.
Academic Press.
Lieberman, O.,
J. Rousseau, &
D.M. Zucker
(2000)
Small-sample likelihood-based inference in the ARFIMA model.
Econometric Theory
16,
231–248.Google Scholar
Lieberman, O.,
J. Rousseau, &
D.M. Zucker
(2003)
Valid Edgeworth expansions for the maximum likelihood estimator of
the parameter of a stationary, Gaussian, strongly dependent series.
Annals of Statistics
31,
586–612.Google Scholar
Phillips, P.C.B.
(1977)
Approximations to some finite sample distributions associated with
a first order stochastic difference equation.
Econometrica
45,
463–485.Google Scholar
Searle, S.R.
(1971)
Linear Models.
Wiley.
Silverman, B.W.
(1986)
Density Estimation for Statistics and Data Analysis.
Chapman and Hall.
Taniguchi, M.
(1984)
Validity of Edgeworth expansions for statistics of time series.
Journal of Time Series Analysis
5,
37–51.Google Scholar
Density of , d = 0.2, n = 20.
Density of , d = 0.4, n = 20.
PP plots for the distribution of , d = 0.2, n = 20.
PP plots for the distribution of , d = 0.4, n = 20.
PP plots for the distribution of , d = 0.2, n = 40.
PP plots for the distribution of , d = 0.4, n = 40.
You have
Access
- 14
- Cited by