1. INTRODUCTION
For the stable first-order autoregressive (AR(1)) model, extensive literature on the finite-sample properties of the ordinary least squares (OLS) estimator of the autoregressive coefficient has developed; see Hurwicz (1950), White (1957, 1961), Shenton and Johnson (1965), Copas (1966), Phillips (1977), Sawa (1978), Grubb and Symons (1987), and Kiviet and Phillips (1993), to list just a few. In these studies, the error term was assumed to be normally distributed to enable the authors to derive analytical results. Peters (1989) considered the nonnormal case and derived the exact moments of the OLS estimator under an Edgeworth–Gram–Charlier–type nonnormal specification for the error term. Abadir and Lucas (2004) analyzed the mean squared error (MSE) in the nonnormal case for Ornstein–Uhlenbeck processes (hence nearly integrated AR models). Recently, Bao and Ullah (2006a) found that the second-order bias of the OLS estimator is robust to nonnormality. Under normality, they were able to derive the analytical approximate MSE result explicitly in terms of model parameters.
The purpose of this note is twofold. First, I will investigate whether the robust bias result still holds if one moves to a higher order, namely, O(T−2), where T is the sample size. Second, I will derive the analytical MSE result under a general error distribution. Consequently, one can study explicitly the effects of nonnormality on the sampling properties of the OLS estimator. In the next section, I consider the AR(1) model without an intercept and the model with an intercept, named the pure and intercept models, respectively. I show that for both models, nonnormality affects the approximate bias and MSE, up to O(T−2), of the OLS estimator through the skewness and kurtosis coefficients of the nonnormal density. Setting the nonnormal parameters equal to zero yields the normal results developed in the literature. The nonnormality coefficients do affect the bias of the OLS estimator through order O(T−2) terms. Section 3 concludes, and technical details are collected in the Appendix.
2. MAIN RESULTS
Consider the pure model
and the intercept model
where, throughout, the nonnormal independent and identically distributed (i.i.d.) error term εt in (1) and (2) is assumed to have the following moments mi = E(εti):
in which γ1 and γ2 are the Pearson's measures of skewness and kurtosis of the distribution and these and γ3,…, γ6 can be regarded as measures for deviation from normality. For a normal distribution, the parameters γ1,…, γ6 are all zero. Note that these γ's can also be expressed as cumulants of εt; e.g., γ1 and γ2 represent the third and fourth cumulants. For the two models, it is assumed that |ρ| < 1, σ2 = O(1), and the start-up observation y0 has been observed, either fixed or stochastic (and independent of εt, t > 0). For stochastic y0, it has bounded moments up to the sixth order, i.e., E(y0i) = O(1), i ≤ 6, and for the case of nonstochastic start-up value, y0 = O(1). For ease of presentation, I drop the expectation sign before y0i in what follows, where it is understood that for stochastic y0, one can simply put back the expectation signs before y0i in various places. For the intercept model, I assume α = O(1).
Following Kiviet and Phillips (1993), one can write the OLS estimator
as
where y−1 = (y0,y1,…, yT−1)′, ε = (ε1,ε2,…, εT)′, and A = I for pure model, A = M = I − T−1ιι′ for the intercept model, in which I is the T × T identity matrix and ι is a unit vector.1
In deriving their bias and MSE results for
, Bao and Ullah (2006a) used a somewhat different approach by following a Nagar-type (Nagar, 1959) expansion of the inverse of the expectation of the gradient of the moment condition used to estimate ρ.
then for the pure model y−1 = y0 F + Cε and for the intercept model y−1 = y0 F + αCι + Cε. Denoting yD = y0 F for the pure model and yD = y0 F + αCι for the intercept model, one can write y−1 = yD + Cε, and by substitution,
where rD = AyD. As such, one can easily verify that N = OP(T1/2), D = OP(T), D − E(D) = OP(T1/2), [D − E(D)][E(D)]−1 = OP(T−1/2). Now using a Nagar-type (Nagar, 1959) expansion, the bias and MSE approximations (for proof see the Appendix) are given as follows.2
It should be pointed that the case of |ρ| = 1 cannot be analyzed by following the Nagar-type expansion because of the stochastic limit of the denominator of (4).
THEOREM. Under nonnormally distributed error terms, the approximate bias and MSE, up to O(T−2), of the OLS estimator
for the pure model are
and for the intercept model,
Remark 1. Up to order O(T−2), only the skewness and kurtosis coefficients, γ1 and γ2, contribute to the bias and MSE of
for both models. Numerically, if one simply programs the calculations, there may be an impression that all the γ's are important in calculating the approximate moments, but in fact terms involving γ3,…, γ6 are of order o(T−2).
Remark 2. Interestingly, for both models, the effects of nonnormality on the MSE are the same, characterized by the last two terms in (7) and (9) inside the brackets. On the other hand, though the kurtosis has the same effects (2γ2 ρ), the skewness has different effects on the order O(T−2) bias for the two models.
Remark 3. The effects of the start-up value y0 and the error variance σ2 on the bias and MSE of
are of order O(T−2), and the asymptotic MSE (variance) of
for both models is given by the well-known formula (1 − ρ2)/T, regardless of the values of y0 and σ2 and the error distribution.
Remark 4. The robust (against nonnormality) bias results found in Bao and Ullah (2006a) are only valid up to order O(T−1), −2ρ/T for the pure model and −(1 + 3ρ)/T for the intercept model. The nonnormality coefficients γ1 and γ2 do affect the bias of
through order O(T−2) terms.
Remark 5. For the fixed start-up case y0 = 0, assuming normal density, the order O(T−2) MSE of the OLS estimator
for the pure model is (1 − ρ2)/T + (14ρ2 − 1)/T2. This was also derived in White (1961), Shenton and Johnson (1965), Phillips (1977), and Bao and Ullah (2006a).
Remark 6. In the expressions (8) and (9), one can see that y0 enters through (α − (1 − ρ)y0)2. This can be explained by rewriting the intercept model as in Evans and Savin (1984, p. 1244).
Remark 7. Along the lines of Abadir and Hadri (2000), one may be interested in finding if the approximate bias and MSE display nonmonotonicities by varying c ≡ |y0 /σ|. Abadir and Hadri (2000) found that for the pure model with normally distributed errors, the exact bias behaves nonmonotonically in the sense that (a) for large c, the exact bias can in fact increase as the sample size grows and (b) for given c and T, the exact bias is almost always nonmonotonic in |ρ|. In general, one should also find such nonmonotonicities by checking the partial derivatives of (6)–(9) with respect to T and ρ, largely because of the presence of the nonnormality parameters.
3For example, for the pure model,
. Even though 2γ12(1 + 4ρ − 10ρ3 + 2ρ4 + 3ρ6)/(1 − ρ3)2 ≥ 0 for all ρ ∈ (−1,1), the sign of
is still indeterminate.
when
and
when
, which implies that the magnitude of the approximate bias is a decreasing function of the sample size for the pure model.
3. CONCLUSION
I have derived the analytical bias and MSE formulas for the OLS estimator in the pure and intercept models under a general error distribution. It is found that the effects of nonnormality on the approximate moments of the OLS estimator come into play through the skewness and kurtosis coefficients of the nonnormal error distribution. Setting the skewness and kurtosis coefficients equal to zero, my general expressions are consistent with the normal results found in the literature.
APPENDIX
A.1. Expectations of
and
.
For expectations of quadratic forms of ε of order up to 3,
and
, where Ai are symmetric (for asymmetric Ai, they can be replaced with (Ai + Ai′)/2); the results were given in Chandra (1983), Ullah, Srivastava, and Chandra (1983), and Ullah (2004). For n = 4, see Bao and Ullah (2006b). The exact expressions are collected in an external document titled “Additional Material” available upon request from the author.
A.2. The Nagar expansion.
Because N = OP(T1/2), D = OP(T), D − E(D) = OP(T1/2), [D − E(D)][E(D)]−1 = OP(T−1/2), from (4), one can employ the following Nagar-type expansion:
where a−i/2 = N [E(D)]−1{−[D − E(D)][E(D)]−1}i−1 = (−1)i−1N [D − E(D)]i−1 ×[E(D)]−i = OP(T−i/2),i = 1,2,3,4. Given the expansion (A.1), one can immediately write the the approximate bias,
, and
, of
, up to order O(T−2), as
Upon substituting (5) into (A.2) and (A.3), the bias result and MSE results for both models essentially involve expectations of products of quadratic forms in the error vector ε (the exact expressions are available from the author upon request).
Numerically, one can always calculate all the expectations in the preceding expansions using the results given in Section A.1 of this Appendix by writing a computer program. Analytically, one can simplify the bias and MSE results in terms of model parameters.
A.3. Proof of theorem.
In general, because E(D) = O(T), for the order O(T−2) bias and MSE approximations (A.2) and (A.3), one only needs to keep terms of at least order O(T−1) in E(N), O(1) in E(ND) and E(N2), O(T) in E(ND2) and E(N2D), and O(T2) in E(ND3) and E(N2D2). As it turns out, many terms will drop out, and one can group the remaining terms as shown in Table A1.
Orders of terms in bias and MSE approximations
For the pure model, A = I, rD = y0 F. Given the special structure of C and F, one can verify the following using the results in Section A.1 of this Appendix by straightforward albeit tedious calculations (where for notational convenience βi = (1 − ρi)−1):
Further, because E(D) = y02 β2 + σ2(Tβ2 − β22) + o(T−1), for q = −1,−2,−3,−4, one can write
where x = T−1[y02/σ2 − β2 + o(T−1)]. Substituting the preceding results into (A.2) and (A.3) yields the bias and MSE results (6) and (7) for the pure model.
For the intercept model, A = M, rD = y0 MF + αMCι. Given the special structure of C, F, and M, one can verify the following using the results in Section A.1 of this Appendix by straightforward albeit tedious calculations:
Further, because E(D) = Tσ2β2 + β2 [y02 + α2β12 − 2αy0 β1 − σ2(ρ2 + 2ρ + 2)β2] + O(T−1), for q = −1,−2,−3,−4, one can write
where x = T−1[y02/σ2 + α2β12/σ2 − 2αy0 β1 /σ2 − (ρ2 + 2ρ + 2)β2 + O(T−1)]. Substituting the preceding results into (A.2) and (A.3) yields the bias and MSE results (8) and (9) for the intercept model.
