1. MOTIVATION AND RESULTS
Consider the AR(1) process yt = ρyt−1 + εt,
, where
is the set of integers, εt ∼ iid(0,1), and ρ ∈ (−1,1]. The error variance is set to unity without loss of generality in the present context because otherwise var(εt)−1/2yt can be considered. Let Δyt = yt − yt−1. The covariance matrix of Δy = (Δy1,…,Δyn)′ is a symmetric Toeplitz matrix with its (t,s) element equal to ω|t−s| := EΔytΔys; in the unit root case Δyt = εt. Similarly, the covariance matrix of the double-differenced series Δ2yt := Δyt − Δyt−1 is another symmetric Toeplitz matrix whose (t,s) element depends on |t − s|.
Evaluating the determinant of covariance matrix for Δyt and Δ2yt is sometimes useful, especially when working with the exact Gaussian likelihood functions derived from the first- and double-differenced data. A prominent example is the simple dynamic panel data model with unobservable fixed effects yit = (1 − ρ)αi + ρyit−1 + εit. One may want to first-difference the data to eliminate the nuisance fixed effects and get Δyit = ρΔyit−1 + Δεit and then derive the likelihood function for {Δyit}. (See Hsiao, Pesaran, and Tahmiscioglu, 2002.) If the model also contains incidental trends as in yit = αi + (1 − ρ)γit + ρyit−1 + εit, then a double-differencing method can eliminate the incidental trends. This possibility is investigated by Han and Phillips (2006), who find (using results in the present note) some interesting facts, e.g., that a panel unit root test based on double-differenced data can outperform point optimal tests such as Ploberger and Phillips (2002) if the time span is small relative to the cross-sectional dimension.
In the preceding dynamic panel data model, approximate or conditional maximum likelihood estimation (MLE) may yield poor estimators especially if the time dimension is small and the cross-sectional sample size is large because then small errors due to inaccurate approximation may accumulate as the cross-sectional dimension grows.
When asymptotics is of the only concern, one may be interested in the divergence rates of the determinants rather than their exact formulas, and so a Taylor expansion may be applied. A general theorem in Grenander and Szegö (1958) is as follows. Let An be a sequence of Toeplitz matrices (not necessarily symmetric) whose (t,s) element is as−t. One of Szegö's theorems states that
where
is the Fourier form, under the regularity condition that log f(x) is bounded for x ∈ [−π,π]. (See Grenander and Szegö, 1958, p. 64.) This theorem is especially useful if the right-hand side of (1) is finite and bigger than zero, in which case n−1 log|An| converges to a nonzero quantity.
But in our cases log f(x) is not bounded. For a stationary autoregressive moving average (ARMA) with a unit root in the moving average (MA) part, the spectral density and thus f(x) are equal to zero at x = 0, leading to unbounded log f(x) in the range [−π,π], and the result is not applicable.
Another paper that examines asymptotics for ARMA processes with possible MA unit roots is McCabe and Leybourne (1998); it does not derive the determinant explicitly. Additionally the analysis in that paper is conditional on the first observation, whereas we are interested in unconditional MLE.
Because neither Szegö's theorem nor the McCabe and Leybourne (1998) method is applicable, the present note explicitly derives the determinants of covariances for first- and double-differenced AR(1) processes. Not only does this derivation clarify the order of the determinants as the time dimension increases, but it also reveals the functional form of the determinant in terms of the autoregressive (AR) coefficient, and so some “local-to-unity” asymptotics (i.e., asymptotics when the AR coefficient marginally departs from unity) may be analyzed. See Han and Phillips (2006) for an application.
Exact Gaussian MLE based on first-differencing is analyzed by Hsiao et al. (2002) in the dynamic panel context with short time dimensions and large cross-sectional sizes, where the determinant of covariance matrix is explicitly provided. Note that this determinant can also be derived from Galbraith and Galbraith (1974, p. 68) using L'Hôpital's rule. The double-differencing case is much more complicated, on the other hand, and has not been explicitly obtained yet as far as the author knows. Haddad (2004) obtains a closed form representation by some recursion for the inverse and the determinant of ARMA(p,q) processes, but his results require stationarity and invertibility (i.e., AR and MA roots outside the unit circle) and hence do not apply to the present case. Galbraith and Galbraith (1974) provide a generally applicable method, but the algebra is quite involved, partly because invertibility is required for the derivation. Zinde-Walsh (1988) provides a general method for computing determinants of ARMA, but applying this general methodology to the double-differenced AR(1) case may be overly complicated. Applying the usual cofactor expansion did not produce useful results, either.1
I thank an anonymous referee for helpful comments on the literature.
The method used in the present note expresses the determinants in terms of difference equations. It is tailored for the differenced AR(1) processes, and so the derivation and proof are relatively simple. The covariance matrix for the first-differenced data has the following simple form of the determinant.
THEOREM 1. Let yt = ρyt−1 + εt with εt ∼ iid(0,1). Let Ωn = EΔyΔy′ where Δy = (Δy1,…,Δyn)′ with Δyt = yt − yt−1. Then
As noted earlier, this result is already known, but we still present it here for completeness and for illustrating our method of derivation and proof. (See the proof that follows.) We can apply the same method to the double-differenced case. The determinant of the covariance matrix in that case is given as follows.
THEOREM 2. Let yt = ρyt−1 + εt with εt ∼ iid(0,1). Let, this time, Ωn = EΔ2yΔ2y′ where Δ2y = (Δ2y1,…,Δ2yn)′ with Δ2yt = Δyt − Δyt−1. Then
2. PROOFS AND DISCUSSION
Proof of Theorem 1. From
for all ρ ∈ (−1,1], we find that Ωn := EΔyΔy′ is a symmetric Toeplitz matrix whose (t,s) element is ω|t−s| such that ω0 = 2/(1 + ρ) and ωj = −ρj−1(1 − ρ)/(1 + ρ) for j ≥ 1. (Also see Karanasos, 1998.) It is easy to see that
thus |Ωn+1| = dn|Ωn|, where dn = ω0 − ξn′Ωn−1ξn. By the inversion formula for partitioned matrices, we have
Now, because ωj+1 = ρωj for j ≥ 1, we have ξn+1 = (ω1,ρξn′)′, implying that
where ξn′Ωn−1ξn = ω0 − dn is used. Thus
for n ≥ 1. Because dn = |Ωn+1|/|Ωn| for n ≥ 1, this difference equation leads to |Ωn+2| = π1|Ωn+1| − π2|Ωn| for n ≥ 1. Now, the exact formulas of ω0 and ω1 imply that π1 = 2 and π2 = 1, and so the preceding identity implies that |Ωn+2| − |Ωn+1| = |Ωn+1| − |Ωn|. Obviously |Ωn| can be computed from a sequence of equal steps, and therefore it is linear in n. Its closed form is derived from the initial condition that |Ω1| = ω0 = 2/(1 + ρ) and |Ω2| = ω02 − ω12 = (3 − ρ)/(1 + ρ), and the final result follows immediately. (For more general treatment of linear difference equations, see, e.g., Hoy, Livernois, McKenna, Rees, and Stengos, 1996.) █
Now let us consider the double-differenced case. The notations in the preceding proof are used here too but with different meanings, which are explained in relevant places. Again yt = ρyt−1 + εt with εt ∼ iid(0,1).
Proof of Theorem 2. Let ωj = EΔ2ytΔ2yt−j. Because
we have ω0 = 2(3 − ρ)/(1 + ρ), ω1 = −(4 − 3ρ + ρ2)/(1 + ρ), and ωj = ρj−2(1 − ρ)3/(1 + ρ) for j ≥ 2. (See also Karanasos, 1998.) Let Δ2y = (Δ2y1,…,Δ2yn) and Ωn = EΔ2yΔ2y′. Then Ωn is the symmetric Toeplitz matrix whose (t,s) element is ω|t−s|, again. The partition (2) and the inverse (3) are still valid with the redefined ξn = (ω1,…,ωn)′ and dn = ω0 − ξn′Ωn−1ξn. But now we have ωj+1 = ρωj for j ≥ 2 (rather than j ≥ 1), and we do not have the simplicity of the previous case. Instead, by noting that ω2 = ρω1 + 1 and ωj+1 = ρωj for j ≥ 2,
where en is the first column of In. The extra (0,en′)′ term slightly complicates the recursion, but we can still proceed as follows.
Similarly to the first-differencing case, using (4) for the first term in (5) and denoting an = en′Ωn−1ξn we get
where the terms are simplified using ξn′Ωn−1ξn = ω0 − dn and en′Ωn−1en = 1/dn−1. So we have
Also, by expanding an+1 = ξn+1′Ωn+1−1en+1 using the partitioned inverse (3), we get
By change of the variable an to cn := ω1 − ρω0 − an, (6) and (7) are rewritten as
where
and
. Similarly to the first-differencing case, we note that dn = |Ωn+1|/|Ωn|, and by further transforming cn to μn := cn|Ωn|, the two difference equations in (8) are, respectively, written as
where Dn ≡ |Ωn| for notational brevity.
Solving (9) and (10) is not straightforward.2
I thank an anonymous referee for suggesting simplifying the proofs, which gave me inspiration that eventually developed into this solution.
instead of (9). Now (10) and (11) imply that (1 − L)5μn+1 = 0, and so μn is a fourth-order polynomial of n. (This is because (1 − L)4μn+1 is constant, and so (1 − L)3μn+1 is linear in n, and so on.) Thus Dn is also a fourth-order polynomial of n because of (10). The coefficients for the polynomial can be determined from D1,…,D5. The detailed algebra is omitted. █
Extension of the present method to higher order AR processes is conceptually possible. Consider the differenced AR(p) process
. Note that multiple AR unit roots are not allowed because then Δyt are not covariance stationary. Furthermore, if φi = 1 for some i, then it leads to a pure AR(p − 1) process
with |φk| < 1 for k ≠ i, for which the determinant is widely available, including Galbraith and Galbraith (1974). So we assume that |φk| < 1 for all k.
The covariance matrix for Δyt in this case is explicitly calculated by Karanasos (1998), and its determinant can be expressed in terms of nonlinear simultaneous difference equations, which are not presented in this note. Solving them is quite challenging even for p = 2, and the double-differencing case seems still more complicated. Yet, derivation would possibly be attempted along this line if other methods (e.g., Galbraith and Galbraith, 1974) are overly complicated.
