1. INTRODUCTION
Empirical models of economic time series often involve deterministic trend functions. Time polynomials and sinusoidal polynomials are the most common functions to appear in such models, and the properties of regressions of time series on these trend functions have been extensively explored in the literature, an early and definitive contribution being Grenander and Rosenblatt (1957, Ch. 7). A common element in much of the asymptotic theory that has been developed is a requirement of the type that ensures the existence of a positive definite limit to a suitably normalized sample second moment matrix of the regressors. Frequently, this requirement appears as one of a general set of conditions on the sample variances and autocovariances of the regressors, such as those that are often characterized (e.g., Hannan 1970, p. 215) as “Grenander's conditions” (see Grenander and Rosenblatt, 1957, pp. 233–234).
Not all deterministic functions of interest are covered by these requirements, and when the conditions fail some adjustments to the asymptotic theory are usually needed. One example that is important in certain empirical applications is the semilogarithmic growth model
where us is an unobserved error process. This type of formulation arises naturally in the study of growth convergence problems and economic transition. For instance, in the study of growth convergence (Barro and Sala-i-Martin, 2004), if σt2 denotes the variation across economies of the logarithm of per capita real output at time t, then under σ convergence, this dispersion declines over time. Accordingly, if we model dispersion in a power law form as σt2 = atβeut for some a > 0,β ≤ 0 and with accompanying disturbances ut, then yt = log(σt2) follows (1), and σ convergence may be tested using a robust sign test on the slope coefficient β.
In quite a different context, an analogous formulation arises in the log periodogram analysis of long memory, a subject on which there is now a large literature (see Robinson, 1995; Hurvich, Deo, and Brodsky, 1998; Phillips, 1999; and the references therein). In that case (discussed in Example (a) in Section 3), ys is the periodogram of the data measured at the Fourier frequencies λs = 2πs/n, s = 1,…, m ≤ n, and the slope coefficient β = −2d, where d is the memory parameter.
The reason model (1) fails to fit within the usual framework is that the sample moment matrix of the regressors is asymptotically singular. Indeed, setting
, and
, we have (cf. eqns. (26) and (27) in Section 3)
and
So, both the sample second moment of the regressors and its inverse have sin-gular limits after standardization, thereby failing Grenander's conditions.
The same problem arises when the logarithmic function in (1) is replaced by any slowly varying (SV) function L(s). In effect, the intercept and any SV function are asymptotically collinear after appropriate standardization. The phenomenon is manifest in a more serious way when one considers polynomial versions of (1) such as
or similar regressions involving polynomials in an SV function. In such cases, one finds that the sample moment matrix of the regressors, whereas of rank p + 1 for all n > p, is singular and of rank unity in the limit after suitable normalization. More generally still, the singularity persists when the regressors constitute a vector of different SV functions, such as {log s,1/log s} involving a logarithmic and inverse logarithmic trend.
In practical statistical work the phenomenon arises in nonlinear regressions of the type
where the trend exponent γ > −½ is to be estimated along with the regression coefficient β. The affine linear form of (2), taken about the true values of the parameters (denoted by β0 and γ0), involves the regressors sγ0 and sγ0(log s), which are regularly varying and whose second moment matrix is asymptotically singular upon appropriate (multivariate) normalization (cf. eqn. (55) in Section 6). It follows that statistical models like (2) manifest asymptotic collinearity analogous to that of the linear regression (1). Wu (1981, p. 509) noted that model (2) failed his conditions (which require a single normalizing quantity and a positive definite limit matrix for the second moment matrix of the affine model) for asymptotic normality and consequently did not provide a limit distribution theory for this model.
The present paper provides a detailed treatment of regressions of this type. The discussion is conducted in terms of SV regressors, and some results on polynomial and multivariate functions of slow variation are obtained that may be of interest outside the present study. The paper is organized as follows. Section 2 lays out some assumptions and preliminary theory. Results for simple regression are given with some common examples in Section 3. Polynomial regressions in SV regressors are covered in Section 4. Some general multivariate extensions are reported in Section 5. Section 6 applies the theory to the nonlinear trend model (2). Sections 7 and 8 contain supplementary technical results and proofs. Notation is tabulated as follows:
2. ASSUMPTIONS AND PRELIMINARY RESULTS
It will be convenient to use some standard theory of SV functions, and, in so doing, we shall repeatedly reference Bingham, Goldie, and Teugels (1987), hereafter designated as BGT. From the Karamata representation (e.g., BGT, Thm. 1.3.1, p. 12), any SV function L(x) has the representation
for some a > 0 and where c(·) is measurable with c(x) → c ∈ (0,∞) and ε(x) → 0 as x → ∞. The function ε in (3) is referred to as the ε-function corresponding to L.
The present paper works with the subclass of (so-called) normalized SV functions for which c(x) is a constant. In the development of an asymptotic theory of regression, little seems to be lost in making the restriction to constant c functions because the asymptotic behavior of L(x) is equivalent to that of (3) with c(x) = c. It is also known that for every SV function L there is an asymptotically equivalent SV function that is arbitrarily smooth (e.g., BGT, Thm. 1.3.3, p. 14). This property is especially helpful in developing asymptotic representations and working with transforms that arise from the process of integration and differentiation. The limit behavior studied subsequently is determined by L and ε, and some properties, as we shall see, are invariant to the particular SV function.
To validate the expansions needed in our development of an asymptotic theory of regression, we shall make the following assumption.
Assumption SSV.
(a) L(x) is a smoothly slowly varying (SSV) function with Karamata representation
for some a > 0 and where c > 0 is a constant, ε ∈ C∞, and ε(x) → 0 as x → ∞.
(b) |ε(x)| is SSV, and ε has Karamata representation
for some (possibly negative) constant cε and where η ∈ C∞, |η| is SSV, and η(x)2 = o(ε(x)) → 0 as x → ∞.
We call ε(x) and η(x) the ε- and η-functions of L(x). Under Assumption SSV we have
and, more generally,
(BGT, p. 44). The class for L(x) covered in Assumption SSV includes all of the common SV deterministic functions such as (for γ > 0) logγ x, 1/logγ x, log log x, and 1/log log x that might appear directly in simple regression formulations or indirectly in nonlinear regression through the corresponding affine linear models. The ε- and η-functions of these functions are given in Examples (a)–(e) in Section 3.
Because we contemplate the use of L as a time series regressor, the value of the initialization a in (4) is not important. In fact, we may reset a = 0 by taking ε(t) = 0 over t ∈ [0,δ] for some small δ > 0 and by interpolating ε over [δ,a] so that ε ∈ C∞[0,∞], thereby assuring existence, integrability, and smooth behavior for L over [0,a]. We shall henceforth presume that this change has been made and that we can majorize L(rn)/L(n) − 1 as follows:
where K(n) is SSV and g(r) ∈ C[0,1]. In consequence, and using the fact that for any SV function K, K(n)/nη → 0 for arbitrary η > 0, we have, given some α > 0 and any positive integer k,
where δ = α − η > 0.
To deliver an asymptotic theory of regression we need to appeal to a central limit result. For this purpose, it is convenient to assume that the regression errors us satisfy the following linear process condition.
Assumption LP. For all t > 0, ut has Wold representation
with et = iid(0,σe2) and μ2p = E|ut|2p < ∞ for some p > 2.
It is well known (e.g., Phillips and Solo, 1992, Thm. 3.4) that Assumption LP is sufficient for the partial sums
to satisfy the functional law
, where B(·) is Brownian motion with variance σ2 = σe2C(1)2. Further, extending the probability space as needed, the partial sum process St may be uniformly strongly approximated by a Brownian motion such as B, in the sense that
for some integer p > 2. Strong approximations such as (8) have been proved by many authors and are reviewed in Shorack and Wellner (1986) and Csörgö and Horváth (1993). A strong approximation justifying (8) in the case where ut is a linear process is given in Phillips (2006) for time series data under Assumption LP. Akonom (1993) gave (8) with an op(n−(1/2)+(1/p)) error under Assumption LP using the weaker moment requirement that μp = E|ut|p < ∞ for some p > 2.
It seems likely that the results of the present paper may be extended under suitable conditions to allow for long memory in ut. It has been shown by Wang, Lin, and Gulati (2003), for instance, that partial sums of long memory processes satisfy a strong approximation analogous to (8) in which the limit process B is replaced by a fractional Brownian motion. After suitable adjustments in convergence rates, that result may be used to extend some of the limit theory of the present paper, including (9) and Lemma 2.1. These extensions are not pursued here and are left for subsequent work.
Under Assumption LP, it follows by partial summation and by taking weak limits that for any f ∈ C1
Some related results hold when f is SV. In particular, we have the following lemma.
LEMMA 2.1. If L(t) satisfies Assumption SSV,
, and ut satisfies Assumption LP, then
(i)
as n → ∞.
(ii)
as n → ∞.
(iii)
as n → ∞.
2.1. Heuristics
As shown in (60) in Section 7 and Lemma 7.2, one of the implications of Assumption SSV is that we have the following asymptotic representation of L(t) for t = nr with r > 0:
Such a function may be called second-order SV (cf. de Haan and Resnick, 1996, who discuss second-order regular variation). For the sample mean L, we have
In consequence, the standardized sums that appear in (i)–(iii) of Lemma 2.1 have the approximate asymptotic forms
to which we may apply a standard central limit argument like that of (9). These cases indicate that, as far as first-order asymptotic theory is concerned, weighted means of ut with arbitrary SV weights behave in a common way, at least up to a normalization factor that depends on the asymptotic form of the SV function and its corresponding ε-function. The “common” form that appears in these expressions is that of a logarithmic trend function log t, whereas the influence of the particular SV function affects the normalization by way of L(n) and ε(n). This characteristic will be seen to apply more generally in regression asymptotics.
3. SIMPLE REGRESSION
We start with the simple regression model
where us satisfies Assumption LP. Initialization of (11) at some suitable finite integer a ensures that L(s) is well defined for s ≥ a in cases such as L(s) = 1/log s where L(1) is undefined. In such cases, L(s) may be redefined as L(s) = L(a) for 1 ≤ s ≤ a and some suitable finite a with no effect on subsequent results. Henceforth, it will be assumed that such adjustments have been made.
Let
and
be the least squares regression coefficients. The limit behavior of these regression coefficients depends on that of the first and second sample moments
The natural approach is to approximate these sample sums by an integral using Euler summation and then determine the asymptotic form of the resulting integrals as n → ∞. Lemma 7.1 gives for the kth moment
where δ > 0 is arbitrarily small, and Lemma 7.2 gives the following explicit asymptotic expansion:
The first two moments are then
and
from which we deduce that
as in Lemma 7.3.
Then
and
The limit theory for the regression coefficients now follows directly from (17) and Lemma 2.1.
THEOREM 3.1. If L(t) satisfies Assumption SSV and ut satisfies Assumption LP, then
Examples
(a) L(s) = log s. This gives the semilogarithmic model. Here, ε(n) = [1/(log n)], L(n)ε(n) = 1, and (18) is
This example also covers log periodogram analysis of long memory. In this case we have the regression
where IX(λs) is the periodogram of a time series (Xt)t=1n and λs = 2πs/n are fundamental frequencies. The spectrum of Xt is assumed to have the local form fx(λ) ∼ Cλ−2d for λ → 0+, and, correspondingly, the regression (20) is taken over a band of frequencies that shrink to the origin, so that (1/m) + (m/n) → 0. Then (20) has the alternate form
where
. Set c = log C, cn = c − 2d log(2π/n). The moment matrix of the regressors in (21) is asymptotically singular, just as in (1). Although the details of the central limit theory differ from Lemma 2.1 because of the properties of the residual terms in (20) (cf. Robinson, 1995; Hurvich et al., 1998), we nevertheless end up with a result analogous to (19) but with sample size m, namely,
Because
, we have
from which we deduce that
a result obtained by Robinson (1995, Thm. 3). The perfect negative asymptotic correlation between the estimates
and
induces a corresponding property between the estimates
and
of the original parameters appearing locally in fx(λ) ∼ Cλ−2d.
(b) L(s) = 1/(log s). This example arises when the regressor decays slowly. Here ε(n) = −[1/(log n)], L(n)ε(n) = −[1/(log2 n)], and (18) is
(c) L(s) = log log s. Here, ε(n) = [1/(log log n)][1/(log n)], L(n)ε(n) = [1/(log n)], and (18) is
(d) L(s) = [1/(log log s)]. Here, ε(n) = −[1/(log log n)][1/(log n)], L(n)ε(n) = −[1/(log2 log n)][1/(log n)], and (18) is
(e) L(s) = logγ s, γ > 0. In this case, ε(n) = [γ/(log n)], L(n)ε(n) = γ logγ−1 n, and (18) is
In all these cases the limit behavior is identical up to appropriate normalization of the coefficients, which is determined solely by L and its ε-function. Some intuition explaining the results is as follows. When L(n) → ∞ as n → ∞, the convergence rate of the slope coefficient,
, exceeds that of the intercept,
, because the signal from the regressor L(s) in (11) is stronger than that of a constant regressor. When L(n) → 0 as n → ∞, the convergence rate of
is less than that of
, because the signal from the regressor L(s) is weaker than that of a constant regressor. The singularity in the asymptotic distribution arises because the regressor and the intercept are asymptotically collinear.
Some of the first-order asymptotic approximations implied by the preceding theory may not be good in finite samples. Refinements of the limit theory can be developed using asymptotic expansions in suitable powers of the SV functions. When L(s) = 1/log s, for example, example (b) given previously is analyzed in Phillips and Sun (2003), where the following asymptotic expansion of the moment matrix of the regressors is developed in powers of L(n):
This series (24) is an asymptotic refinement of (58) in Section 7 for the case where L(s) = 1/log s and shows that the error in such approximations is in powers of 1/log n and therefore goes to zero rather slowly as n → ∞. However, such expansions are not needed in practical work because expressions such as
may be calculated directly from the data. Phillips and Sun (2003) provide some alternate approximations based on the logarithmic integral in this case.
3.1. Standard Errors
These are computed by scaling the square root of the diagonal elements of the inverse of the second moment matrix with an estimate of σ2 obtained from the regression residuals (either the sample variance, in the case where ut is iid(0,σ2), or an estimate of σ2 = σe2C(1)2 obtained by kernel methods in the stationary time series case (7)). Using (13) and (14), we have
where
and
Upon standardization with the diagonal matrix
, (25) becomes
Similarly, upon inversion, we have
which, upon standardization by
, gives
It follows from these formulas that, in spite of the singularity in the limit matrix, the covariance matrix of the regression coefficients is consistently estimated as in conventional regression when an appropriate estimate s2 of σ2 is employed.
4. POLYNOMIAL REGRESSION IN L(x)
In this model the regressors are polynomials in the SSV function L(s), and the data are generated by
where the regression error us satisfies Assumption LP. This model may be analyzed using the approach of the previous section. But, as the degree p increases in (28), the analysis becomes complicated because higher order expansions than (14) of the sample moments of L(s) are needed to develop a complete asymptotic theory. An alternate approach is to rewrite the model (28) in a form wherein the moment matrix of the regressors has a full rank limit. The degeneracy in the new model, which has an array format, then passes from the data matrix to the coefficients and is simpler to analyze.
The process is first illustrated with model (1), which we can write in the form
The regressors {1,log(s/n)} in (29) are not collinear. Writing k(r) = [1,log r]′ and using standard manipulations, we obtain
Because
, we deduce that
which leads directly to the earlier result (19).
Extending this process to the model (28) gives the representation
where
Define
and the model (28) becomes
Least squares estimation gives
The limit behavior of these coefficient estimates depends on that of the regressors Knj(t/n), and sample moment asymptotics for Knj follow from that of its sample mean. Define the vector Kn(t/n) = (Knj(t/n)) and the normalization matrix Dnε = diag[1,ε(n),ε(n)2,…, ε(n)p].
THEOREM 4.1.
(i) If L(t) satisfies Assumption SSV, then
where [ell ]p(r) = [1,log r,…,logp r]′ and
which is positive definite.
(ii) If L(t) satisfies Assumption SSV and ut satisfies Assumption LP, then
Next, we rewrite this limit distribution in terms of the original coefficients using relations (30)–(32). It transpires that only the final component,
, in
(which translates to the component
in the original coordinates) determines the nondegenerate part of the limit theory for the full set of coefficients.
THEOREM 4.2. If L(t) satisfies Assumption SSV and ut satisfies Assumption LP, then
where DnL = diag(1,L(n),…, L(n)p),
, and vp+1,p+1 = (p!)−2 is the p + 1th diagonal element of
.
The limit distribution of
has a support given by the range of the vector μp+1 and is therefore of dimension one. The variance matrix of
is given by
which, as we now show, is consistently estimated by the usual regression formula. The following result gives expressions for the asymptotic form of
and (L′L)−1, showing that, indeed, (37) is the asymptotic form of (L′L)−1.
THEOREM 4.3. If L(t) satisfies Assumption SSV, then
(i)
where ip+1 is a p + 1 vector with unity in each element.
(ii)
where [ell ]p(r) and μp+1 are given in Theorems 4.1 and 4.2.
It follows from (39) that, in spite of the singularity in the limit matrix, the covariance matrix of the regression coefficients is consistently estimated as in conventional regression by s2(L′L)−1 whenever s2 is a consistent estimate of σ2.
5. REGRESSION WITH MULTIPLE SSV REGRESSORS
Multiple regression with different SV functions as regressors is also of some interest in applications. One such formulation is given in the example that appears later in this section and involves an SV growth component in conjunction with a trend decay component that slowly adjusts the intercept in the regression to a lower level. Such a model is relevant in empirical research where one wants to capture simultaneously two different opposing trends in the data. Such models can be analyzed by the methods of the previous section, with the SV regressors replacing the polynomials in a given function L(s). We shall provide results for a model with two different regressors, which is the case of principal interest in practice and where our assumptions allow for a full treatment. We also briefly discuss the general case, where more structure is needed for a complete treatment.
Let Lj(s) (j = 1,2) be SSV functions with corresponding ε- and η-functions εj and ηj (j = 1,2). We consider the two-variable regression model
where the regression error us satisfies Assumption LP. An asymptotic theory of regression in this model is obtained by showing that (40) has an alternate, asymptotically equivalent, form involving a quadratic function of the simpler regressor log(s/n). Analysis similar to the previous section then applies.
Rewrite (40) as follows:
To transform the regressors in this version of the model, we note from Lemma 7.5 that Lj has a higher order representation in terms of its ε- and η-functions that has the asymptotic form
Equation (41) shows Lj to be third-order SV in the sense that
thereby extending the concept of second-order slow variation that appears in the earlier expression (10). Using the expansion (41) we write
giving a new form of the model with regressors that comprise a quadratic function in log(s/n). The new coefficients satisfy the system
For further asymptotic analysis, we impose the condition
which is necessary if we are to solve (43) for the original coefficients in (40). If (44) does not hold, then the regressors L1 and L2 are collinear to the second order in (41). In that case, the situation is more complex—higher order representations are needed to develop an asymptotic theory, and rates of convergence need to be adjusted. The following result holds under (44), uses only the second-order form (41), and gives the limit theory for the original coefficients in (40).
THEOREM 5.1. If L(t) satisfies Assumption SSV, ut satisfies Assumption LP, and δ(n) ≠ 0, then
where
and
5.1. Discussion
- Equation (42) indicates that multiple regression with different SV functions is asymptotically equivalent to polynomial regression on a logarithmic function. Theorem 5.1 shows that the outcome is analogous to that of a polynomial regression, but the rates of convergence are affected by the respective natures of the SV functions. The actual rate of convergence of the estimates depends not just on the asymptotic behavior of the functions Lj(n) and their ε-functions but also on the divergence, δ(n), between the sum of the ε- and η-functions of the two regressors L1 and L2. In effect, the more divergent are the Lj asymptotically, then the faster the rate of convergence of the regression estimates.
- The scaling factor εmin(n) in (45) relates to the constant in the regression and determines that its rate of convergence is affected by that of the more slowly converging regression coefficient.
- If Li(x) = log x for some i then there is no second-order term in (41) and εi(n) + ηi(n) = 0 in that case. The first matrix in (43) is simpler in this case and can be made upper triangular by permuting coefficients if necessary.
- Just as in the polynomial regression case, the limit distribution (45) is singular and has rank unity.
Example
The following example has iterated logarithmic growth, a trend decay component, and a constant regressor:
The secondary functions are ε1(n) = −[1/(log n)], η1(n) = −[1/(log n)], ε2(n) = [1/(log log n)][1/(log n)], and η2(n) = −[1/(log log n)] − [1/(log n)]. Then
We deduce that
The coefficient of the growth term converges fastest but at less than an
rate. The intercept converges next fastest, and finally the coefficient of the evaporating trend. All of these outcomes relate to the strength of the signal from the respective regressor.
5.2. The General Case
Consider the model
where L0(s) = 1. As in the preceding two-variable case, this model can be rewritten as
Assume that each Lj has a higher order representation extending (41) in terms of the following asymptotic expansion:
where εj1(n) = εj(n) and
for each j and each i > 1, so the coefficients, εji(n), in (48) decrease in order of magnitude as i increases. Such a higher order expansion can be developed under conditions analogous to Assumption SSV in which each function in the sequence L, ε, η,… itself has a Karamata representation with an ε-function that is SSV. Applying (48) in (47), we obtain the transformed model
The coefficients in this system satisfy
where
Define
and note that, in view of (49), we have
so that the final row (i = p) of Ξn has elements of the smallest order and the other rows decrease in magnitude as i increases. Then,
and, in view of the property of Ξn just mentioned, the first p − 1 columns of Mn = det(Ξn)Ξn−1 are of smaller order as n → ∞ than the final column of Mn. (Indeed, the columns of Mn progressively increase in order of magnitude from left to right.) We therefore have
so that
Turning to the intercept, we have αn0 = [1,L1(n),…, Lp(n)]β. Define
to be the ratio with the smallest order of magnitude as n → ∞. Then, we have
and scaling by det(Ξn)εmin and noting that det(Ξn)εmin = o(1) as n → ∞, we deduce that
giving the following result.
THEOREM 5.2. If L(t) satisfies Assumption SSV, ut satisfies Assumption LP, and det Ξn ≠ 0, then
where
is the ratio with the smallest order of magnitude as n → ∞.
In (50) the scale coefficients Lj(n)εj(n)/ηjp(n) and also εmin are implicitly signed. That is, the elements εj(n)/ηjj(n) may have positive or negative signs. In consequence, because the signs are built into the normalization factor, the covariance matrix of the limit distribution,
displays perfect positive correlation among the elements of the standardized vector in the limit.
6. NONLINEAR TREND REGRESSION
In the nonlinear trend model (2), let us satisfy Assumption LP, let θ0 = (β0,γ0) be the true values of the parameters, and assume that (β0,γ0) lies in the interior of the parameter space Θ = [0,b] × [−½,c] where 0 < b,c < ∞. Wu (1981, Exmp. 4, pp. 507, 509) considered the case where us is iid(0,σ2 > 0) and noted that the model satisfies his conditions for strong consistency of the least squares estimator
but not his conditions for asymptotic normality. There are two reasons for the failure: (i) the Hessian requires different standardizations for the parameters β and γ (whereas Wu's approach uses a common standardization); and (ii) the Hessian is asymptotically singular because of the asymptotic collinearity of the functions sγ0 and sγ0 log s that appear in the score (whereas Wu's theory requires the variance matrix to have a positive definite limit). Both issues are addressed by a version of the methods given earlier in the paper designed to deal with extremum estimation problems.
Setting
, the estimates
solve the extremum problem
and satisfy the first-order conditions
where
Expanding Sn(θ) about Sn(θ0), we have
where the Hessian Hn* is evaluated at mean values between θ0 and
and
The following lemmas assist in characterizing the asymptotic behavior of these quantities.
LEMMA 6.1 Let L be an SV function satisfying Assumption SSV and suppose us satisfies Assumption LP. Let Cn be a diagonal matrix all of whose elements diverge to ∞ as n → ∞. Define Nn0 = {θ ∈ Θ : ∥Cn(θ − θ0)∥ ≤ 1} to be a shrinking neighborhood of θ0 for any point θ0 in the interior of a compact parameter space Θ. Let f (r;θ) ∈ C2 over (r,θ) ∈ [0,1] × Θ and let the derivatives fθ = ∂f/∂θ, fr = ∂f/∂r, frθ = ∂2f/∂θ∂r be dominated as follows:
by functions {gi : i = 1,2,3}, which are absolutely integrable over [0,1]. Then
uniformly over θ ∈ Nn0.
LEMMA 6.2. Suppose us satisfies Assumption LP and let the true parameter vector θ0 = (β0,γ0) lie in the interior of Θ = [0,b] × [−½,c] where 0 < b, c < ∞. Define the normalization matrices
Define Cn = Dn /nδ for some small positive δ ∈ (0,γ0 + ½) and the following shrinking neighborhood of θ0:
Then
(i)
(ii)
(iii)
(iv)
(v)
Remarks.
(a) Part (i) of Lemma 6.2 reveals that the order of convergence of the first member of (52), the score Sn(θ0), is determined by the scaling factor Dn−1. However, from part (ii), the Hessian matrix under the same standardization by Dn−1 evidently has a singular limit as n → ∞, which prevents the application of the usual approach of solving (52) to find a limit theory for a standardized form of
. Part (iv) shows that upon standardization by Fn, rather than Dn, the inverse Hessian matrix converges but also has a singular limit.
(b) Part (v) is useful in showing that, after rescaling, the third term of (52) can be neglected in the asymptotic behavior of
.
(c) As the following result shows, the appropriate scaling factor for (52) is the matrix Fn−1, not Dn−1, even though Dn−1Sn(θ0) is Op(1).
THEOREM 6.3. In the model (2), let us satisfy Assumption LP and let the true parameter vector θ0 = (β0,γ0) lie in the interior of Θ = [0,b] × [−½,c] where 0 < b,c < ∞. Then, the least squares estimator
is consistent and has the following limit distribution as n → ∞:
Remarks.
(a) The estimator
has a convergence rate that is slower by a factor of log n than that of the score Sn(θ0). The reason is that the (conventionally standardized) Hessian Dn−1Hn(θ0)Dn−1 has an inverse that diverges at the rate log2 n and this divergence slows down the convergence rate of the estimator. Both the score and the Hessian need to be rescaled to achieve the appropriate convergence rate for
. With the new scaling we have
and then
From Lemma 6.2(iv), the matrix Fn Hn(θ0)−1Fn = Op(1) and has a singular limit. Also, as shown in (98) in Section 8, the matrix Fn−1[Hn* − Hn(θ0)]Fn−1 is op(1). Then
from which the limit distribution follows. Interestingly, even though the individual elements of Fn−1Sn(θ0) diverge, the relevant linear combination (Fn Hn(θ0)−1Fn)Fn−1Sn(θ0) is Op(1).
(b) The variance matrix for
is singular but is consistently estimated by
, where s2 is a consistent estimator of σ2, because
7. TECHNICAL SUPPLEMENT
LEMMA 7.1 (Averages of SV functions). If L(t) satisfies Assumption SSV, then for B ≥ 1
where δ > 0 is arbitrarily small.
Proof. Using Euler summation (e.g., Knopp, 1990, p. 521) we have
Because
for all δ > 0, we may choose a constant C such that for all t ≥ C and any δ > 0
Then, the final term in (56) may be bounded as follows:
It follows that
for any δ > 0 as n → ∞. █
LEMMA 7.2.
Proof. From Assumption SSV(b), |ε(x)| is SSV and
where η(n) → 0 as n → ∞. Like ε, η ∈ C∞, and if |η| is SSV
Then, using repeated integration by parts and the formulas L′(t) = L(t)ε(t)/t and ε′(t) = ε(t)η(t)/t, we find
because nη′(n)/η(n) = o(1) and η(n)2 = o(ε(n)) as n → ∞, thereby giving the stated result. █
Example
In the logarithmic case L(t) = log t, and ε(t) = [1/(log t)] and η(t) = −[1/(log t)]. Lemma 7.2 then gives the expansion
whereas successive integration by parts gives the exact result
so that the expansion in (57) is accurate to the third order.
LEMMA 7.3.
Proof. Applying the expansion from Lemma 7.2 and using expressions (15) and (16), we get
because η(n)2 = o(ε(n)). █
LEMMA 7.4. If L(t) is SV and satisfies Assumption SSV and (6), then
as n → ∞.
Proof. In view of Assumption SSV, we have
and, because |ε(t)| is SV, it follows by Karamata's theorem (e.g., BGT, Prop. 1.5.9a, p. 26) that for all r > 0
Then
The function L is second-order SV (for second-order regular variation, see de Haan and Resnick, 1996), in the sense that
Integration by parts gives
and so
giving the stated result. █
LEMMA 7.5. If L(t) satisfies Assumption SSV, then for all r > 0
as n → ∞.
Proof. Because both L and ε are SSV functions we have both (59) and
and, as in (60), we get for ε
Then
and we deduce that
as stated. █
Example
L(n) = [1/(log n)], ε(n) = −[1/(log n)], and η(n) = −[1/(log n)]. Then, by direct expansion we have for large n
which agrees with the third-order expansion given in (63).
LEMMA 7.6.
where (−k)j = (−k)(−k + 1) …(−k + j − 1).
Proof. This follows by successive integration by parts. █
LEMMA 7.7.
(i)
.
(ii)
.
Proof. Both parts follow by direct calculation. █
LEMMA 7.8.
(i)
where
and
(ii)
.
(iii)
.
Proof. Note that the (i,j)th element of the matrix
is (−1)i+j−2(i + j − 2)!. Consider the (i,j)th element of the matrix product Hp Fp2Hp′ and let j = k ≤ i. By direct calculation and using the representation
we find that this element is
where
is the hypergeometric function. Noting that the series terminates (because 1 − k is zero or a negative integer) and applying the summation formula (e.g., Erdélyi, 1953, p. 61),
where Γ is the gamma function, (64) reduces to
giving the required result and part (i). Parts (ii) and (iii) follow directly. █
8. PROOFS
Proof of Lemma 2.1. Part (i). By partial summation we have
where
. So
We now use the embedding of the standardized partial sum
in Brownian motion given in equation (8) following Assumption LP, namely,
Then
Next
has mean zero and variance
Observe that
For the last equality, note that L(t)ε(t) is (up to sign) on SSV function. We can then use Karamata's theorem, namely, that for α > −1 and an SV function [ell ], we have the asymptotic equivalence
(e.g., BGT, Prop. 1.5.8, p. 26), setting α = 0 to obtain (69). Using (69) in (68), the dominant term is
by applying (70) again. It follows that
as n → ∞. We deduce from (66), (67), and (71) that
Part (ii). We have
Lemma 7.2 gives
and using (66) and (67)
so that
Now, in view of the local law of the iterated logarithm for Brownian motion, we have
So, as in (70), we have
It follows from (73) that (72) is
as stated.
Part (iii). Start by considering
. By partial summation and the strong approximation in equation (8) following Assumption LP, we obtain, as in (67),
From (60) we have
so that
Thus,
as required. █
Proof of Theorem 4.1. Using Euler summation with f (t) = [(L(t)/L(n)) − 1]j we obtain, as in Lemma 7.1,
for arbitrarily small δ > 0, in view of (6). Hence, from (62) in the proof of Lemma 7.4, we have
so that
from which the stated limit results follow. The matrix
is positive definite because
implies a′[ell ]p(r) = 0 for all r, which implies a = 0, and part (i) is established.
To prove part (ii), we note by Lemma 2.1(iii) that
and proceeding in the same way as in the proof of that lemma but with an arbitrary linear combination of the preceding elements for j = 0,1,…, p, we get
By the Cramér–Wold device, we deduce that
Then, from (34), (35), and (75) we obtain
Proof of Theorem 4.2. From (32) and (36), we get for the final coefficient
where V−1 = (vi,j ) and
. A calculation (see Lemma 7.8(iii)) gives the final diagonal element of the inverse matrix V−1,
For the next coefficient, we have
and so
leading to
Next, for k = p − 2 we have
so that
More generally, proceeding in this way for p − 1 > k ≥ 0 (under the convention that
), we have
so that
We establish by induction (for decreasing k) that
We have already shown (77) to be valid for k = p − 1 and p − 2. Assume that it is valid for k + 1. Then, using (76) and Lemma 7.6 we have
showing that the result holds for k also.
Equation (77) gives an asymptotic correspondence between the elements of the least squares estimate
and its final component
that has the form
where DnL = diag(1,L(n),…, L(n)p) and
. We deduce that
giving the stated result. The explicit formula vp+1,p+1 = 1/(p!)2 follows from Lemma 7.8(iii). █
Proof of Theorem 4.3. First, transform the regressor space in (28) as follows:
where
and En = diag[1,ε(n),ε(n)2,…, ε(n)p], DnL = diag[1,L(n),L(n)2,…, L(n)p], and
In (78) an = Jn′ β = Enαn where αn is the parameter vector in (33) whose elements αnj are given in (30)–(32). Because Xs = Jn−1Ls = En−1H−1DnL−1Ls, we may rewrite (78) as
In view of (60), the vector En−1Kn(s/n) has elements
and so
The sample second moment matrix, L′L, of the regressors can now be written as
Because ε(n) → 0 as n → ∞, the matrix En = e1 e1′ + o(1), where e1 = (1,0,…,0)′, and the final expression given previously is
where ip+1 is the p + 1 sum vector (i.e., it has unity in each component). This gives the first result (38).
Next, consider the inverse sample moment matrix
Now observe that En−1 is dominated by its final diagonal element, and so we can write En−1 = (1/ε(n)p)ep+1 ep+1′[1 + o(1)] where ep+1 = (0,0,…,1)′. The final expression given previously is asymptotically equivalent to
giving the stated result. The second equality (79) holds because H−1′ep+1 = μp+1, the final column of H−1′, as is apparent from the fact that H′μp+1 = ep+1, which can be verified by direct multiplication using Lemma 7.7(ii). █
Proof of Theorem 5.1. Solving (43) for β1 and β2, we get
and so
Because εj(n) + ηj(n) = o(1) for j = 1,2, we have
where the coefficient 1/(2!)2 comes from the third diagonal element of the inverse matrix
. Finally, the constant term satisfies
which, in combination with (80), leads to
where
and
We deduce that
which gives the stated result upon scaling. █
Proof of Lemma 6.1. Setting
, using partial summation, and proceeding as in the proof of Lemma 2.1(i) we have
Assume that the probability space is constructed so that we can embed the standardized partial sum
in Brownian motion as in equation (8) following Assumption LP, namely,
Then, the first term of (81) clearly satisfies
as n → ∞ uniformly over θ ∈ Nn0. The second term of (81) is
Start with T1. We have f (t/n;θ) − f (t/n;θ0) = fθ(t/n;θ*)(θ − θ0) for some θ* ∈ Nn0, and so
as
as n → ∞. It follows that
uniformly over θ ∈ Nn0. But, just as in the proof of Lemma 2.1(i),
and so T1 = op(1) uniformly over θ ∈ Nn0.
Next, consider T2. We have f (t/n;θ) − f ((t − 1)/n;θ) = fr(t*/n;θ) for some t* ∈ (t − 1,t), and then
Now fr(t*/n;θ) − fr(t*/n;θ0) = frθ(t*/n;θ*)(θ − θ0) for some θ* ∈ Nn0 and
and, just as in (84), we find that
Moreover,
as n → ∞. It follows from (85) and (86) that
uniformly over θ ∈ Nn0. We deduce from (81)–(83) and (87) that
uniformly over θ ∈ Nn0, giving the stated result. █
Proof of Lemma 6.2. Part (i). For any slowly varying function L satisfying Assumption SSV and any function f ∈ C1, we can show in the same way as Lemma 6.1 that
extending (9). The limit (54) follows directly.
Part (ii). Using Lemma 6.1 and (89), the asymptotic form of Dn−1Hn(θ0)Dn−1 is
as stated.
Part (iii). Upon calculation of (90) and rescaling, we deduce that
whose eigenvalues are evidently O(log2 n) and O(log n), respectively, provided β0 ≠ 0.
Part (iv). First calculate
Then
We deduce that
as given.
Part (v). Define Cn = Dn /nδ for some small positive δ ∈ (0,γ0 + ½), so that Cn Dn−1 = O(n−δ) = o(1) and λmin(Cn) → ∞ as n → ∞, where λmin denotes the smallest eigenvalue. Construct the following shrinking neighborhood of θ0:
and define the matrix
We show that
Note that in Nn0 we have
Also, because γ0 > −½ we can choose ε > 0 such that γ0 > −½ + ε, and then we have the dominating function
Consider the individual elements of Cn−1[Hn(θ) − Hn(θ0)]Cn−1 in turn. First, for γ* between γ and γ0, we have
Next
uniformly for θ ∈ Nn0 in view of the majorization (93). It follows that for large enough n
Next,
We find
in the same way as (94). For term T2, in view of (89) we have for each γ ∈ Nn0
By Lemma 7.1 we have
uniformly over θ ∈ Nn0, and
uniformly over θ ∈ Nn0 for large enough n. Hence,
as n → ∞. The argument for the term a22,n is entirely analogous, and (92) therefore follows. █
Proof of Theorem 6.3. Standard asymptotic arguments of nonlinear regression for nonstationary dependent time series (e.g., Wooldridge, 1994, Thm. 8.1) may be applied. But, modifications to the arguments need to be made to attend to the singularity arising from the asymptotically collinear elements sγ0 and sγ0 log s that appear in the score Sn(θ0). First, the demonstration that there is a consistent root of the first-order conditions (51) in an open, shrinking neighborhood of θ0 follows as in the proof of Wooldridge's Theorem 8.1 using (52) and Lemma 6.2(v). There are two changes in the proof that are needed: (i) the standardizing matrix is Fn−1 in place of Dn−1, as discussed in Remark (c) following Lemma 6.2; (ii) the scaled Hessian matrix Fn−1Hn(θ0)Fn−1 does not tend to a positive definite limit with finite eigenvalues bounded away from the origin. Instead, as shown in the proof of Lemma 6.2(iii), Fn−1Hn(θ0)Fn−1 is positive definite for all large n and has eigenvalues of order O(log2 n) and O(log n), and the smallest eigenvalue λmin(Fn−1Hn(θ0)Fn−1) = O(log n) → ∞. With these changes, the remainder of the consistency argument in Wooldridge's Theorem 8.1 holds, and we obtain
.
Next, scaling the first-order conditions (52), we have
and then
Note that Fn = (1/log n)Dn = diag[nγ0+1/2/log n,nγ0+1/2] and because Dn−1Sn(θ0) = Op(1) from Lemma 6.2(i), we have Fn−1Sn(θ0) = Op(log n). However, from (91) in the proof of Lemma 6.2(iv) we have
and from Lemma 6.2(v) we have
where Nn0 = {θ ∈ Θ : ∥Cn(θ − θ0)∥ ≤ 1}, a shrinking neighborhood around θ0 with Cn = Dn /nδ for some small δ > 0. Because
, it follows that
with probability approaching unity as n → ∞, where θ* is a generic mean value between
and θ0. Hence
and so combining (96) and (98) we have
Then, from (91), (95), and (99) we deduce that
giving the stated result. █
