1. INTRODUCTION AND OVERVIEW
In many statistical applications a data-based model-selection step precedes the final parameter estimation and inference stage. For example, the specification of the model (choice of functional form, choice of regressors, number of lags, etc.) is often based on the data. In contrast, the traditional theory of statistical inference is concerned with the properties of estimators and inference procedures under the central assumption of an a priori given model. That is, it is assumed that the model is known to the researcher prior to the statistical analysis, except for the value of the true parameter vector. As a consequence, the actual statistical properties of estimators or inference procedures following a data-driven model-selection step are not described by the traditional theory, which assumes an a priori given model; in fact, they may differ substantially from the properties predicted by this theory; cf., e.g., Danilov and Magnus (2004), Dijkstra and Veldkamp (1988), Pötscher (1991, Sect. 3.3), and Rao and Wu (2001, Sect. 12). Ignoring the additional uncertainty originating from the data-driven model-selection step and (inappropriately) applying traditional theory can hence result in very misleading conclusions.
Investigations into the distributional properties of post-model-selection estimators, i.e., of estimators constructed after a data-driven model-selection step, are relatively few and of recent vintage. Sen (1979) obtained the unconditional large-sample limit distribution of a post-model-selection estimator in an independent and identically distributed (i.i.d.) maximum likelihood framework when selection is between two competing nested models. In Pötscher (1991) the asymptotic properties of a class of post-model-selection estimators (based on a sequence of hypothesis tests) were studied in a rather general setting covering nonlinear models, dependent processes, and more than two competing models. In that paper, the large-sample limit distribution of the post-model-selection estimator was derived, both unconditional and conditional on having chosen a correct model, not necessarily the minimal one. See also Pötscher and Novak (1998) for further discussion and a simulation study and Nickl (2003) for extensions. The finite-sample distribution of a post-model-selection estimator, both unconditional and conditional on having chosen a particular (possibly incorrect) model, was derived in Leeb and Pötscher (2003) in a normal linear regression framework; this paper also studied asymptotic approximations that are in a certain sense superior to the asymptotic distribution derived in Pötscher (1991). The distributions of corresponding linear predictors constructed after model selection were studied in Leeb (2005, 2006). Related work can also be found in Sen and Saleh (1987), Kabaila (1995), Pötscher (1995), Ahmed and Basu (2000), Kapetanios (2001), Hjort and Claeskens (2003), Dukić and Peña (2005), and Leeb and Pötscher (2005a). The latter paper provides a simple exposition of the problems of inference post model selection and may serve as an entry point to the present paper.
It transpires from the papers mentioned previously that the finite-sample distributions (and also the large-sample limit distributions) of post-model-selection estimators typically depend on the unknown model parameters, often in a complicated fashion. For inference purposes, e.g., for the construction of confidence sets, estimators of these distributions would be desirable. Consistent estimators of these distributions can typically be constructed quite easily, e.g., by suitably replacing unknown parameters in the large-sample limit distributions by estimators; cf. Section 2.2.1. However, the merits of such “plug-in” estimators in small samples are questionable: It is known that the convergence of the finite-sample distributions to their large-sample limits is typically not uniform with respect to the underlying parameters (see Appendix B in the present paper and Leeb and Pötscher, 2003, Cor. 5.5), and there is no reason to believe that this nonuniformity will disappear when unknown parameters in the large-sample limit are replaced by estimators. This observation is the main motivation for the present paper to investigate in general the performance of estimators of the distribution of a post-model-selection estimator, where the estimators of the distribution are not necessarily “plug-in” estimators based on the limiting distribution. In particular, we ask whether estimators of the distribution function of post-model-selection estimators exist that do not suffer from the non-uniformity phenomenon mentioned earlier. As we show in this paper the answer in general is “No.” We also show that these negative results extend to the problem of estimating the distribution of linear functions (e.g., linear predictors) of post-model-selection estimators. Similar negative results apply also to the estimation of the mean squared error or bias of post-model-selection estimators; cf. Remark 4.7.
To fix ideas consider for the moment the linear regression model
where V and W, respectively, represent n × k and n × l nonstochastic regressor matrices (k ≥ 1,l ≥ 1) and the n × 1 disturbance vector u is normally distributed with mean zero and variance-covariance matrix σ2In. We also assume for the moment that (V : W)′(V : W)/n converges to a nonsingular matrix as the sample size n goes to infinity and that limn→∞ V′W/n ≠ 0 (for a discussion of the case where this limit is zero see Example 1 in Sect. 2.2.2). Now suppose that the vector χ represents the parameters of interest and the parameter vector ψ and the associated regressors in W have been entered into the model only to avoid possible misspecification. Suppose further that the necessity to include ψ or some of its components is then checked on the basis of the data; i.e., a model-selection procedure is used to determine which components of ψ are to be retained in the model, the inclusion of χ not being disputed. The selected model is then used to obtain the final (post-model-selection) estimator
for χ. We are now interested in the unconditional finite-sample distribution of
(appropriately scaled and centered). Denote this k-dimensional cumulative distribution function (cdf) by Gn, θ, σ(t). As indicated in the notation, this distribution function depends on the true parameters θ = (χ′, ψ′)′ and σ. For the sake of definiteness of discussion assume for the moment that the model-selection procedure used here is the particular “general-to-specific” procedure described at the beginning of Section 2; we comment on other model selection procedures, including the Akaike information criterion (AIC) and thresholding procedures, later.
As mentioned previously, it is not difficult to construct a consistent estimator of Gn,θ,σ(t) for any t, i.e., an estimator
satisfying
for each δ > 0 and each θ, σ; see Section 2.2.1. However, it follows from the results in Section 2.2.2 that any estimator satisfying (2), i.e., any consistent estimator of Gn,θ,σ(t), necessarily also satisfies
for suitable positive constants R and δ that do not depend on the estimator. That is, although the probability in (2) converges to zero for every given θ by consistency, relation (3) shows that it does not do so uniformly in θ. It follows that
can never be uniformly consistent (not even when restricting consideration to uniform consistency over all compact subsets of the parameter space). Hence, a large sample size does not guarantee a small estimation error with high probability when estimating the distribution function of a post-model-selection estimator. In this sense, reliably assessing the precision of post-model-selection estimators is an intrinsically hard problem. Apart from (3), we also provide minimax lower bounds for arbitrary (not necessarily consistent) estimators of the cumulative distribution function Gn,θ,σ(t). For example, we provide results that imply that
holds for suitable positive constants R and δ, where the infimum extends over all estimators of Gn,θ,σ(t). The results in Section 2.2.2 in fact show that the balls ∥θ∥ < R in (3) and (4) can be replaced by suitable balls (not necessarily centered at the origin) shrinking at the rate n−1/2. This shows that the nonuniformity phenomenon described in (3) and (4) is a local, rather than a global, phenomenon. In Section 2.2.2 we further show that the nonuniformity phenomenon expressed in (3) and (4) typically also arises when the parameter of interest is not χ but some other linear transformation of θ = (χ′,ψ′)′. As discussed in Remark 4.3, the results also hold for randomized estimators of the unconditional distribution function Gn,θ,σ(t). Hence no resampling procedure whatsoever can alleviate the problem. This explains the anecdotal evidence in the literature that resampling methods are often unsuccessful in approximating distributional properties of post-model-selection estimators (e.g., Dijkstra and Veldkamp, 1988; Freedman, Navidi, and Peters, 1988). See also the discussion on resampling in Section 6.
The results outlined previously are presented in Section 2.2 for the particular “general-to-specific” model-selection procedure described at the beginning of Section 2. Analogous results for a large class of model-selection procedures, including the AIC and thresholding procedures, are then given in Section 3, based on the results in Section 2.2. In fact, the nonuniformity phenomenon expressed in (3) and (4) is not specific to the model-selection procedures discussed in Sections 2 and 3 of the present paper but will occur for most (if not all) model-selection procedures, including consistent ones; cf. Sections 5 and 6 for more discussion. Section 5 also shows that the results are—as is to be expected—by no means limited to the linear regression model.
We focus on the unconditional distributions of post-model-selection estimators in the present paper. One can, however, also envisage a situation where one is more interested in the conditional distribution given the outcome of the model-selection procedure. In line with the literature on conditional inference (see, e.g., Robinson, 1979; Lehmann and Casella, 1998, p. 421), one may argue that, given the outcome of the model-selection step, the relevant object of interest is the conditional rather than the unconditional distribution of the post-model-selection estimator. In this case similar results can be obtained and are reported in Leeb and Pötscher (2006b). We note that on a technical level the results in Leeb and Pötscher (2006b) and in the present paper require separate treatment.
The plan of the paper is as follows: Post-model-selection estimators based on a “general-to-specific” model-selection procedure are the subject of Section 2. After introducing the basic framework and some notation, such as the family of models Mp from which the “general-to-specific” model-selection procedure
selects, and also the post-model-selection estimator
, the unconditional cdf Gn,θ,σ(t) of (a linear function of) the post-model-selection estimator
is discussed in Section 2.1. Consistent estimators of Gn,θ,σ(t) are given in Section 2.2.1. The main results of the paper are contained in Sections 2.2.2 and 3: In Section 2.2.2 we provide a detailed analysis of the nonuniformity phenomenon encountered in (3) and (4). In Section 3 the “impossibility” result from Section 2.2.2 is extended to a large class of model-selection procedures including the AIC and to selection from a nonnested collection of models. Some remarks are collected in Section 4, and Section 5 discusses extensions and the scope of the results of the paper. Conclusions are drawn in Section 6. All proofs and some auxiliary results are collected in Appendixes. Finally a word on notation: The euclidean norm is denoted by ∥·∥, and λmax(E) denotes the largest eigenvalue of a symmetric matrix E. A prime denotes transposition of a matrix. For vectors x and y the relation x ≤ y (x < y, respectively) denotes xi ≤ yi (xi < yi, respectively) for all i. As usual, Φ denotes the standard normal distribution function.
2. RESULTS FOR POST-MODEL-SELECTION ESTIMATORS BASED ON A “GENERAL-TO-SPECIFIC” MODEL SELECTION PROCEDURE
Consider the linear regression model
where X is a nonstochastic n × P matrix with rank(X) = P and u ∼ N(0,σ2In), σ2 > 0. Here n denotes the sample size, and we assume n > P ≥ 1. In addition, we assume that Q = limn→∞ X′X/n exists and is nonsingular. In this section we shall—in a similar way as in Pötscher (1991)—consider model selection from the collection of nested models
, where
is specified by the user and where for 0 ≤ p ≤ P the model Mp is given by
(In Section 3 also general nonnested families of models will be considered.) Clearly, the model Mp corresponds to the situation where only the first p regressors in (5) are included. For the most parsimonious model under consideration, i.e., for
, we assume that
satisfies
; if
, this model contains as free parameters only those components of the parameter vector θ that are not subject to model selection. (In the notation used in connection with (1) we then have
and
.) Furthermore, note that M0 = {(0,…,0)′} and that MP = RP. We call Mp the regression model of order p.
The following notation will prove useful. For matrices B and C of the same row dimension, the column-wise concatenation of B and C is denoted by (B : C). If D is an m × P matrix, let D[p] denote the m × p matrix consisting of the first p columns of D. Similarly, let D[¬p] denote the m × (P − p) matrix consisting of the last P − p columns of D. If x is a P × 1 vector, we write in abuse of notation x[p] and x[¬p] for (x′[p])′ and (x′[¬p])′, respectively. (We shall use the preceding notation also in the “boundary” cases p = 0 and p = P. It will always be clear from the context how expressions containing symbols such as D[0], D[¬P], x[0], or x[¬P] are to be interpreted.) As usual, the ith component of a vector x is denoted by xi, and the entry in the ith row and jth column of a matrix B is denoted by Bi,j.
The restricted least-squares estimator of θ under the restriction θ[¬p] = 0, i.e., under θp+1 = ··· = θP = 0, will be denoted by
, 0 ≤ p ≤ P (in the case p = P the restriction being void). Note that
is given by the P × 1 vector
where the expressions
and
, respectively, are to be interpreted as the zero vector in RP and as the unrestricted least-squares estimator of θ. Given a parameter vector θ in RP, the order of θ (relative to the nested sequence of models Mp) is defined as
Hence, if θ is the true parameter vector, a model Mp is a correct model if and only if p ≥ p0(θ). We stress that p0(θ) is a property of a single parameter and hence needs to be distinguished from the notion of the order of the model Mp introduced earlier, which is a property of the set of parameters Mp.
A model-selection procedure is now nothing else than a data-driven (measurable) rule
that selects a value from
and thus selects a model from the list of candidate models
. In this section we shall consider as an important leading case a “general-to-specific” model-selection procedure based on a sequence of hypothesis tests. (Results for a larger class of model-selection procedures, including the AIC, are provided in Sect. 3.) This procedure is given as follows: The sequence of hypotheses H0p : p0(θ) < p is tested against the alternatives H1p : p0(θ) = p in decreasing order starting at p = P. If, for some
, H0p is the first hypothesis in the process that is rejected, we set
. If no rejection occurs until even
is not rejected, we set
. Each hypothesis in this sequence is tested by a kind of t-test where the error variance is always estimated from the overall model (but see the discussion following Theorem 3.1 in Sect. 3 for other choices of estimators of the error variance). More formally, we have
with
to ensure a well-defined
in the range
. For
, the critical values cp satisfy 0 < cp < ∞ and are independent of sample size (but see also Remark 4.2). The test statistics are given by
with the convention that T0 = 0. Furthermore,
denotes the nonnegative square root of the pth diagonal element of the matrix indicated, and
is given by
Note that under the hypothesis H0p the statistic Tp is t-distributed with n − P degrees of freedom for 0 < p ≤ P. It is also easy to see that the so-defined model-selection procedure
is conservative: The probability of selecting an incorrect model, i.e., the probability of the event
, converges to zero as the sample size increases. In contrast, the probability of the event
, for p satisfying
, converges to a positive limit; cf., e.g., Proposition 5.4 and equation (5.6) in Leeb (2006).
The post-model-selection estimator
can now be defined as follows: On the event
,
is given by the restricted least-squares estimator
, i.e.,
where 1(·) denotes the indicator function of the event shown in the argument.
2.1. The Distribution of the Post-Model-Selection Estimator
We now introduce the distribution function of a linear transformation of
and summarize some of its properties that will be needed in the subsequent development. To this end, let A be a nonstochastic k × P matrix of rank k, 1 ≤ k ≤ P, and consider the cdf
Here Pn,θ,σ(·) denotes the probability measure corresponding to a sample of size n from (5).
Depending on the choice of the matrix A, several important scenarios are covered by (8): The cdf of
is obtained by setting A equal to the P × P identity matrix IP. In the case
, the cdf of those components of
that correspond to the parameter of interest χ in (1) can be studied by setting A to the
matrix
as we then have
. Finally, if A ≠ 0 is a 1 × P vector, we obtain the distribution of a linear predictor based on the post-model-selection estimator. See the examples at the end of Section 2.2.2 for more discussion.
The cdf Gn,θ,σ and its properties have been analyzed in detail in Leeb and Pötscher (2003) and Leeb (2006). To be able to access these results we need some further notation. Note that on the event
the expression
equals
in view of (7). The expected value of the restricted least-squares estimator
will be denoted by ηn(p) and is given by the P × 1 vector
with the conventions that ηn(0) = (0,…,0)′ ∈ RP and that ηn(P) = θ. Furthermore, let Φn,p denote the cdf of
, i.e., the cdf of
times the restricted least-squares estimator based on model Mp centered at its mean. Hence, Φn,p is the cdf of a k-variate Gaussian random vector with mean zero and variance-covariance matrix σ2A[p](X[p]′X[p]/n)−1A[p]′ in the case p > 0, and it is the cdf of point mass at zero in Rk in the case p = 0. If p > 0 and if the matrix A[p] has full row rank k, then Φn,p has a density with respect to Lebesgue measure, and we shall denote this density by φn,p. We note that ηn(p) depends on θ and that Φn,p depends on σ (in case p > 0), although these dependencies are not shown explicitly in the notation.
For p > 0 we introduce
and
with ζn,p ≥ 0. Here Cn(p) = A[p](X[p]′X[p]/n)−1ep, where ep denotes the pth standard basis vector in Rp and B− denotes a generalized inverse of a matrix B. (Observe that ζn,p2 is invariant under the choice of the generalized inverse. The same is not necessarily true for bn,p but is true for bn,pz for all z in the column space of A[p]. Also note that (12), which follows, depends on bn,p only through bn,pz with z in the column space of A[p].) We observe that the vector of covariances between
and
is precisely given by σ2n−1Cn(p) (and hence does not depend on θ). Furthermore, observe that
and
are uncorrelated if and only if ζn,p2 = ξn,p2 if and only if bn,pz = 0 for all z in the column space of A[p]; cf. Lemma A.2 in Leeb (2005).
Finally, for a univariate Gaussian random variable
with zero mean and variance s2, s ≥ 0, we write Δs(a,b) for
, a ∈ R ∪ {−∞,∞}, b ∈ R. Note that Δs(·,·) is symmetric around zero in its first argument and that Δs(−∞,b) = Δs(∞,b) = 0 holds. In the case s = 0,
is to be interpreted as being equal to zero; hence a [map ] Δ0(a,b) reduces to the indicator function of the interval (−b,b).
We are now in a position to present the explicit formula for Gn,θ,σ(t) derived in Leeb (2006):
In the preceding display, Φn,p(dz) denotes integration with respect to the measure induced by the normal cdf Φn,p on Rk, and h denotes the density of
; i.e., h is the density of (n − P)−1/2 times the square root of a chi-square distributed random variable with n − P degrees of freedom. The finite-sample distribution of the post-model-selection estimator given in (12) is in general not normal; e.g., it can be bimodal; see Figure 2 in Leeb and Pötscher (2005a) and Figure 1 in Leeb (2006). (An exception where (12) is normal is the somewhat trivial case where Cn(p) = 0, i.e., where
and
are uncorrelated, for
; for more discussion, see Leeb, 2006, Sect. 3.3.) We note for later use that
where Gn,θ,σ(t|p) represents the cdf of
conditional on the event
and where πn,θ,σ(p) is the probability of this event under Pn,θ,σ. Note that πn,θ,σ(p) is always positive for
; cf. Leeb (2006, Sect. 3.2).
To describe the large-sample limit of Gn,θ,σ, some further notation is necessary. For p satisfying 0 < p ≤ P, partition the matrix Q = limn→∞ X′X/n as
where Q[p : p] is a p × p matrix. Let Φ∞,p be the cdf of a k-variate Gaussian random vector with mean zero and variance-covariance matrix σ2A[p]Q[p : p]−1A[p]′, 0 < p ≤ P, and let Φ∞,0 denote the cdf of point mass at zero in Rk. Note that Φ∞,p has a Lebesgue density if p > 0 and the matrix A[p] has full row rank k; in this case, we denote the Lebesgue density of Φ∞,p by φ∞,p. Finally, for p = 1,…,P, define
where C∞(p) = A[p]Q[p : p]−1ep, with ep denoting the pth standard basis vector in Rp; furthermore, take ζ∞,p and ξ∞,p as the nonnegative square roots of ζ∞,p2 and ξ∞,p2, respectively. As the notation suggests, Φ∞,p is the large-sample limit of Φn,p, and C∞(p), ξ∞,p2, and ζ∞,p2 are the limits of Cn(p), ξn,p2, and ζn,p2, respectively; moreover, bn,pz converges to b∞,pz for each z in the column space of A[p]. See Lemma A.2 in Leeb (2005).
The next result describes the large-sample limit of the cdf under local alternatives to θ and is taken from Leeb (2006, Cor. 5.6). Recall that the total variation distance between two cdfs G and G* on Rk is defined as ∥G − G*∥TV = supE|G(E) − G*(E)|, where the supremum is taken over all Borel sets E. Clearly, the relation |G(t) − G*(t)| ≤ ∥G − G*∥TV holds for all t ∈ Rk. Thus, if G and G* are close with respect to the total variation distance, then G(t) is close to G*(t), uniformly in t.
PROPOSITION 2.1. Suppose θ ∈ RP and γ ∈ RP and let σ(n) be a sequence of positive real numbers that converges to a (finite) limit σ > 0 as n → ∞. Then the cdf
converges to a limit G∞,θ,σ,γ in total variation, i.e.,
The large-sample limit cdf G∞,θ,σ,γ(t) is given by
where
. Here for 0 ≤ p ≤ P
with the convention that β(p) = −Aγ if p = 0 and that β(p) = (0,…,0)′ if p = P. Furthermore, we have set νp = γp + (Q[p : p]−1Q[p : ¬p]γ[¬p])p for p > 0. (Note that
for p ≥ p0(θ) and that
for p > p0(θ). Here ηn( p) is defined as in (9) but with
replacing θ.)
If p* > 0 and if the matrix A[p*] has full row rank k, then the Lebesgue density φ∞,p of Φ∞,p exists for all p ≥ p*, and hence the density of (15) exists and is given by
As with the finite-sample distribution, the limiting distribution of the post-model-selection estimator given in (15) is in general not normal. An exception is the case where C∞(p) = 0 for p > p*, in which case G∞,θ,σ,γ reduces to Φ∞,P; see Remark A.1 in Appendix A. If γ = 0, we write G∞,θ,σ(t) as shorthand for G∞,θ,σ,0(t) in the following discussion.
2.2. Estimators of the Finite-Sample Distribution
For the purpose of inference after model selection the finite-sample distribution of the post-model-selection estimator is an object of particular interest. As we have seen, it depends on unknown parameters in a complicated manner, and hence one will have to be satisfied with estimators of this cdf. As we shall see, it is not difficult to construct consistent estimators of Gn,θ,σ(t). However, despite this consistency result, we shall find in Section 2.2.2 that any estimator of Gn,θ,σ(t) typically performs unsatisfactorily in that the estimation error cannot become small uniformly over (subsets of) the parameter space even as sample size goes to infinity. In particular, no uniformly consistent estimators exist, not even locally.
2.2.1. Consistent Estimators.
We construct a consistent estimator of Gn,θ,σ(t) by commencing from the asymptotic distribution. Specializing to the case γ = 0 and σ(n) = σ in Proposition 2.1, the large-sample limit of Gn,θ,σ(t) is given by
with
. Note that G∞,θ,σ(t) depends on θ only through p*. Let
denote the cdf of a k-variate Gaussian random vector with mean zero and variance-covariance matrix
, 0 < p ≤ P; we also adopt the convention that
denotes the cdf of point mass at zero in Rk. (We use the same convention for
in the case
, which is a probability zero event.) An estimator
of Gn,θ,σ(t) is now defined as follows: We first employ an auxiliary procedure p that consistently estimates p0(θ) (e.g., p could be obtained from the Bayesian information criterion [BIC] or from a “general-to-specific” hypothesis testing procedure employing critical values that go to infinity but are o(n1/2) as n → ∞). The estimator
is now given by the expression in (16) but with p*, σ, b∞,p, ζ∞,p, ξ∞,p, and Φ∞,p replaced by
,
, bn,p, ζn,p, ξn,p, and
, respectively. A little reflection shows that
is again a cdf. We have the following consistency results.
PROPOSITION 2.2. The estimator
is consistent (in the total variation distance) for Gn,θ,σ and G∞,θ,σ. That is, for every δ > 0
for all θ ∈ RP and all σ > 0.
Although the estimator constructed previously on the basis of the formula for G∞,θ,σ is consistent, it can be expected to perform poorly in finite samples because convergence of Gn,θ,σ to G∞,θ,σ is typically not uniform in θ (cf. Appendix B) and because in the case where the true θ is “close” to Mp0(θ)−1 the auxiliary decision procedure p (although being consistent for p0(θ)) will then have difficulties making the correct decision in finite samples. In the next section we show that this poor performance is not particular to the estimator
constructed earlier but is a genuine feature of the estimation problem under consideration.
2.2.2. Performance Limits and Impossibility Results.
We now provide lower bounds for the performance of estimators of the cdf Gn,θ,σ(t) of the post-model-selection estimator
; i.e., we give lower bounds on the worst-case probability that the estimation error exceeds a certain threshold. These lower bounds are large, being 1 or ½, depending on the situation considered; furthermore, they remain lower bounds even if one restricts attention only to certain subsets of the parameter space that shrink at the rate n−1/2. In this sense the “impossibility” results are of a local nature. In particular, the lower bounds imply that no uniformly consistent estimator of the cdf Gn,θ,σ(t) exists, not even locally.
In the following discussion the asymptotic “correlation” between
and
as measured by C∞(p) = limn→∞ Cn( p) will play an important rôle. (Recall that
denotes the least-squares estimator of θ based on model Mp and that Aθ is the parameter vector of interest. Furthermore, the vector of covariances between
and
is given by σ2n−1Cn( p) with Cn( p) = A[ p](X [ p]′X [ p]/n)−1ep.) Note that C∞( p) equals A[ p]Q[ p : p]−1ep and hence does not depend on the unknown parameters θ or σ. In the important special case discussed in the Introduction, cf. (1), the matrix A equals the
matrix
, and the condition C∞( p) ≠ 0 reduces to the condition that the regressor corresponding to the pth column of (V : W) is asymptotically correlated with at least one of the regressors corresponding to the columns of V. See Example 1, which follows, for more discussion.
In the result to follow we shall consider performance limits for estimators of Gn,θ,σ(t) at a fixed value of the argument t. An estimator of Gn,θ,σ(t) is now nothing else than a real-valued random variable Γn = Γn(Y,X). For mnemonic reasons we shall, however, use the symbol
instead of Γn to denote an arbitrary estimator of Gn,θ,σ(t). This notation should not be taken as implying that the estimator is obtained by evaluating an estimated cdf at the argument t or that it is a priori constrained to lie between zero and one. We shall use this notational convention mutatis mutandis also in subsequent sections. Regarding the nonuniformity phenomenon, we then have a dichotomy that is described in the following two results.
THEOREM 2.3. Suppose that
and
are asymptotically correlated, i.e., C∞(q) ≠ 0, for some q satisfying
and let q* denote the largest q with this property. Then the following holds for every θ ∈ Mq*−1, every σ, 0 < σ < ∞, and every t ∈ Rk: There exist δ0 > 0 and ρ0, 0 < ρ0 < ∞, such that any estimator
of Gn,θ,σ(t) satisfying
for each δ > 0 (in particular, every estimator that is consistent) also satisfies
The constants δ0 and ρ0 may be chosen in such a way that they depend only on t, Q, A, σ, and the critical values cp for
. Moreover,
and
where the infima in (21) and (22) extend over all estimators
of Gn,θ,σ(t).
Remark 2.1. Assume that the conditions of the preceding theorem are satisfied. Suppose further that p[odot ],
, is such that either p[odot ] > 0 and some row of A[p[odot ]] equals zero or such that p[odot ] = 0. Then there exist δ0 > 0 and 0 < ρ0 < ∞ such that the left-hand side of (21) is not less than ½ for each θ ∈ Mp[odot ].
Theorem 2.3 a fortiori implies a corresponding “impossibility” result for estimation of the function Gn,θ,σ(·) when the estimation error is measured in the total variation distance or the sup-norm; cf. also Section 5.
It remains to consider the—quite exceptional—case where the assumption of Theorem 2.3 is not satisfied, i.e., where C∞(q) = 0, for all q in the range
. Under this “uncorrelatedness” condition it is indeed possible to construct an estimator of Gn,θ,σ that is uniformly consistent: It is not difficult to see that the asymptotic distribution corresponding to Gn,θ,σ reduces to Φ∞,P under this “uncorrelatedness” condition. Furthermore, the second half of Proposition B.1 in Appendix B shows that then the convergence of Gn,θ,σ to its large-sample limit is uniform with respect to θ, suggesting
, an estimated version of Φ∞,P, as an estimator for Gn,θ,σ.
PROPOSITION 2.4. Suppose that
and
are asymptotically uncorrelated, i.e., C∞(q) = 0, for all q satisfying
. Then
holds for each δ > 0 and for any constants σ* and σ* satisfying 0 < σ* ≤ σ* < ∞.
Inspection of the proof of Proposition 2.4 in Appendix C shows that (23) continues to hold if the estimator
is replaced by any of the estimators
for
. We also note that in the case
the assumption of Proposition 2.4 is never satisfied in view of Proposition 4.4 in Leeb and Pötscher (2006b), and hence Theorem 2.3 always applies in that case. Another consequence of Proposition 4.4 in Leeb and Pötscher (2006b) is that—under the “uncorrelatedness” assumption of Proposition 2.4—the restricted least-squares estimators
for
perform asymptotically as well as the unrestricted estimator
; this clearly shows that the case covered by Proposition 2.4 is highly exceptional.
In summary we see that it is typically impossible to construct an estimator of Gn,θ,σ(t) that performs reasonably well even asymptotically. Whenever Theorem 2.3 applies, any estimator of Gn,θ,σ(t) suffers from a nonuniformity defect that is caused by parameters belonging to shrinking “tubes” surrounding Mq*−1. For the sake of completeness, we remark that outside a “tube” of fixed positive radius that surrounds Mq*−1 the nonuniformity need not be present: Let q* be as in Theorem 2.3 and define the set U as U = {θ ∈ RP : |θq*| ≥ r} for some fixed r > 0. Then
is an estimator of Gn,θ,σ(t) that is uniformly consistent over θ ∈ U; more generally, it can be shown that then the relation (23) holds if the supremum over θ on the left-hand side is restricted to θ ∈ U.
We conclude this section by illustrating the preceding results with some important examples.
Example 1 (The distribution of
)
Consider the model given in (1) with χ representing the parameter of interest. Using the general notation of Section 2, this corresponds to the case
with A representing the
matrix
. Here
. The cdf Gn,θ,σ then represents the cdf of
. Assume first that limn→∞ V′W/n ≠ 0. Then C∞(q) ≠ 0 holds for some
. Consequently, the “impossibility” results for the estimation of Gn, θ, σ given in Theorem 2.3 always apply. Next assume that limn→∞ V′W/n = 0. Then C∞(q) = 0 for every
. In this case Proposition 2.4 applies, and a uniformly consistent estimator of Gn, θ, σ indeed exists. Summarizing we note that any estimator of Gn, θ, σ suffers from the nonuniformity phenomenon except in the special case where the columns of V and W are asymptotically orthogonal in the sense that limn→∞ V′W/n = 0. But this is precisely the situation where inclusion or exclusion of the regressors in W has no effect on the distribution of the estimator
asymptotically; hence it is not surprising that also the model-selection procedure does not have an effect on the estimation of the cdf of the post-model-selection estimator
. This observation may tempt one to enforce orthogonality between the columns of V and W by either replacing the columns of V by their residuals from the projection on the column space of W or vice versa. However, this is not helpful for the following reasons: In the first case one then in fact avoids model selection as all the restricted least-squares estimators for χ under consideration (and hence also the post-model-selection estimator
) in the reparameterized model coincide with the unrestricted least-squares estimator. In the second case the coefficients of the columns of V in the reparameterized model no longer coincide with the parameter of interest χ (and again are estimated by one and the same estimator regardless of inclusion/exclusion of columns of the transformed W matrix).
Example 2 (The distribution of
)
For A equal to IP, the cdf Gn,θ,σ is the cdf of
. Here,
reduces to
, and hence
and
are perfectly correlated for every
. Consequently, the “impossibility” result for estimation of Gn,θ,σ given in Theorem 2.3 applies. (In fact, the slightly stronger result mentioned in Remark 2.1 always applies here.) We therefore see that estimation of the distribution of the post-model-selection estimator of the entire parameter vector is always plagued by the nonuniformity phenomenon.
Example 3 (The distribution of a linear predictor)
Suppose A ≠ 0 is a 1 × P vector and one is interested in estimating the cdf Gn,θ,σ of the linear predictor
. Then Theorem 2.3 and the discussion following Proposition 2.4 show that the nonuniformity phenomenon always arises in this estimation problem in the case
. In the case
, the nonuniformity problem is generically also present, except in the degenerate case where C∞(q) = 0, for all q satisfying
(in which case Proposition 4.4 in Leeb and Pötscher, 2006b, shows that the least-squares predictors from all models Mp,
, perform asymptotically equally well).
3. EXTENSIONS TO OTHER MODEL-SELECTION PROCEDURES INCLUDING AIC
In this section we show that the “impossibility” result obtained in the previous section for a “general-to-specific” model-selection procedure carries over to a large class of model-selection procedures, including the widely used AIC. Again consider the linear regression model (5) with the same assumptions on the regressors and the errors as in Section 2. Let {0,1}P denote the set of all 0–1 sequences of length P. For each
let
denote the set {
for
} where
represents the ith component of
; i.e.,
describes a linear submodel with those parameters θi restricted to zero for which
. Now let
be a user-supplied subset of {0,1}P. We consider model selection procedures that select from the set
or equivalently from the set of models
. Note that there is now no assumption that the candidate models are nested (e.g., if
all possible submodels are candidates for selection). Also cases where the inclusion of a subset of regressors is undisputed on a priori grounds are obviously covered by this framework upon suitable choice of
.
We shall assume throughout this section that
contains
and also at least one element
satisfying
, where
represents the number of nonzero coordinates of
. Let
be an arbitrary model-selection procedure; i.e.,
is a random variable taking its values in
. We furthermore assume throughout this section that the model-selection procedure
satisfies the following mild condition: For every
with
there exists a positive finite constant c (possibly depending on
) such that for every
that has exactly P − 1 nonzero coordinates
holds for every 0 < σ < ∞. Here [utrif ] denotes the symmetric difference operator and
represents the usual t-statistic for testing the hypothesis
in the full model, where
denotes the index of the unique coordinate of
that equals zero.
The preceding condition is quite natural for the following reason: For
with exactly P − 1 nonzero coordinates, every reasonable model-selection procedure will—with probability approaching unity—decide only between
and
; it is then quite natural that this decision will be based (at least asymptotically) on the likelihood ratio between these two models, which in turn boils down to the t-statistic. As will be shown subsequently, condition (24) holds in particular for AIC-like procedures.
Let A be a nonstochastic k × P matrix of full row rank k, 1 ≤ k ≤ P, as in Section 2.1. We then consider the cdf
of a linear transformation of the post-model-selection estimator θ obtained from the model-selection procedure
, i.e.,
where the P × 1 vector
represents the restricted least-squares estimator obtained from model
, with the convention that
in the case
. We then obtain the following result for estimation of Kn,θ,σ(t) at a fixed value of the argument t that parallels the corresponding “impossibility” result in Theorem 2.3.
THEOREM 3.1. Let
satisfy
and let
denote the index of the unique coordinate of
that equals zero; furthermore, let c be the constant in (24) corresponding to
. Suppose that
and
are asymptotically correlated, i.e.,
, where
denotes the
th standard basis vector in RP. Then for every
that has exactly P − 1 nonzero coordinates, for every σ, 0 < σ < ∞, and for every t ∈ Rk the following holds: There exist δ0 > 0 and ρ0, 0 < ρ0 < ∞, such that any estimator
of Kn,θ,σ(t) satisfying
for each δ > 0 (in particular, every estimator that is consistent) also satisfies
The constants δ0 and ρ0 may be chosen in such a way that they depend only on t, Q, A, σ, and c. Moreover,
and
hold, where the infima in (28) and (29) extend over all estimators
of Kn,θ,σ(t).
The basic condition (24) on the model-selection procedure employed in the preceding result will certainly hold for any hypothesis testing procedure that (i) asymptotically selects only correct models, (ii) employs a likelihood ratio test (or an asymptotically equivalent test) for testing
versus smaller models (at least versus the models
with
as in condition (24)), and (iii) uses a critical value for the likelihood ratio test that converges to a finite positive constant. In particular, this applies to usual thresholding procedures and also to a variant of the “general-to-specific” procedure discussed in Section 2 where the error variance in the construction of the test statistic for hypothesis H0p is estimated from the fitted model Mp rather than from the overall model. We next verify condition (24) for AIC-like procedures. Let
denote the residual sum of squares from the regression employing model
and set
where
denotes a sequence of real numbers satisfying
and
is a positive real number. Of course,
if
. The model-selection procedure
is then defined as a minimizer (more precisely, as a measurable selection from the set of minimizers) of
over
. It is well known that the probability that
selects an incorrect model converges to zero. Hence, elementary calculations show that condition (24) is satisfied for
.
The analysis of post-model-selection estimators based on AIC-like model-selection procedures given in this section proceeded by bringing this case under the umbrella of the results obtained in Section 2. Verification of condition (24) is the key that enables this approach. A complete analysis of post-model-selection estimators based on AIC-like model-selection procedures, similar to the analysis in Section 2 for the “general-to-specific” model-selection procedure, is certainly possible but requires a direct and detailed analysis of the distribution of this post-model-selection estimator. (Even the mild condition that
contains
and also at least one element
satisfying
can then be relaxed in such an analysis.) We furthermore note that in the special case where
and an AIC-like model-selection procedure as in (30) is used, the results in the preceding theorem in fact hold for all
.
4. REMARKS AND EXTENSIONS
Remark 4.1. Although not emphasized in the notation, all results in the paper also hold if the elements of the design matrix X depend on sample size. Furthermore, all results are expressed solely in terms of the distributions Pn,θ,σ(·) of Y, and hence they also apply if the elements of Y depend on sample size, including the case where the random vectors Y are defined on different probability spaces for different sample sizes.
Remark 4.2. The model-selection procedure considered in Section 2 is based on a sequence of tests that use critical values cp that do not depend on sample size and satisfy 0 < cp < ∞ for
. If these critical values are allowed to depend on sample size such that they now satisfy cn,p → c∞,p as n → ∞ with 0 < c∞,p < ∞ for
, the results in Leeb and Pötscher (2003) and in Leeb (2005, 2006) continue to hold; see Remark 6.2(i) in Leeb and Pötscher (2003) and Remark 6.1(ii) in Leeb (2005). As a consequence, the results in the present paper can also be extended to this case quite easily.
Remark 4.3. The “impossibility” results given in Theorems 2.3 and 3.1 (and also the variants thereof discussed in Remarks 4.4–4.7, which follow) also hold for the class of all randomized estimators (with Pn,θ,σ* replacing Pn,θ,σ in those results, where Pn,θ,σ* denotes the distribution of the randomized sample). This follows immediately from Lemma 3.6 and the attending discussion in Leeb and Pötscher (2006a).
Remark 4.4.
(i) Let ψn,θ,σ denote the expectation of
under Pn,θ,σ and consider the cdf
. Results for the cdf Hn,θ,σ quite similar to the results for Gn,θ,σ obtained in the present paper can be established. A similar remark applies to the post-model-selection estimator θ considered in Section 3.
(ii) In Leeb (2006) the cdf Gn,θ,σ* also is analyzed, which corresponds to a (typically infeasible) model-selection procedure that makes use of knowledge of σ. Results completely analogous to the ones in the present paper can also be obtained for this cdf.
Remark 4.5. Results similar to the ones in Section 2.2.2 can also be obtained for estimation of the asymptotic cdf G∞,θ,σ(t) (or of the asymptotic cdfs corresponding to the variants discussed in the previous remark). Because these results are of limited interest, we omit them. In particular, note that an “impossibility” result for estimation of G∞,θ,σ(t) per se does not imply a corresponding “impossibility” result for estimation of Gn,θ,σ(t), because Gn,θ,σ(t) in general does not converge uniformly to G∞,θ,σ(t) over the relevant subsets in the parameter space; cf. Appendix B. (An analogous remark applies to the model-selection procedures considered in Section 3.)
Remark 4.6. Let πn,θ,σ(p) denote the model-selection probability
,
corresponding to the model-selection procedure discussed in Section 2. The finite-sample properties and the large-sample limit behavior of these quantities are thoroughly analyzed in Leeb (2006); cf. also Leeb and Pötscher (2003). For these model-selection probabilities the following results can be established, which we discuss here only briefly:
(i) The model-selection probabilities πn,θ,σ(p) converge to well-defined large-sample limits that we denote by π∞,θ,σ(p). In a similar way as in Proposition B.1 in Appendix B, the convergence of πn,θ,σ(p) to π∞,θ,σ(p) is nonuniform with respect to θ. (For the case
, this phenomenon is described in Corollary 5.6 of Leeb and Pötscher, 2003.)
(ii) The model-selection probabilities πn,θ,σ(p) can be estimated consistently. However, uniformly consistent estimation is again not possible. A similar remark applies to the large-sample limits π∞,θ,σ(p).
Remark 4.7. “Impossibility” results similar to the ones given in Theorems 2.3 and 3.1 for the cdf can also be obtained for other characteristics of the distribution of a linear function of a post-model-selection estimator such as the mean squared error or the bias of
.
5. ON THE SCOPE OF THE “IMPOSSIBILITY” RESULTS
The nonuniformity phenomenon described, e.g., in (20) of Theorem 2.3 is caused by a mechanism that can informally be described as follows. Under the assumptions of that theorem, one can find an appropriate θ and an appropriate sequence
exhibiting two crucial properties:
a. The probability measures Pn,ϑn,σ corresponding to ϑn are “close” to the measures Pn,θ,σ corresponding to θ, in the sense of contiguity. This entails that an estimator, which converges to some limit in probability under Pn,θ,σ, converges to the same limit also under Pn,ϑn,σ.
b. For given t, the estimands Gn,ϑn,σ(t) corresponding to ϑn are “far away” from the estimands Gn,θ,σ(t) corresponding to θ, in the sense that Gn,ϑn,σ(t) and Gn,θ,σ(t) converge to different limits; i.e., G∞,θ,σ,0(t) is different from G∞,θ,σ,γ(t).
In view of property a, an estimator
satisfying
in probability under Pn,θ,σ also satisfies
in probability under Pn,ϑn,σ. In view of property b, such an estimator
is hence “far away” from the estimand Gn,ϑn,σ(t) with high probability under Pn,ϑn,σ. In other words, an estimator that is close to Gn,θ,σ(t) under Pn,θ,σ must be far away from Gn,ϑn,σ(t) under Pn,ϑn,σ. Formalized and refined, this argument leads to (20) and, as a consequence, to the nonexistence of uniformly consistent estimators for Gn,θ,σ(t). (There are a number of technical details in this formalization process that need careful attention to obtain the results in their full strength as given in Sects. 2 and 3.)
The preceding informal argument that derives (20) from properties a and b can be refined and formalized in a much more general and abstract framework; see Section 3 of Leeb and Pötscher (2006a) and the references therein. That paper also provides a general framework for deriving results like (21) and (22) of Theorem 2.3. The mechanism leading to such lower bounds is similar to the one outlined earlier, where for some of the results the concept of contiguity of the probability measures involved has to be replaced by closeness of these measures in total variation distance. We use the results in Section 3 of Leeb and Pötscher (2006a) to formally convert properties a and b into the “impossibility” results of the present paper; cf. Appendix C.
Verifying the aforementioned property a in the context of the present paper is straightforward because we consider a Gaussian linear model. What is technically more challenging and requires some work is the verification of property b; this is done in Appendix A inter alia and rests on results of Leeb (2002, 2005, 2006).
Two important observations on properties a and b are in order: First, property a is typically satisfied in general parametric models under standard regularity conditions; e.g., it is satisfied whenever the model is locally asymptotically normal. Second, property b relies on limiting properties only and not on the finite-sample structure of the underlying statistical model. Now, the limit distributions of post-model-selection estimators in sufficiently regular parametric or semiparametric models are typically the same as the limiting distributions of the corresponding post-model-selection estimators in a Gaussian linear model (see, e.g., Sen, 1979; Pötscher, 1991; Nickl, 2003; Hjort and Claeskens, 2003). Hence, establishing property b for the Gaussian linear model then typically establishes the same result for a large class of general parametric or semiparametric models.1
Some care has to be taken here. In the Gaussian linear case the finite-sample cdfs converge at every value of the argument t, cf. Proposition 2.1. In a general parametric model, sometimes the asymptotic results (e.g., Hjort and Claeskens, 2003, Thm. 4.1) only guarantee weak convergence. Hence, to ensure convergence of the relevant cdfs at a given argument t as required in property b, additional considerations have to be employed. (This is, however, of no concern in the context discussed in the seventh paragraph in Sect. 5.)
The nonuniformity results in Theorem 2.3 are for (conservative) “general-to-specific” model selection from a nested family of models. Theorem 3.1 extends this to more general (conservative) model-selection procedures (including AIC and related procedures) and to more general families of models. The proof of Theorem 3.1 proceeds by reducing the problem to one where only two nested models are considered and then appealing to the results of Theorem 2.3. The condition on the model-selection procedures that enables this reduction is condition (24). It is apparent from the discussion in Section 3 that this condition is satisfied for many model-selection procedures. Furthermore, for the same reasons as given in the preceding paragraph, Theorem 3.1 also can easily be extended to sufficiently regular parametric and semiparametric models.
The “impossibility” results in the present paper are formulated for estimating Gn,θ,σ(t) for a given value of t. Suppose that we are now asking the question whether the cdf Gn,θ,σ(·) viewed as a function can be estimated uniformly consistently, where consistency is relative to a metric that metrizes weak convergence.2
Or, in fact, any metric with respect to which the relevant cdfs converge.
The present paper, including the discussion in this section, has focused on conservative model-selection procedures. However, the discussion should make it clear that similar “impossibility” results plague consistent model selection. Section 2.3 in Leeb and Pötscher (2006a) in fact gives such an “impossibility” result in a simple case.
We close with the following observations. Verification of property b, whether it is for G∞,θ,σ,0(t) and G∞,θ,σ,γ(t) (for given t) or for G∞,θ,σ,0(·) and G∞,θ,σ,γ(·), shows that the post-model-selection estimator
is a so-called nonregular estimator for Aθ: Consider an estimator
in a parametric model {Pn,β : β ∈ B} where the parameter space B is an open subset of euclidean space
. Suppose
, properly scaled and centered, has a limit distribution under local alternatives, in the sense that
converges in law under
to a limit distribution L∞,β,γ(·) for every γ. The estimator
is called regular if for every β the limit distribution L∞,β,γ(·) does not depend on γ; cf. van der Vaart (1998, Sect. 8.5). Suppose now that the model is, e.g., locally asymptotically normal (hence the contiguity property in property a is satisfied). The informal argument outlined at the beginning of this section (and which is formalized in Lemma 3.1 of Leeb and Pötscher, 2006a) then in fact shows that the cdf of any nonregular estimator cannot be estimated uniformly consistently (where consistency is relative to any metric that metrizes weak convergence).
6. CONCLUSIONS
Despite the fact that we have shown that consistent estimators for the distribution of a post-model-selection estimator can be constructed with relative ease, we have also demonstrated that no estimator of this distribution can have satisfactory performance (locally) uniformly in the parameter space, even asymptotically. In particular, no (locally) uniformly consistent estimator of this distribution exists. Hence, the answer to the question posed in the title has to be negative. The results in the present paper also cover the case of linear functions (e.g., predictors) of the post-model-selection estimator.
We stress here that resampling procedures such as, e.g., the bootstrap or subsampling do not solve the problem at all. First note that standard bootstrap techniques will typically not even provide consistent estimators of the finite-sample distribution of the post-model-selection estimator, as the bootstrap can be shown to stay random in the limit (Kulperger and Ahmed, 1992; Knight, 1999, Exp. 3).3
Brownstone (1990) claims the validity of a bootstrap procedure that is based on a conservative model-selection procedure in a linear regression model. Kilian (1998) makes a similar claim in the context of autoregressive models selected by a conservative model-selection procedure. Also the paper by Hansen (2003) contains such a claim for a stationary bootstrap procedure based on a conservative model-selection procedure. The preceding discussion intimates that these claims are at least unsubstantiated.
The “impossibility” results in Theorems 2.3 and 3.1 are derived in the framework of a normal linear regression model (and a fortiori these results continue to hold in any model that includes the normal linear regression model as a special case), but this is more a matter of convenience than anything else: As discussed in Section 5, similar results can be obtained in general statistical models allowing for nonlinearity or dependent data, e.g., as long as standard regularity conditions for maximum likelihood theory are satisfied.
The results in the present paper are derived for a large class of conservative model-selection procedures (i.e., procedures that select overparameterized models with positive probability asymptotically) including the AIC and typical “general-to-specific” hypothesis testing procedures. For consistent model-selection procedures—such as BIC or testing procedures with suitably diverging critical values cp (cf. Bauer, Pötscher, and Hackl, 1988)—the (pointwise) asymptotic distribution is always normal. (This is elementary; cf. Lemma 1 in Pötscher, 1991.) However, as discussed at length in Leeb and Pötscher (2005a), this asymptotic normality result paints a misleading picture of the finite-sample distribution, which can be far from normal, the convergence of the finite-sample distribution to the asymptotic normal distribution not being uniform. “Impossibility” results similar to the ones presented here can also be obtained for post-model-selection estimators based on consistent model-selection procedures. These will be discussed in detail elsewhere. For a simple special case such an “impossibility” result is given in Section 2.3 of Leeb and Pötscher (2006a).
The “impossibility” of estimating the distribution of the post-model-selection estimator does not per se preclude the possibility of conducting valid inference after model selection, a topic that deserves further study. However, it certainly makes this a more challenging task.
APPENDIX A: Auxiliary Lemmas
LEMMA A.1. Let Z be a random vector with values in Rk and let W be a univariate standard Gaussian random variable independent of Z. Furthermore, let C ∈ Rk and τ > 0. Then
is constant as a function of x ∈ R if and only if C = 0 or
for each x ∈ R.
Proof of Lemma A.1. Suppose C = 0 holds. Using independence of Z and W it is then easy to see that (A.1) reduces to
, which is constant in x. If
for every x ∈ R, then
, and hence (A.1) is again constant, namely, equal to zero.
To prove the converse, assume that (A.1) is constant in x ∈ R. Letting x → ∞, we see that (A.1) must be equal to
. This entails that
holds for every x ∈ R. Write F(x) as shorthand for
and let Φ(z) and φ(z) denote the cdf and density of W, respectively. Then the expression in the preceding display can be written as
We now further assume that C ≠ 0 and that F(x) ≠ 0 for at least one x ∈ R and show that this leads to a contradiction.
Consider first the case where all components of C are nonnegative. Because F is not identically zero, it is then, up to a scale factor, the cdf of a random variable on the real line. But then (A.2) cannot hold for all x ∈ R as shown in Example 7 in Leeb (2002) (cf. also equation (7) in that paper). The case where all components of C are nonpositive follows similarly by applying the preceding argument to F(−x) and upon observing that both Φ(x + τ) − Φ(x − τ) and φ(x) are symmetric around x = 0.
Finally, consider the case where C has at least one positive and one negative component. In this case clearly limx→−∞ F(x) = limx→∞ F(x) = 0 holds. Because F(x) is continuous in view of (A.2), we see that F(x) attains its (positive) maximum at some point x1 ∈ R. Now note that (A.2) with x1 replacing x can be written as
This immediately entails that F(x) = F(x1) for each x ∈ [x1 − τ,x1 + τ] (because F(x) is continuous and because of the definition of x1). Repeating this argument with x1 − τ replacing x1 and proceeding inductively, we obtain that F(x) = F(x1) for each x satisfying x ≤ x1 + τ, a contradiction with limx→−∞ F(x) = 0. █
LEMMA A.2. Let M and N be matrices of dimension k × p and k × q, respectively, such that the matrix (M : N) has rank k (k ≥ 1, p ≥ 1, q ≥ 1). Let t ∈ Rk and let V be a random vector with values in Rp whose distribution assigns positive mass to every (nonempty) open subset of Rp (e.g., it possesses an almost everywhere positive Lebesgue density). Set
, x ∈ Rq. If one of the rows of M consists of zeros only, then f is discontinuous at some point x0. More precisely, there exist x0 ∈ Rq, z ∈ Rq, and a constant c > 0, such that f(x0 + δz) ≥ c and f(x0 − δz) = 0 hold for every sufficiently small δ > 0.
Proof of Lemma A.2. The case where M is the zero matrix is trivial. Otherwise, let I0 denote the set of indices i, 1 ≤ i ≤ k, for which the ith row of M is zero. Let (M0 : N0) denote the matrix consisting of those rows of (M : N) whose index is in I0 and let (M1 : N1) denote the matrix consisting of the remaining rows of (M : N). Clearly, M0 is then the zero matrix. Furthermore, note that N0 has full row rank. Moreover, let t0 denote the vector consisting of those components of t whose index is in I0 and let t1 denote the vector containing the remaining components. With this notation, f(x) can be written as
.
For vectors μ ∈ Rp and η ∈ Rq to be specified in a moment, set t* = t + Mμ + Nη and let t0* and t1* be defined similarly to t0 and t1. Because the matrix (M : N) has full rank k, we can choose μ and η such that t0* = 0 and t1* > 0. Choose z ∈ Rq such that N0z > 0, which is possible because N0 has full row rank. Set x0 = η. Then for every
we have
Because t1* > 0, we can find a t1** such that 0 < t1** < t1* + εN1z holds for every ε with |ε| small enough. If now ε > 0 then
The right-hand side in the preceding display is positive because t1** > 0 and because the distribution of M1(V + μ) assigns positive mass to any neighborhood of the origin, because the same is true for the distribution of V + μ and because M1 maps neighborhoods of zero into neighborhoods of zero. Setting
, we have f(x0 + εz) ≥ c > 0 for each sufficiently small ε > 0. Furthermore, for ε < 0 we have f(x0 + εz) = 0, because
in view of N0z > 0. █
LEMMA A.3. Let Z be a random vector with values in Rp, p ≥ 1, with a distribution that is absolutely continuous with respect to Lebesgue measure on Rp. Let B be a k × p matrix, k ≥ 1. Then the cdf
of BZ is discontinuous at t ∈ Rk if and only if
and if for some i0, 1 ≤ i0 ≤ k, the i0th row of B and the i0th component of t are both zero, i.e., Bi0,· = (0,…,0) and ti0 = 0.
Proof of Lemma A.3. To establish sufficiency of the preceding condition, let
, ti0 = 0, and Bi0,· = (0,…,0) for some i0, 1 ≤ i0 ≤ k. Then, of course,
. For tn = t − n−1ei0, where ei0 denotes the i0th unit vector in Rk, we have
for every n. Consequently,
is discontinuous at t.
To establish necessity, we first show that the following holds: If tn ∈ Rk is a sequence converging to t ∈ Rk as n → ∞, then every accumulation point of the sequence
has the form
for some m, 0 ≤ m ≤ k, and for some permutation (i1,…,ik) of (1,…,k). This can be seen as follows: Let α be an accumulation point of
. Then we may find a subsequence such that
converges to α along this subsequence. From this subsequence we may even extract a further subsequence along which each component of the k × 1 vector tn converges to the corresponding component of t monotonously, i.e., either from above or from below. Without loss of generality, we may also assume that those components that converge from below are strictly increasing. The resulting subsequence will be denoted by nj in what follows. Assume that the components of tnj with indices i1,…,im converge from above and the components with indices im+1,…,ik converge from below. Now
where
denotes the distribution of BZ. The integrand in (A.4) now converges to
for all z ∈ Rk as nj → ∞. The right-hand side of (A.4) converges to the expression in (A.3) as nj → ∞ by the dominated convergence theorem, and the left-hand side of (A.4) converges to α by construction. This establishes the claim regarding (A.3).
Now suppose that
is discontinuous at t; i.e., there exists a sequence tn converging to t as n → ∞ such that
does not converge to
as n → ∞. From the sequence tn we can extract a subsequence tns along which
converges to a limit different from
as ns → ∞. As shown earlier, the limit has to be of the form (A.3), and m < k has to hold. Consequently, the limit of
is smaller than
. The difference of
and the limit of
is positive and because of (A.3) can be written as
We thus see that
for some j0 satisfying m + 1 ≤ j0 ≤ k. As Z is absolutely continuous with respect to Lebesgue measure on Rp, this can only happen if Bij0,· = (0,…,0) and tij0 = 0. █
LEMMA A.4. Suppose that
and
are asymptotically correlated, i.e., C∞(q) ≠ 0, for some q satisfying
, and let q* denote the largest q with this property. Moreover let θ ∈ Mq*−1, let σ satisfy 0 < σ < ∞, and let t ∈ Rk. Then G∞,θ,σ,γ(t) is nonconstant as a function of γ ∈ Mq*[setmn ]Mq*−1. More precisely, there exist δ0 > 0 and ρ0, 0 < ρ0 < ∞ such that
holds. The constants δ0 and ρ0 can be chosen in such a way that they depend only on t, Q, A, σ, and the critical values cp for
.
LEMMA A.5. Suppose that
and
are asymptotically correlated, i.e., C∞(q) ≠ 0, for some q satisfying
, and let q* denote the largest q with this property. Suppose further that for some p[odot ] satisfying
either p[odot ] = 0 holds or p[odot ] > 0 and A[p[odot ]] has a row of zeros. Then, for every θ ∈ Mp[odot ], every σ, 0 < σ < ∞, and every t ∈ Rk the quantity G∞,θ,σ,γ(t) is discontinuous as a function of γ ∈ Mq*. More precisely, for each
, there exist vectors β* and γ* in Mq* and constants δ* > 0 and ε* > 0 such that
holds for every θ satisfying
and for every ε with 0 < ε < ε*. The quantities δ*, ε*, β*, and γ* can be chosen in such a way that—besides t, Q, A, σ, and the critical values cp for
—they depend on θ only through
.
Before we prove the preceding lemmas, we provide a representation of G∞,θ,σ,γ(t) that will be useful in the following discussion: For 0 < p ≤ P define
, where C∞(r) has been defined after (13) and the random variables Wr are independent normally distributed with mean zero and variances σ2ξ∞,r2. For convenience, let Z0 denote the zero vector in Rk. Observe that Zp, p > 0, is normally distributed with mean zero and variance-covariance matrix σ2A[p]Q[p : p]−1A[p]′ because it has been shown in the proof of Proposition 4.4 in Leeb and Pötscher (2006b) that the asymptotic variance-covariance matrix σ2A[p]Q[p : p]−1A[p]′ of
can be expressed as
. Also the joint distribution of Zp and the set of variables Wr, 1 ≤ r ≤ P, is normal, with the covariance vector between Zp and Wr given by σ2C∞(r) in the case r ≤ p; otherwise Zp and Wr are independent. Define the constants νr = γr + (Q[r : r]−1Q[r : ¬r]γ[¬r])r for 0 < r ≤ P. It is now easy to see that for
the quantity β(p) defined in Proposition 2.1 equals
. (This is seen as follows: It was noted in Proposition 2.1 that
for p ≥ p0(θ), when ηn( p) is defined as in (9), but with
replacing θ. Using the representation (20) of Leeb (2005) and taking limits, the result follows if we observe that
for r > p ≥ p0(θ).) The cdf in (15) can now be written as
That the terms corresponding to p = p* in (A.7) and (15) agree is obvious. Furthermore, for each p > p* the terms under the product sign in (A.7) and (15) coincide by definition of the function Δs(a,b). It is also easy to see that the conditional distribution of Wp given Zp = z is Gaussian with mean b∞,pz and variance σ2ζ∞,p2. Consequently, the probability of the event {|Wp + νp| ≥ cpσξ∞,p} conditional on Zp = z is given by the integrand shown in (15). Because Zp has distribution Φ∞,p as noted before, it follows that (A.7) and (15) agree.
Remark A.1. If C∞(p) = 0 for all p > p*, then in view of the preceding discussion Zp* = Zp = ZP, and hence Φ∞,p* = Φ∞,p = Φ∞,P holds for all p > p*. Using the independence of Wr, r > p*, from Zp*, inspection of (A.7) shows that G∞,θ,σ,γ reduces to Φ∞,P; see also Leeb (2006, Rmk. 5.2).
Proof of Lemma A.4. From (A.7) (or (15)) it follows that the map γ [map ] G∞,θ,σ,γ(t) depends only on t, Q, A, σ, the critical values cp for
, and also on θ; however, the dependence on θ is only through
. It hence suffices to find, for each possible value of p* in the range
, constants 0 < ρ0 < ∞ and δ0 > 0 such that (A.5) is satisfied for some (and hence all) θ returning this particular value of
. For this in turn it is sufficient to show that for every θ ∈ Mq*−1 the quantity G∞,θ,σ,γ(t) is nonconstant as a function of γ ∈ Mq*[setmn ]Mq*−1.
Let θ ∈ Mq*−1 and assume that G∞,θ,σ,γ(t) is constant in γ ∈ Mq*[setmn ]Mq*−1. Observe that, by assumption, C∞(q*) is nonzero and C∞(p) = 0 for p > q*. For γ ∈ Mq*, we clearly have νq* = γq* and νr = 0 for r > q*. Letting γq*−1 → ∞ while γq* is held fixed, we see that νq*−1 → ∞; hence,
It follows that (A.7) converges to
By assumption, the expression in the preceding display is constant in γq* ∈ R[setmn ]{0}. Dropping the terms that do not depend on γq* and observing that
is never zero for
, we see that
has to be constant in γq* ∈ R[setmn ]{0}. We now show that the expression in (A.9) is in fact constant in γq* ∈ R: Observe first that
is positive and continuous in γq* ∈ R; also the probability
is continuous in γq* ∈ R because Wq*, being normal with mean zero and positive variance, is absolutely continuously distributed. Concerning the remaining term in (A.9), we note that Zq*−1 = MV where M = [ξ∞,1−2C∞(1),…,ξ∞,q*−1−2C∞(q*−1)] and V = (W1,…,Wq*−1)′. In the case where no row of M is identically zero, Lemma A.3 also shows that
is continuous in γq* ∈ R. Hence, in this case (A.9) is indeed constant for all γq* ∈ R. In the case where a row of M is identically zero, define N = ξ∞,q*−2C∞(q*) and rewrite the probability in question as
. Note that (M : N) has full row rank k, because
by definition of q* and because the latter matrix is nonsingular in view of rank A = k. Lemma A.2 then shows that there exists a γq*(0) ∈ R, z ∈ {−1,1}, and a constant c > 0 such that
and
holds for arbitrary small δ > 0. Observe that γq*(0) − δz and γq*(0) + δz are nonzero for sufficiently small δ > 0. But then (A.9)—being constant for γq* ∈ R[setmn ]{0}—gives the same value for γq* = γq*(0) − δz and γq* = γq*(0) + δz and all sufficiently small δ > 0. Letting δ go to zero in this equality and using the continuity properties for the second and third probability in (A.9) noted above we obtain that
which is impossible because c > 0 and
. Hence we have shown that (A.9) is indeed constant for all γq* ∈ R.
Now write Z, W, C, τ, and x for Zq*−1 − t, −Wq*/σξ∞,q*, σξ∞,q*−1C∞(q*), cq*, and γq*/σξ∞,q*, respectively. Upon observing that Zq* equals Zq*−1 + ξ∞,q*−2C∞(q*)Wq*, it is easy to see that (A.9) can be written as in (A.1). By our assumptions, this expression is constant in x = γq*/σξ∞,q* ∈ R. Lemma A.1 then entails that either C = 0 or
for each x ∈ R. Because C equals σξ∞,q*−1C∞(q*), it is nonzero by assumption. Hence,
must hold for every value of γq*. But the preceding probability is just the conditional probability that Zq* ≤ t given Wq* = −γq*. It follows that
equals zero also. By our assumption C∞(p) = 0 for p > q*, and hence Zq* = ZP. We thus obtain
, a contradiction with the fact that ZP is a Gaussian random variable on Rk with nonsingular variance-covariance matrix σ2AQ−1A′. █
Inspection of the preceding proof shows that it can be simplified if the claim of nonconstancy of G∞,θ,σ,γ(t) as a function of γ ∈ Mq*[setmn ]Mq*−1 in Lemma A.4 is weakened to nonconstancy for γ ∈ Mq*. The strong form of the lemma as given here is needed in the proof of Proposition B.1.
Proof of Lemma A.5. Let p[oplus ] be the largest index p,
, for which A[p] has a row of zeros and set p[oplus ] = 0 if no such index exists. We first show that p[oplus ] satisfies p[oplus ] < q*. Suppose p[oplus ] ≥ q* would hold. Because Zp[oplus ] is a Gaussian random vector with mean zero and variance-covariance matrix σ2A[p[oplus ]]Q[p[oplus ] : p[oplus ]]−1A[p[oplus ]]′, at least one component of Zp[oplus ] is equal to zero with probability one. However, Zp[oplus ] equals ZP because of p[oplus ] ≥ q* and the definition of q*. This leads to a contradiction because ZP has the nonsingular variance-covariance matrix σ2AQ−1A′. Without loss of generality, we may hence assume that p[odot ] = p[oplus ].
In view of the discussion in the first paragraph of the proof of Lemma A.4, it suffices to establish, for each possible value s in the range
, the result (A.6) for some θ with
. Now fix such an s and θ (and also, of course, t, Q, A, σ, and the critical values cp for
. Then (A.7) expresses the map γ [map ] G∞,θ,σ,γ(t) in terms of ν = (ν1,…,νP)′. It is easy to see that the correspondence between γ and ν is a linear bijection from
onto itself and that γ ∈ Mq* if and only if ν ∈ Mq*. It is hence sufficient to find a δ* > 0 and vectors ν and μ in Mq* such that (A.7) with ν + εμ in place of ν and (A.7) with ν − εμ in place of ν differ by at least δ* for sufficiently small ε > 0. Note that (A.7) is the sum of P − p* + 1 terms indexed by p = p*,…,P. We shall now show that ν and μ can be chosen in such a way that, when replacing ν with ν + εμ and ν − εμ, respectively, (i) the resulting terms in (A.7) corresponding to p = p[odot ] differ by some d > 0 and (ii) the difference of the other terms becomes arbitrarily small, provided that ε > 0 is sufficiently small.
Consider first the case where s = p* = p[odot ]. Using the shorthand notation
note that the p[odot ]th term in (A.7) is given by g(ν) multiplied by a product of positive probabilities that are continuous in ν. To prove property (i) it thus suffices to find a constant c > 0 and vectors ν and μ in Mq* such that |g(ν + εμ) − g(ν − εμ)| ≥ c holds for each sufficiently small ε > 0.
In the subcase p[odot ] = 0 choose c = 1 and set
and
observing that the matrix to be inverted is indeed nonsingular because—as discussed after Lemma A.5—it is up to a multiplicative factor σ2 identical to the variance-covariance matrix σ2AQ−1A′ of ZP. But then ν and μ satisfy
and
if we note that by the definition of q*
holds and that a similar relation holds with μ replacing ν. Because
, it is then obvious that g(ν + εμ) and g(ν − εμ) differ by one for each ε > 0.
In the other subcase p[odot ] > 0, define
,
, and V = (W1,…,Wp[odot ])′. It is then easy to see that g(ν) equals f((νp[odot ]+1,…,νq*)′), with f defined as in Lemma A.2, and that M has a row of zeros. Furthermore, the matrix (M : N) has rank k by the same argument as in the proof of Lemma A.4; cf. (A.10). By Lemma A.2, we thus obtain vectors x0 and z, and a c > 0 such that |f(x0 + εz) − f(x0 − εz)| ≥ c holds for each sufficiently small ε > 0. Setting (νp[odot ]+1,…,νq*)′ = x0, (μp[odot ]+1,…,μq*)′ = z, setting ν[¬q*] and μ[¬q*] each equal to zero, and setting ν[p[odot ]] and μ[p[odot ]] to arbitrary values, we see that g(ν ± εμ) has the desired properties.
To complete the proof in the case s = p* = p[odot ], we need to establish property (ii), for which it suffices to show that, for p > p[odot ], the pth term in (A.7) depends continuously on ν. For p > q*, the pth term does not depend on ν, because C∞(r) = 0 for r = q*,…,P. For p satisfying p[odot ] < p ≤ q*, it suffices to show that
is a continuous function. Suppose that (νp(m),…,νq*(m)) converges to (νp,…,νq*) as m → ∞. For arbitrary α > 0,
and
differ by less than α in each coordinate, provided that m is sufficiently large. This implies
observing that the latter probability is obviously continuous in the single variable νp (because Wp has an absolutely continuous distribution). Letting α decrease to zero we obtain lim supm→∞ h(νp(m),…, νq*(m)) ≤ h(νp,…, νq*). A similar argument establishes
, |Wp + νp| ≥ cpσξ∞, p). The proof of the continuity of h is then complete if we can show that
, |Wp + νp| ≥ cpσξ∞, p) is continuous or, equivalently, that
is a continuous cdf. Because p > p[odot ], the variance-covariance matrix σ2A[ p]Q[ p : p]−1A[ p]′ of Zp only has nonzero diagonal elements. Consequently, when representing Zp as B(W1,…,Wp)′, the matrix B cannot have rows that consist entirely of zeros. The conditional distribution of (W1,…,Wp)′ given the event {|Wp + νp| ≥ cpσξ∞, p} is clearly absolutely continuous with respect to p-dimensional Lebesgue measure. But then Lemma A.3 delivers the desired result.
The case where s = p* < p[odot ] is reduced to the previously discussed case as follows: It is easy to see that, for νp[odot ] → ∞, the expression in (A.7) converges to a limit uniformly with respect to all νp with p ≠ p[odot ]. Then observe that this limit is again of the form (A.7) but now with p[odot ] taking the rôle of p*. █
APPENDIX B: Nonuniformity of the Convergence of the Finite-Sample cdf to the Large-Sample Limit
PROPOSITION B.1.
(i) Suppose that
and
are asymptotically correlated, i.e., C∞(q) ≠ 0, for some q satisfying
, and let q* denote the largest q with this property. Then for every θ ∈ Mq*−1, every σ, 0 < σ < ∞, and every t ∈ Rk there exists a ρ, 0 < ρ < ∞, such that
holds. The constant ρ may be chosen in such a way that it depends only on t, Q, A, σ, and the critical values cp for
.
(ii) Suppose that
and
are asymptotically uncorrelated, i.e., C∞(q) = 0, for all q satisfying
. Then Gn,θ,σ converges to Φ∞,P in total variation uniformly in θ ∈ RP; more precisely
holds for any constants σ* and σ* satisfying 0 < σ* ≤ σ* < ∞.
Under the assumptions of Proposition B.1(i), we see that convergence of Gn,θ,σ(t) to G∞,θ,σ(t) is nonuniform over shrinking “tubes” around Mq*−1 that are contained in Mq*. (On the complement of a “tube” with a fixed positive radius, i.e., on the set U = {θ ∈ RP : |θq*| ≥ r} with fixed r > 0, convergence of Gn,θ,σ(t) to G∞,θ,σ(t) is in fact uniform [even with respect to the total variation distance], as can be shown. Note that for θ ∈ U the cdf G∞,θ,σ(t) reduces to the Gaussian cdf Φ∞,P(t), i.e., to the asymptotic distribution of the least-squares estimator based on the overall model; cf. Remark A.1.) A precursor to Proposition B.1(i) is Corollary 5.5 of Leeb and Pötscher (2003), which establishes (B.1) in the special case where
and where A is the P × P identity matrix. Proposition B.1(ii) describes an exceptional case where convergence is uniform. (In this case G∞,θ,σ reduces to the Gaussian cdf Φ∞,P for all θ and Φ∞,P = Φ∞,p,
, holds; cf. Remark A.1.) Recall that under the assumptions of part (ii) of Proposition B.1 we necessarily always have (i)
and (ii) rank
; cf. Proposition 4.4 in Leeb and Pötscher (2006b).
Proof of Proposition B.1. We first prove part (i). As noted at the beginning of the proof of Lemma A.4, the map γ [map ] G∞,θ,σ,γ(t) depends only on t, Q, A, σ, the critical values cp for
, and also on θ, but the dependence on θ is only through
. It hence suffices to find, for each possible value of p* in the range
, a constant 0 < ρ < ∞ such that (B.1) is satisfied for some (and hence all) θ returning this particular value of
. For this in turn it is sufficient to show that given such a θ we can find a γ ∈ Mq* such that
holds. Note that (B.2) is equivalent to
in light of Proposition 2.1. To establish (B.3), we proceed as follows: For each γ ∈ Mq* with γq* ≠ 0,
in (15) reduces to Φ∞,q*(t) as is easily seen from (A.7) because
, which in turn follows from p0(θ) < q* and γq* ≠ 0. Furthermore, Lemma A.4 entails that G∞,θ,σ,γ(t) is nonconstant in γ ∈ Mq*[setmn ]Mq*−1. But this shows that (B.3) must hold.
To prove part (ii), we write
where the conditional cdfs Gn,θ,σ(·|p) and the model-selection probabilities πn,θ,σ(p) have been introduced after (12). By the “uncorrelatedness” assumption, we have that Φ∞,p = Φ∞,P for all p in the range
; cf. Remark A.1. We hence obtain
Now for every p with
and for every ρ, 0 < ρ < ∞, we can write
In the case p = P, we use the convention that the second term in the maximum is absent and that the first supremum in the first term in the maximum extends over all of
. Letting first n and then ρ go to infinity in (B.5), we may apply Lemmas C.2 and C.3 in Leeb and Pötscher (2005b) to conclude that the left-hand side of (B.5), and hence the left-hand side of (B.4), goes to zero as n → ∞. █
APPENDIX C: Proofs for Sections 2.1–2.2.2
In the proofs that follow it will be convenient to show the dependence of Φn,p and Φ∞,p on σ in the notation. Thus, in the following discussion we shall write Φn,p,σ and Φ∞,p,σ, respectively, for the cdf of a k-variate Gaussian random vector with mean zero and variance-covariance matrix σ2A[p](X[p]′X[p]/n)−1A[p]′ and σ2A[p]Q[p : p]−1A[p]′, respectively. For convenience, let Φn,0,σ and Φ∞,0,σ denote the cdf of point mass at zero in Rk.
The following lemma is elementary to prove if we recall that bn,pz converges to b∞,pz as n → ∞ for every z ∈ ImA[p], the column space of A[p].
LEMMA C.1. Suppose
. Define Rn,p(z,σ) = 1 − Δσζn,p(bn,pz,cpσξn,p) and R∞,p(z,σ) = 1 − Δσζ∞,p(b∞,pz,cpσξ∞,p) for z ∈ ImA[p], 0 < σ < ∞. Let σ(n) converge to σ, 0 < σ < ∞. If ζ∞,p ≠ 0, then Rn,p(z,σ(n)) converges to R∞,p(z,σ) for every z ∈ ImA[p]; if ζ∞,p = 0, then convergence holds for every z ∈ ImA[p], except possibly for z ∈ ImA[p] satisfying |b∞,pz| = cpσξ∞,p. (This exceptional subset of ImA[p] has rank(A[p])-dimensional Lebesgue measure zero because cpσξ∞,p > 0.)
The following observation is useful in the proof of Proposition 2.2, which follows: Because the proposition depends on Y only through its distribution (cf. Remark 4.1), we may assume without loss of generality that the errors in (5) are given by ut = σεt, t ∈ N, with i.i.d. εt that are standard normal. In particular, all random variables involved are then defined on the same probability space.
Proof of Proposition 2.2. Because Pn,θ,σ(p = p0(θ)) → 1 by consistency, we may replace
by
in the formula for
for the remainder of the proof. Furthermore, because
in Pn,θ,σ probability, each subsequence contains a further subsequence along which
almost surely (with respect to the probability measure on the common probability space supporting all random variables involved), and we restrict ourselves to such a further subsequence for the moment. In particular, we write
for the event that
converges to σ along the subsequence under consideration; clearly, the event
has probability one. Also note that we can assume without loss of generality that
holds on this event (at least from some data-dependent n onward), because σ > 0 holds. But then obviously
converges to
, and
converges to Φ∞,p*,σ(t) in total variation by Lemma A.3 of Leeb (2005) in the case p* > 0 and trivially so in the case p* = 0. This proves that the first term in the formula for
converges to the corresponding term in the formula for G∞,θ,σ in total variation.
Next, consider the term in
that carries the index p > p*. By Lemma A.3 in Leeb (2005),
has a density
with respect to Φ∞,p,σ, which converges to one except on a set that has measure zero under Φ∞,p,σ. By Scheffé's lemma (Billingsley, 1995, Thm. 16.12),
converges to one also in the L1(Φ∞,p,σ) sense. By Lemma C.1,
converges to R∞,p(z,σ) except possibly on a set that has measure zero under Φ∞,p,σ. (Recall that Φ∞,p,σ is concentrated on ImA[p] and is not degenerate there.) Observing that
is uniformly bounded by one, we obtain that
also converges to R∞,p(z,σ) in the L1(Φ∞,p,σ) sense. Hence,
where ∥·∥ denotes the L1(Φ∞,p,σ) norm. Because
obviously converges to
, the relation (C.1) shows that the term in
carrying the index p converges to the corresponding term in G∞,θ,σ in the total variation sense. This proves (18) along the subsequence under consideration. However, because any subsequence contains such a further subsequence, this establishes (18). Because Gn,θ,σ converges to G∞,θ,σ in total variation by Proposition 2.1, the claim in (17) also follows. █
Before we prove the main result we observe that the total variation distance between Pn,θ,σ and Pn,ϑ,σ satisfies ∥Pn,θ,σ − Pn,ϑ,σ∥TV ≤ 2Φ(∥θ − ϑ∥λmax1/2(X′X)/2σ) − 1; furthermore, if θ(n) and ϑ(n) satisfy ∥θ(n) − ϑ(n)∥ = O(n−1/2), the sequence Pn,ϑ(n),σ is contiguous with respect to the sequence Pn,θ(n),σ (and vice versa). This follows exactly in the same way as Lemma A.1 in Leeb and Pötscher (2006a).
Proof of Theorem 2.3. We first prove (20) and (21). For this purpose we make use of Lemma 3.1 in Leeb and Pötscher (2006a) with α = θ ∈ Mq*−1, B = Mq*, Bn = {ϑ ∈ Mq* : ∥ϑ − θ∥ < ρ0n−1/2}, β = ϑ, φn(β) = Gn,ϑ,σ(t),
, where ρ0, 0 < ρ0 < ∞, will be chosen shortly (and σ is held fixed). The contiguity assumption of this lemma (and also the mutual contiguity assumption used in the corrigendum to Leeb and Pötscher, 2006a) is satisfied in view of the preparatory remark preceding this proof. It hence remains only to show that there exists a value of ρ0, 0 < ρ0 < ∞, such that δ* in Lemma 3.1 of Leeb and Pötscher (2006a) (which represents the limit inferior of the oscillation of φn(·) over Bn) is positive. Applying Lemma 3.5(i) of Leeb and Pötscher (2006a) with ζn = ρ0n−1/2 and the set G0 equal to the set G, it remains, in light of Proposition 2.1, to show that there exists a ρ0, 0 < ρ0 < ∞, such that G∞,θ,σ,γ(t) as a function of γ is nonconstant on the set {γ ∈ Mq* : ∥γ∥ < ρ0}. In view of Lemma 3.1 of Leeb and Pötscher (2006a), the corresponding δ0 can then be chosen as any positive number less than one-half of the oscillation of G∞,θ,σ,γ(t) over this set. That such a ρ0 indeed exists now follows from Lemma A.4 in Appendix A, where it is also shown that ρ0 and δ0 can be chosen such that they depend only on t, Q, A, σ, and cp for
. This completes the proof of (20) and (21).
To prove (22) we use Corollary 3.4 in Leeb and Pötscher (2006a) with the same identification of notation as before, with ζn = ρ0n−1/2 and with V = Mq* (viewed as a vector space isomorphic to Rq*). The asymptotic uniform equicontinuity condition in that corollary is then satisfied in view of ∥Pn,θ,σ − Pn,ϑ,σ∥TV ≤ 2Φ(∥θ − ϑ∥λmax1/2(X′X)/2σ) − 1. Given that the positivity of δ* has already been established in the previous paragraph, applying Corollary 3.4(i) of Leeb and Pötscher (2006a) then establishes (22). █
Proof of Remark 2.1. The proof is similar to the proof of (22) just given, except for using Corollary 3.4(ii) and Lemma 3.5(ii) in Leeb and Pötscher (2006a) instead of Corollary 3.4(i) and Lemma 3.5(i) from that paper. Furthermore, Lemma A.5 in Appendix A instead of Lemma A.4 is used. █
Proof of Proposition 2.4. In view of Proposition B.1(ii) and the fact that
holds (in the case
), it suffices to show that
hold for each δ > 0 and for any constants σ* and σ* satisfying 0 < σ* ≤ σ* < ∞. (Note that the probability in (C.3) in fact does not depend on θ.) But this has already been established in the proof of Proposition 4.3 of Leeb and Pötscher (2005b). █
APPENDIX D: Proof for Section 3
Proof of Theorem 3.1. After rearranging the elements of θ (and hence the regressors) if necessary and then correspondingly rearranging the rows of the matrix A, we may assume without loss of generality that
and hence that
. That is,
and
. Furthermore, note that after this arrangement C∞(P) ≠ 0. Let
be the model-selection procedure introduced in Section 2 with
, cP = c, and
. Let
be the corresponding post-model-selection estimator and let Gn,θ,σ(t) be as defined in Section 2.1. Condition (24) now implies that for every θ ∈ MP−1 that has exactly P − 1 nonzero coordinates
holds for every 0 < σ < ∞. Because the sequences Pn,ϑ(n),σ and Pn,θ,σ are contiguous for ϑ(n) satisfying ∥θ − ϑ(n)∥ = O(n−1/2) as remarked prior to the proof of Theorem 2.3 in Appendix C, it follows that condition (D.1) continues to hold with Pn,ϑ(n),σ replacing Pn,θ,σ. This implies that for every sequence of positive real numbers sn with sn = O(n−1/2), for every σ, 0 < σ < ∞, and for every θ ∈ MP−1 that has exactly P − 1 nonzero coordinates
holds as n → ∞. From (D.2) we conclude that the limit of
(with respect to total variation distance) exists and coincides with G∞,θ,σ,γ. Repeating the proof of Theorem 2.3 with q* = P, with Kn,ϑ,σ(t) replacing Gn,ϑ,σ(t) and with
replacing
gives the desired result. █
