3.1. A Valid Edgeworth Expansion for the DLE and for the ELR Statistic

Let us introduce some additional notation and assumptions. Let ∥·∥ denote the Euclidean norm; assume that for all λ ∈ Γ_τ and some α,β ∈ Z₊

where, for

denote the vector containing all the different DL derivatives up to lth order evaluated at λ^r = 0. Conditions DL2 and DL3 are sufficient for proving the validity of Edgeworth expansions; notice that DL2 is satisfied if E∥ f (z,θ₀)∥^αβ < ∞.

Let

denote the formal Edgeworth expansion (Bhattacharya and Ghosh, 1978, eq. (1.14)) of the distribution of

, where φ_q,κrs(·) denote the q-dimensional normal density with mean 0 and covariance κ_rs = E [f_r(z,θ₀) f_s(z,θ₀)] . The following theorem shows that the DLE admits a valid Edgeworth expansion in the sense of Bhattacharya and Ghosh (1978).

THEOREM 1. Assume that GS1, EL, and DL2 with α = 4, β = 3 hold. Then, for some constant C > 0,

Furthermore, assume that the Cramér condition DL3 holds with l = 3. Then,

where

is the class of Borel subsets in

satisfying

Before obtaining an Edgeworth expansion for the ELR under a sequence of local alternatives, we first discuss what type of local alternatives is consistent with the DL approach to EL inference. A simple modification of the argument of Mykland (1995, pp. 410–411) shows that the sequence of local alternatives induced by the DL is of the form

for some finite nonrandom vector

. This type of local alternatives assumes implicitly that the DL under the alternative hypothesis belongs to the same parametric subfamily specified under the null hypothesis (i.e., the product integral of λ^rf_r(z_i,θ₀)), ruling out effectively the possibility of considering local alternatives where both λ and θ are allowed to depend on a sequence of local alternatives. This type of local analysis is introduced by Chesher and Smith (1997) as a generalization of the standard likelihood based local analysis (see, e.g., Peers, 1971) and requires the specification of an augmented local alternative of the form H_n^a : φ_n = [θ_n λ_n], where θ_n = θ₀ + θ/n^1/2 and λ_n = λ(θ/n^1/2) = λ/n^1/2 for some finite nonrandom vector θ. This general framework, however, might break down the DL construction for it requires the consideration of alternatives that might not necessarily lie in the (log)DL subfamily defined in (4). Bearing this in mind, it should be clear why the only sequence of local alternatives for the ELR consistent with the DL construction takes the form

.

For simplicity we reparameterize H_n as

where κ_rs^1/2 is the square root of the symmetric positive definite matrix κ_rs, and we define the scaled arrays (moments)

so that κ_rs = δ^rs—the Kronecker delta, i.e., δ^rr = 1, δ^rs = 0 for r ≠ s. Let G_q,τ(·) denote the distribution function of a noncentral chi square random variable with q degrees of freedom and noncentrality parameter τ. The following theorem shows that the ELR test under a sequence of local alternatives H_n admits a valid Edgeworth expansion in the sense of Chandra and Ghosh (1980).

THEOREM 2. Suppose that the assumptions of Theorem 1 hold. Then, uniformly in c ∈ [c₀,∞) for some c₀ ≥ 0 (c₀ = 0 if r > 1),

uniformly over compact subsets of

, where

and

where

and, e.g.,

is the sum over the three different ways to partition a set of four indices into two subsets of two indices each.

The preceding expansion is useful for two reasons: first, it can be used to compute the approximate local power function of the test H₀ : λ = 0 versus alternatives of the form

(i.e., Pr(W(θ₀) > c_α|H_n) where c_α = Pr(χ_q² ≥ c_α) = α). Second, it can be used to obtain a valid Edgeworth expansion of the ELR statistic under H₀ with remainder of order o(n⁻¹). To improve the latter to the order O(n⁻²) we need to strengthen DL2 and DL3 as in the following corollary; let g_q(·) denote the density of a chi square random variable with q degrees of freedom and let B = B^rr.

COROLLARY 3. Assume that GS1, EL, and DL2 with α = 6 and β = 5 and DL3 with l = 4 hold. Then, for some c₀ ≥ 0

Let W^B(θ₀) = W(θ₀)/(1 + B/(nq)) and c_α = Pr(χ_q² ≥ c_α) = α. Then

Expansion (12) shows clearly that the Bartlett factor B is the main error term for the ELR statistic; moreover because B enters linearly in the expansion it is clear that scaling W(θ₀) by the same factor improves the coverage error to the order O(n⁻²), as in (13). This remarkable property of the ELR statistic is a direct consequence of the connection relating dual and empirical likelihood in the case of independent observations: because the ELR can be interpreted as a DLR for λ, and a DLR inherits properties of an ordinary parametric likelihood ratio statistic via the Bartlett-type identities (5), it is perhaps not surprising that the ELR shares the Bartlett-correctability property of an ordinary likelihood ratio statistic. In particular, as shown in the proof of Theorem 2 in the Appendix, the Bartlett-type identities (5) imply that, for DL2 with α = 5 and β = 4 and DL3 with l = 3, the signed square root W_r(θ₀) of the ELR (i.e., an

-valued random vector such that W_r(θ₀)W_r(θ₀) = W(θ₀) + O_p(n^−3/2)) is asymptotically

where c_r and c_r,s are constants defined as k_r² and k_r,s³ (with

) in (A.20) in the Appendix. This result was originally proved by DiCiccio and Romano (1989), using a different technique involving some lengthy algebra, and shows that W_r(θ₀) can be mean and variance corrected so that the resulting adjusted statistic is N(0,δ^rs) + O(n^−3/2), as is typically the case for the signed square root of ordinary parametric likelihood ratios. The existence of a Bartlett correction for the ELR then follows from this result combined with an Edgeworth expansion argument. The latter shows in fact that the density of W_r(θ₀)W_r(θ₀) + O_p(n^−3/2) is proportional to exp(−x²/2)(x²)^q/2−1[1 + ψ(x²)/n] + O(n⁻²) where ψ(x²) is a linear function in x², so that scaling W(θ₀) by a factor 1 + B/(nq) with B = c_r c_r + c_rr eliminates the coefficient of n⁻¹ in the expansion of W^B(θ₀) yielding (13).

3.2. Estimation of the Bartlett Correction

The Bartlett correction for GS

depends on the third and fourth multivariate moments of f_r(z,θ₀). The computation of the moments involved in the threefold summation in (15) takes O(nq³) computing time, so unless q is very large the computational cost of the Bartlett correction is not very high. Recall that B is based on the scaled moments κ_r1…rk = E [(κ^{r₁ s₁})^1/2f_s1(z,θ₀)…(κ^{r_k s_k})^1/2f_sk(z,θ₀)] and suppose that

; then a simple estimator of (15) can be based on

where

. The following theorem shows that

is consistent for (15) and that replacing B with

in Corollary 3 does not alter the order of magnitude of the approximation error (13). Let

denote a neighborhood of θ₀ and make the following assumption.

THEOREM 4. Assume that GS2 holds. Then,

. Furthermore, suppose that the assumptions of Corollary 3 hold and that

. Then, for some c₀ ≥ 0

Remark. The assumption of i.i.d. sampling can be relaxed by considering (z_in)_i≤n,n≥1 as a triangular array of

-valued random vectors as in Owen (1991); i.e., for each n, f_r(z_1n,θ),…,f_r(z_nn,θ) are independent but not identically distributed

valued random vectors. The results presented in Sections 3.1 and 3.2 are still valid by replacing some of the previous regularity conditions with the following uniform (in n) version.

GS1. (i) E [f_r(z_in,θ₀)] = 0 for a unique θ₀ ∈ Θ for all n, (ii) lim inf_n→∞ ζ_n /n > 0, where ζ_n is the smallest eigenvalue of [sum ] E [f_r(z_in,θ₀) f_s(z_in,θ₀)],

GS2.

EL. Pr(0 ∈ ch{f_r(z_1n,θ₀),…,f_r(z_nn,θ₀)}) = 1 as n → ∞,

DL2.

DL3. lim sup_n→∞ sup_∥t∥≥δ|E [exp(ıt^rZ_{nr 0}^l)]| < 1 for all δ > 0 and some l ∈ Z₊,

where W_nλ(·) and Z_{nr 0}^l are, respectively, the log DL defined in (4) and the

-valued vector of DL derivatives evaluated at λ = 0 for the triangular array f_r(z_in,θ₀) n ≥ 1. In particular GS1, GS2(i), EL, and DL2(iii) with β = 3 and Z_{nr 0}^l = Z_{nr 0} for l = 1 imply Theorem 2 of Owen (1991), whereas DL2(i)–(ii) and DL3 imply that we can use Theorem 20.6 of Bhattacharya and Rao (1976) together with Theorem 3.2 and Remarks 3.3 and 3.4 in Skovgaard (1981) to justify the validity of the Edgeworth expansions in Theorems 1 and 2. The results shown in Section 3.3, which follows, are also valid by replacing BDL with uniform versions similar to those just shown.

3.3. A Valid Edgeworth Expansion for the Bootstrap DLE and the Bootstrap ELR Statistic

In this section we consider the higher order properties of bootstrap dual/EL inference. Let χ_n* = (z₁*,…,z_n*) denote a bootstrap sample, i.e., a resample drawn independently and uniformly from the observed sample χ_n with the property that Pr(z_i* = z_j|χ_n) = 1/n for 1 ≤ i,j ≤ n, and let f_r(z_i*,θ₀) denote the corresponding bootstrap GS.

We first consider bootstrapping the DLE

. Let

denote the bootstrap (log) DL where, in analogy to Section 2, the dual parameter λ*^r is free-varying. The bootstrap DLE (BDLE)

solves 0 = ∂W_λ**(θ₀)/∂λ^r. Assume that for some α,β ∈ Z₊ and γ > 0

BDL.

and let Pr* denote the bootstrap probability conditional on χ_n. The following theorem establishes the higher order equivalence between the original and BDLE.

THEOREM 5. Assume that GS1, EL, and BDL with α = 6, β = 3, and some γ > 0 hold. Then, for some constant C > 0

except if χ_n is contained in a set with probability o(n⁻¹). Furthermore, assume that the Cramér condition DL3 holds with l = 3. Then, for every class

of Borel subsets in

satisfying (8)

except if χ_n is contained in a set of probability o(n⁻¹).

Notice that another way to express (18) (and analogously (19) and also (21), which follows) is

We now consider bootstrapping the ELR. It is important to note that the validity of the bootstrap in the context of EL inference depends crucially on the validity of the bootstrap moment condition E*[f_r(z_i*,θ)] = 0 at θ = θ₀, where E* is the bootstrap expectation under Pr*. A straightforward method to achieve this is to consider the centered bootstrap GS f_r*(z_i,θ) = f_r(z_i*,θ) − E*[f_r(z_i*,θ₀)] , which mimics the original GS as defined in (1) because by construction the bootstrap moment condition E*[f_r*(z_i,θ₀)] = 0. Alternatively, we can consider the so-called biased bootstrap GS f_r(z_i^[dagger],θ), where the z_i^[dagger]'s are resamples drawn independently from the original sample χ_n with the property that Pr(z_i^[dagger] = z_j|χ_n) = p_j for 1 ≤ i,j ≤ n, and the p_js are the estimated EL probabilities. This resampling procedure was proposed, originally in 1992, by Brown and Newey (2001) in the context of bootstrapping for generalized method of moments (GMM); the term biased used here is borrowed from Hall and Presnell (1999). Notice that f_r(z_i^[dagger],θ) does not need to be centered, because by definition E^[dagger][f_r(z_i^[dagger],θ₀)] = 0 where E^[dagger] is the expectation under Pr^[dagger]. Thus both methods lead to unbiased (conditional on χ_n) bootstrap GS at θ = θ₀. The latter fact is the key to the validity of bootstrap ELR because it implies that the distribution of the bootstrap ELR (BELR) resembles the null distribution of the original ELR regardless of whether the null hypothesis is true or not. More important, at least in the context of this paper, the results of Brown and Newey (2001) (see also Horowitz, 2001) imply that biased bootstrap leads to the same theoretical improvements to the finite-sample distribution of the ELR statistic as those obtained using the standard (uniform) bootstrap (cf. Corollary 7). Bearing this in mind, in the remaining part of this section we consider bootstrap EL inference based on the centered bootstrap GS f_r*(z_i,θ).

An immediate consequence of centering is that the bootstrap equivalent of assumption EL holds, i.e.,

The latter implies that the bootstrap analogue of the profile ELR function defined in (2),

admits a unique solution at θ = θ₀ and that such a solution can be obtained by the same Lagrangian argument of Section 2. Let

denote the resulting log BELR. Solving (20) and calculating

for b = 1,…,B bootstrap samples χ_n* yields an estimator of the distribution of the ELR that can be used to form critical values for test statistics (and/or confidence regions) in the usual way.

As in Section 2, the BELR

can be interpreted as a bootstrap DLR test for the dual parameter λ*^r, i.e.,

where

is the bootstrap DL based on the centered bootstrap GS f_r*(z_i,θ₀), W₀*(θ₀) is the “restricted” bootstrap DL evaluated at λ* = 0, and

is the BDLE that solves 0 = ∂W_λ**(θ₀)/∂λ^r using centered bootstrap GS f_r*(z_i,θ₀). Note that because of the centering

converges (in bootstrap probability) to 0 and not to the DLE

as in Theorem 5. Because W₀*(θ₀) = 0, let

; the following theorem shows that the BELR statistic approximates the distribution of the ELR statistic under the null hypothesis up to the order o(n⁻¹).

THEOREM 6. Suppose that the assumptions of Theorem 2 with α = 6 hold. Then for some c₀ ≥ 0

except if χ_n is contained in a set with probability o(n⁻¹).

Notice that the moment assumption of Theorem 6 (which is slightly stronger than the one assumed in Theorem 2) is necessary to ensure that the BELR test is accurate up to o(n⁻¹); i.e., the BELR test has rejection probabilities that are correct up to the same order. To see this note that (21) is equivalent to

thus using the same arguments as Andrews (2002, pp. 149–150), it follows that for c_α* = inf{c ∈ [c₀,∞) : Pr*(W*(θ₀) ≥ c) ≥ α}

which implies that, for large n, |1 − α − Pr(W(θ₀) ≤ c_α*)| ≤ n⁻¹, or equivalently

Thus, with the bootstrap calibration the level of ELR test is correct through the order O(n⁻¹). In contrast, as shown in Theorem 2, the ELR test with χ² calibration is accurate up to O(n⁻¹). On the other hand, using a slightly refined argument based on generalized Cornish–Fisher expansions (Hill and Davies, 1968), the approximation error in (22) can be improved to O(n⁻²), as next corollary shows.

COROLLARY 7. Suppose that the assumptions of Corollary 3 with α = 8 hold. Then,

Corollary 7 shows that bootstrapping the ELR delivers the same type of higher order accurate inference implied by the existence of a Bartlett correction. The same phenomenon was noted by Beran (1988) in the context of bootstrap likelihood ratio inference. Unless dim[f (z,θ)] is very large, it is clear that the Bartlett-corrected ELR is computationally more convenient than the BELR. On the other hand, as already mentioned in the Introduction and further illustrated in Section 4 by some simulations, it seems that the BELR has better finite-sample accuracy than the Bartlett-corrected ELR. Moreover, the higher order refinements delivered by the bootstrap approach to ELR inference do not depend in general on Bartlett-type identities. This should be important when considering the ELR statistic with nuisance parameters. To see why, let η be a finite-dimensional vector of nuisance parameters and

denote the resulting profile ELR (PELR) where

. As shown by Lazar and Mykland (1999) the fact that the Bartlett-type identities (5) do not hold for η implies that in general

is not Bartlett-correctable. Suppose however that

admits a valid asymptotic expansion in the sense of Chandra and Ghosh (1979) and that

. Then, at least theoretically, generalized Cornish–Fisher expansions can be brought to bear to show that

with adjusted critical values is accurate to an order O(n⁻²), independent of whether the Bartlett-type identities hold or not. Furthermore, using arguments similar to those used in the proofs of Theorem 6 and Corollary 7 in the Appendix it might be possible to obtain higher order refinements to the distribution of the PELR using critical values based on the bootstrap PELR

.

Example I. Nonlinear regressions

We consider nonlinear regression models y = g(x^r,θ₀^r) + ε for a given (differentiable) parametric function g(·) and E(ε|x^r) = 0. Following Newey (1990), the GS is given by

which corresponds to an IV based estimating function. Notice that as long as the regression function is correctly specified, we do not need to model explicitly the (conditional) variance σ²(x) of ε; clearly if we knew the functional form of σ²(x) we could use an IV based GS with instruments equal to ∂g(x^r,θ^r)/∂θ^s and σ²(x)⁻¹ to gain efficiency, but we stress here our view of the EL method as a technique to improve the accuracy of inference.

For this class of models the bootstrap calibration is based on the resamples (x_i^r*,y_i*), and the centered bootstrap GS is

where ε_i* = y_i* − g(x_i*^r,θ₀^r).

In the Monte Carlo study, we specify g(·) as exp(θ¹ + θ²x) with x ∼ N(0,1) and ε ∼ N(0,1) or ∼t(5) or χ₄² − 4. The values of θ¹ and θ² are both set equal to 1. Table 1 reports the finite-sample sizes of the ELR test (3), of the Bartlett-corrected ELR with the estimated Bartlett correction (16), and of the ELR with bootstrap critical values (23) of the null hypothesis H₀ : [θ¹ θ²] = [1 1] at 0.10, 0.05, and 0.01 nominal size. The results are based on 5,000 replications, and the bootstrap critical values are calculated from 1,000 bootstrap replications.

Example II. Robust regressions

We consider robust regression models with fixed regressors x_i^r that are assumed to satisfy max_1≤i≤n∥x_i^r∥ = O(1). The GS in this case is given by

for the psi function

satisfying E [ψ(y_i − x_i^rθ₀^r)] = 0. For this model, the centered bootstrap GS is

where y_i* = x_i^rθ₀^r + ε_i* is the bootstrap pseudo-observation, ε_i* is the bootstrap sample drawn from ε_i = y_i − x_i^rθ₀^r, and ε_n* = E*[ψ(ε_i*)] .

Following Huber (1973), we specify the psi function ψ(·) as

with the constant k = 1.4, the scale parameter σ² = 1, and Sgn{·} denoting the sign function.

Table 2 reports some Monte Carlo results for a simple two covariates design with an intercept and a single fixed regressor x_i generated as an equally spaced grid of numbers between −1 and 1 and points at −3 and 3, so that we have a rather substantial leverage effect. As with the previous example, the error ε is specified to be N(0,1) or t(5), or χ₄² − 4, and the two parameters θ¹ and θ² are set equal to 1. Table 2 reports the finite-sample sizes of the ELR test (3), of the Bartlett-corrected ELR with estimated Bartlett correction (16), and of the ELR with bootstrap critical values (23) of the null hypothesis H₀ : [θ¹ θ²] = [1 1] at 0.10, 0.05, and 0.01 nominal size. The results are based on 5,000 replications, and the bootstrap critical values are calculated from 1,000 bootstrap replications.

Bearing in mind that the scale of the simulation study is small, the results of Tables 1 and 2 seem to indicate the following. First, with the exception of the nonlinear regression case with N(0,1) errors, the ELR test is characterized by a noticeable size distortion, although the distortion tends to diminish when the sample size increases. Not surprisingly, the size distortion is more severe for the skewed χ₄² − 4 errors. Second, in the case of errors from a symmetric distribution, both the Bartlett correction and the bootstrap reduce the finite-sample size distortion of the ELR, although some size distortion is still present, especially at the 0.01 nominal level. On the other hand, for skewed errors the effectiveness of the Bartlett correction is reduced considerably. Finally, the ELR with bootstrap calibration has smaller size distortion than the ELR with a Bartlett-corrected χ² calibration. The first point supports the findings of Corcoran et al. (1995) and Baggerly (1998) about the relative poor quality of the χ² approximation to the distribution of the ELR statistic. The second point depends clearly on the form of the Bartlett correction (15), which implies that nonzero skewness typically reduces the magnitude of the Bartlett correction itself. The last point is related to the first one and should not therefore come as a surprise: the effectiveness of the Bartlett correction depends crucially on the quality of the χ² approximation. Thus for data sets for which the latter is not reasonable there should be significant gains by using the bootstrap.

To conclude this section, it is worth mentioning that the EL framework can easily incorporate additional information, most noticeably information about the second moment. For example, the conditional variance σ²(x) of the innovations ε in Example I can be parameterized by a known function h(x^r,η^a) that depends on an additional

-valued vector of parameters η^a—which may include θ^r also—and, similarly, in Example II one can introduce an estimating equation for the scale parameter σ².