2. ESTIMATION PROCEDURE

Our task is to find a function

such that

We consider x* a scalar to simplify the exposition, although a multivariate extension is clearly possible.

at a given point

where x_l* and y_l for l = 1…n denote the data points and the kernel K_h(·) is of the form

and h is the bandwidth parameter. The problem we are facing is that x* is not observed. As shown in Schennach (2004) the availability of two repeated measurements of x*

provides enough information to identify any moment of the form E [u(y,x*)] for any function u(y,x*). Because the probability limit (at constant bandwidth h) of the Nadaraya–Watson kernel estimator is the ratio

a similar technique can be applied here, setting

, for k = 0,1. The extension of the existing results to a nonparametric setting nevertheless requires additional steps to handle the fact that we need to characterize an infinite family of moments, indexed by

. Fortunately, this complication can be elegantly handled by observing that the convolution operations involved in computing the Nadaraya–Watson estimator are converted into simple products through the Fourier transform operation, enabling the whole family of moments to be estimated in a single operation. The formal result that permits identification is summarized in the following set of assumptions and associated theorem. Throughout the paper, we will take the convention that integrals without explicit bounds are taken over the whole real line.

Assumption 1.

Δz and x* are mutually independent.

Assumption 2. E [|x*|] , E [|Δx|], and E [|y|] are finite.

Assumption 3.

for all

, any h > 0, and k = 0,1.

THEOREM 1. Under Assumptions 1–3, and provided |E [e^iξz]| > 0 for any finite ξ, the function

for

, can be expressed solely in terms of moments that involve the observable variables y, x, and z:

where, for k = 0,1,

and where φ_k(ξ) ≡ E [y^k exp(iξx*)] is given by³

where

is the Fourier transform of the kernel K(x*) and

Note that knowledge of the moments m_a(ξ), for a = 1,x,y, which involve observable variables only, is sufficient to identify

. Because the moments m_a(ξ) can be estimated from the corresponding sample averages, we propose the following estimator.

DEFINITION 1. Let (x_i,y_i,z_i), for i = 1,…,n denote a sample of size n. For a given

and some sequence of bandwidths h_n → 0, let

where, for k = 0,1,

and where, for a = 1,x,y,

An interesting property of this estimator is that it reduces to the Nadaraya–Watson estimator in the absence of measurement error (i.e., when z = x = x*). Indeed, in that case,

, and equation (19) can be integrated analytically to yield

, thus implying that equation (20) becomes

. With these equalities in mind, equation (18) then defines the Fourier representation of the numerator and the denominator of the Nadaraya–Watson estimator.

To ensure that the proposed estimator is well behaved, we need to make the following assumption.

Assumption 4. The Fourier transform of the kernel, κ(ξ), is (i) bounded and (ii) compactly supported (without loss of generality, we consider the support to be [−1,1]).

The boundedness of κ(ξ) is a very weak requirement because any kernel K(z) violating it would necessarily fail to be absolutely integrable. The assumption of compact support of κ(ξ) is commonly made in the derivation of the asymptotic properties of kernel deconvolution estimators (Fan and Truong, 1993). The need for this assumption arises from the fact that the estimator involves a division by an asymptotically vanishing characteristic function. Under very mild smoothness requirements, characteristic functions decay to zero as frequency increases toward infinity. A compactly supported kernel (in Fourier representation) explicitly makes the frequency range considered in a given sample finite, ensuring that the divergence is kept under control.

The restriction of compact support (in Fourier representation) poses few problems in practice, because one can take any given kernel K(x*) and construct a modified kernel

that exhibits most of the properties of the original kernel, while possessing a compact support in Fourier representation. This is achieved by computing the Fourier transform κ(ξ) of the original kernel K(x*) and multiplying it by a “windowing” function W(ξ) that vanishes beyond a given frequency:

Judicious choice of a windowing function will ensure that the modified kernel

keeps most of the properties of the original kernel. For instance, a windowing function such as

will leave the order of the kernel unaffected, because the windowing function is constant in the neighborhood of the origin. The fact that this windowing function is infinitely many times differentiable will guarantee that the modified kernel

decays faster than any power of x* as |x*| → ∞ (provided that the original kernel K(x*) had this property).

3. ASYMPTOTIC PROPERTIES

This section is organized as follows. To facilitate the analysis of the asymptotic properties of the proposed estimator

, we first provide a linear representation of this estimator, denoted

, that will be shown to be asymptotically equivalent to

. This linearization serves two purposes. First, it will enable the derivation of the convergence rate of the estimator using techniques that are analogous to the standard bias and variance decomposition used in the context of conventional kernel estimators. Second, a linear representation is essential to establish the asymptotic normality of the estimator.

3.1. Linearization

In this section, we will provide very general results that summarize the properties of a linearized estimator

that will be used to establish the asymptotic properties of

. The form of the estimator prompts for two levels of linearization. First, as is commonly done in the analysis of nonparametric conditional expectation kernel estimators, the ratio of

in equation (17) is expanded in a Taylor series up to first order. Second, unlike the usual Nadaraya–Watson estimator and kernel deconvolution estimators,

themselves take the form of nonlinear functionals of the data generating process. It is thus convenient to carry out the linearization a step further by calculating the Fréchet derivative of

with respect to the estimated moment

in the vicinity of

. The following definition gives a linearized version

of the estimator

.

⁴

The calculation of the Fréchet derivative can be found in the proof of Lemma 2 in the Appendix.

DEFINITION 2. For

, let

where, for

is given by equation (13),

The advantage of the linear representation provided by Definition 2 is that it is possible to decompose the error

into well-defined “bias” and “variance” terms, as given by Lemma 1, which follows.

Assumption 5. [y_i,x_i,z_i,x_i*,Δy_i,Δx_i,Δz_i] for i = 1…n is an independent and identically distributed (i.i.d.) sequence.

Assumption 6. E [y^2−j|z|^j] < ∞, E [x^2−j|z|^j] < ∞, for j = 0,1.

Assumption 7. The density of x* is nonzero at

.

LEMMA 1. Under Assumptions 1–7, for

,

where the bias term

and the variance term

are given by

and where

satisfies

and

where

for k₁,k₂ = 0,1, where

is given in Definition 2, where

for k = 0,1 is given in Theorem 1, and where [dagger] denotes complex conjugation, and

Under our assumptions, the expectation and the variance of

are well defined, even though the corresponding moments of

may not exist. As long as the remainder

can be shown to be asymptotically negligible in probability, the mean and the variance of

can be interpreted as the mean and the variance of the limiting distribution of

, whether or not the first two moments of

are bounded. This situation is not unique, as these observations apply to any estimator involving ratios of random quantities. To ascertain that the linear approximation

is appropriate, the following lemma provides the order of the remainder of the linearization of

and also the order of the statistical fluctuations in

. This result is included for completeness, but it is not essential for the reader to master it to understand the main results of the subsequent sections.

LEMMA 2. Let Assumptions 1–7 hold and let, for φ₀(ζ), φ₁(ζ), and m₁(ζ) as in Theorem 1,

for φ₀′(ξ) ≡ dφ₀(ξ)/dξ.

⁵

Note that the ratio |φ₀′(ζ)|/|φ₀(ζ)| entering the definitions of λ(h_n) and U(h_n) can equivalently be written as |m_x(ζ)|/|m₁(ζ)| because |m_x(ζ)|/|m₁(ζ)| = |E [xe^iζz]|/|E [e^iζz]| = |E [x*e^iζz]|/|E [e^iζz]|=(|E [x*e^iζx*]|/|E [e^iζx*]|)(|E [e^iζΔz]|/|E [e^iζΔz]|)=|E [x*e^iζx*]|/|E [e^iζx*]|=|φ₀′(ζ)|/ |φ₀(ζ)|.

If the sequence h_n is such that (i)

, (ii) U(h_n)n^−1/2 → 0, and (iii) λ(h_n)n^−1/2+ε → 0 for some ε > 0, then

If, in addition, (iii)

, then

The quantity U(h_n) is defined so that it bounds any of the quantities defined in equations (26) that enter the expression of the asymptotic variance of the estimator, whereas λ(h_n) bounds the remainder terms from the linearization performed in Definition 2. As expected, the preceding stochastic expansion is written in terms of successive powers of n^−1/2, with the exception that the second term is proportional to n^−1+ε instead of n⁻¹, because bounding the second remainder term involves uniformly bounding various random functions, which slows the rate down by a factor n^ε.

In the proof of our convergence rate and asymptotic normality results, we will subsequently verify that the hypotheses of Lemma 2 are implied by more primitive regularity conditions. The first conclusion of the Lemma (equations (36) and (37)) will be sufficient to obtain the convergence rate of the estimator. Indeed, if it can be shown that λ(h_n)n^−1/2+ε → 0, the convergence rate is then simply given by O_p(U(h_n)n^−1/2). Because O_p(U(h_n)n^−1/2) is an upper bound on the convergence rate, which may or may not be binding, the second, slightly stronger conclusion of Lemma 2 (equation (38)) will be needed to obtain the limiting distribution of the estimator. The basic intuition behind the additional condition (iii) is that, for the O_p(U(h_n)n^−1/2λ(h_n)n^−1/2+ε) nonlinear remainder to have no effect on the limiting distribution, it must be asymptotically negligible relative to the exact standard deviation of

, which is given by

, by Lemma 1.

3.2. Regularity Conditions

We now provide primitive regularity conditions that will enable us to derive explicit convergence rates. These regularity conditions take the form of smoothness restrictions imposed via constraints on the tail behavior of various Fourier transforms. To specify the regularity conditions, we employ the following convenient notation.

DEFINITION 3. An expression of the form f (ζ) [prcue ] g(ζ) for

indicates that there exists a constant C > 0, independent of ζ, such that f (ζ) ≤ Cg(ζ) for all

(and similarly for [sccue ]). Analogously, a_n [prcue ] b_n for two sequences a_n,b_n indicates that there exists a constant C independent of n such that a_n ≤ Cb_n for all

.

The literature focusing on “kernel deconvolution estimators” (see, e.g., Carroll et al., 1995) and related estimators (Fan and Truong, 1993) traditionally distinguishes between “ordinarily smooth” functions (whose Fourier transform decays as |ζ|^γ, γ < 0 as |ζ| → ∞) and “supersmooth” functions (whose Fourier transform decays as exp(α|ζ|^β), α < 0,β > 0 as |ζ| → ∞). For the benefit of conciseness, our regularity conditions are given in terms of expressions of the form (1 + |ζ|)^γ exp(α|ζ|^β), thereby simultaneously covering the ordinarily smooth and supersmooth cases.

Assumption 8. The functions φ₀(ζ) = E [e^iζx*] , φ₀′(ζ) ≡ dφ₀(ζ)/dζ, φ₁(ζ) = E [ye^iζx*] , and m₁(ζ) = E [e^iζz] satisfy

for some γ_r ≥ 0 and

for some

such that γ_φ β_φ ≥ 0 and γ_m β_m ≥ 0.

A few remarks are in order. While the rate of decay of φ₀(ζ), the characteristic function of x*, is entirely determined by the smoothness of the density f (x*) of x*, the rate of decay of φ₁(ζ) is governed by the smoothness of f (x*)E [y|x*]. Verifying equation (40) would first involve finding bounds on |φ₀(ζ)| and |φ₁(ζ)| individually before taking the most slowly decaying term. Regrouping φ₀(ζ) and φ₁(ζ) in a single assumption is possible without loss of generality, because both quantities enter the expression of the estimator in a similar fashion. This grouping is also notationally convenient, as it will reduce the number of independent orders of magnitude that have to be considered when determining the convergence rates of the estimator.

As is always the case in deconvolution-type estimators, one quantity (here m₁(ζ)) needs to be bounded below (in equation (41)), instead of above, because it appears in a denominator in the expression of the estimator. Note that equation (41) is implied by separate lower bounds on the modulus of the characteristic functions of x* and Δz because m₁(ζ) = E [e^iζz] = E [e^iζx*]E [e^iζΔz]. The grouping of E [e^iζx*] and E [e^iζΔz] is also aimed at reducing the notational burden. Although the constraint on the ratio φ₀′(ζ)/φ₀(ζ) imposed by equation (39) may appear unusual, it is clear that it is implied by a more familiar upper bound on |φ′(ζ)| and a lower bound on |φ(ζ)|. The absence of a term of the form exp(α_r|ζ|^β_r) in equation (39) results in very little loss of generality, because all common ordinarily smooth and supersmooth functions are such that equation (39) holds for γ_r = 1.

Before we can derive the convergence rate of the estimator, we also need to characterize the type of kernel K(x*) used. While most studies of measurement error in nonparametric settings focus either on finite-order kernels (Fan, 1991b; Fan and Truong, 1993) or on infinite-order kernels (Politis and Romano, 1999; Li and Vuong, 1998), we will consider both finite- and infinite-order kernels. The traditional finite-order kernels we consider are defined in Assumption 9.

Assumption 9. ∫K(x*) dx* = 1 and, for some integer γ_κ > 0,

We also consider the following class of “infinite-order” kernels.

Assumption 10. The Fourier transform of the kernel, κ(ξ), is such that κ(ξ) = 1 for |ξ| < ξ for some ξ > 0.

Assumption 10 allows for a kernel of the form

which is particularly suited to the Fourier representation because its Fourier transform is 1 in the [−1,1] interval and zero elsewhere. This type of kernel has previously been used in other Fourier-based estimators (Li and Vuong, 1998) and amounts to truncating the Fourier transform above a given frequency. When both E [y|x*] and the density of x* are infinitely many times differentiable, an infinite-order kernel will guarantee that the bias goes to zero faster than any power of the bandwidth. The bias could then, for instance, be an exponentially decaying function of the inverse bandwidth h⁻¹.

3.3. Rate of Convergence in Probability

The procedure to determine the asymptotic rates of pointwise convergence in probability can be outlined as follows.

1. Bound the bias of the linearized estimator in terms of the bounds given in Assumption 8.
2. Bound the variance of the linearized estimator in terms of the bounds given in Assumption 8.
3. Find the sequence of bandwidths h_n that makes the order of the bias squared and of the variance equal and verify that the higher order terms are asymptotically negligible, so that the asymptotic properties of the estimator
   
   can be obtained from the properties of its linearization
   
   .

Step 1.

Calculation of the bias

. We distinguish two cases, depending on whether the kernel used satisfies Assumption 9 or Assumption 10. In the following two lemmas, recall that the parameters γ_φ, α_φ, and β_φ, defined in Assumption 8, describe the smoothness of the density f (x*) of x* and of the conditional expectation E [y|x*] by specifying that their Fourier transforms both decay at least as fast as ζ^γ_φ exp(α_φ|ζ|^β_φ) as frequency ζ → ∞.

LEMMA 3. Under Assumptions 1–8, if the kernel is of order γ_κ, as defined by Assumption 9, then the bias satisfies

where α_b = 0, β_b = 0, and

LEMMA 4. Under Assumptions 1–8, if the kernel satisfies Assumption 10 for some constant ξ, then the bias satisfies

where

In short, when a finite-order kernel is used, the rate of decrease of the bias is controlled either by the order of the kernel γ_κ or by the smoothness of f (x*) and E [y|x*] , whichever is more limiting. In particular, when both f (x*) and E [y|x*] are supersmooth, so that α_φ ≠ 0, it is the order of the kernel that determines the rate of decrease of the bias. When an infinite-order kernel is used, only the smoothness of f (x*) and E [y|x*] matters. Note that the bias term is identical to that of a traditional kernel estimator that would be used if x* were perfectly observed, because, via equations (12) and (13), the bias can be expressed entirely in terms of φ_k(ζ) for k = 0,1 and the kernel, which are nonrandom measurement error-free quantities.

Step 2.

Calculation of the order of the variance term

.

LEMMA 5. Under Assumptions 1–8, the variance term satisfies

where

Note that the order of the variance term is determined not only by the smoothness of f (x*) and E [y|x*] (through γ_φ, α_φ, β_φ, and γ_r) but also by the smoothness of the density of the measurement error Δz (through the terms γ_m, α_m, and β_m). It is important to point out that the variance term increases much faster as h → 0 (at constant n) than that of a standard kernel estimator with perfectly observed variables (whose variance term is O_p((h_n n)^−1/2)). Combined with the fact that the bias term is unchanged, as indicated in step 1, this implies that the achievable convergence rates will generally be slower than for a conventional kernel estimator.

Step 3.

Determination of the rate of decrease of the bandwidth that offers the best trade-off between bias squared and variance. To obtain explicit rates of convergence, we need to distinguish various cases, based on the values of β_b, which characterizes the rate of convergence of the bias term as the bandwidth shrinks, and β_v, which characterizes the rate of divergence of the variance term as the bandwidth shrinks (at constant sample size). Both β_b and β_v represent an “exponent of supersmoothness,” that is, the constant β in an expression of the form (h_n⁻¹)^γ exp(α(h_n⁻¹)^β).

THEOREM 2. Under Assumptions 1–8 and either Assumption 9 or 10, the optimal bandwidth choices and the corresponding convergence rates in probability of the estimator can be expressed in terms of the constants α_b, β_b,γ_b,α_v,β_v,γ_v defined by Lemmas 3–5. Let ε > 0 be arbitrarily small, let C₁,C₂ be some positive constants, and let

be given.

Case 1. If β_v > β_b > 0

Case 2. If β_v > 0 and β_b = 0 (with α_b = 0 and γ_b < 0)

Case 3. If β_b = β_v ≠ 0

Case 4. If β_b = β_v = 0 (with α_b = α_v = 0 and γ_b < 0)

A few remarks are in order. First, it can be verified (see the proof of Theorem 2 in the Appendix) that the bandwidth sequences given above are such that conditions (i) and (ii) of Lemma 2 hold, thus implying that the nonlinear remainders are indeed negligible and that our simple bias-variance decomposition is justified. Second, the arbitrarily small ε was introduced to drastically simplify the calculations and the statement of the results at the expense of a very small loss in precision. Third, it is impossible to have β_b > β_v because β_b = β_φ, β_v = β_m, and

The convergence rate of the proposed estimator varies substantially as a function of the smoothness of the densities and the conditional expectations involved. An important trend to observe among these rates is that large values of β_b (indicating a rapidly decreasing bias as h → 0) and small values of β_v (indicating a slowly increasing variance as h → 0) are desirable. The convergence rates obtained are typically slower than that of the Nadaraya–Watson kernel estimator used when the variables are perfectly observed. This limitation is not an artifact of our estimation procedure: it has also been observed in the simpler Fan and Truong estimator, which is known to be optimal under stronger assumptions than ours (see Fan and Truong, 1993). The different cases will be discussed—and compared to Fan and Truong's findings—in more detail in Section 4.

Although we have focused on pointwise convergence rates, our results also provide information regarding global convergence rates. The upper bounds on the pointwise bias and variance (and of the nonlinear remainder terms) are in fact independent of

. If the density of x* is bounded away from zero over some finite interval [a,b] , it is straightforward to show that

converges to zero in probability at the same rate as the pointwise rates derived earlier for any bounded weighting function

and any p ∈ [1,2]. However, rates of uniform convergence in probability do not follow directly from the results presented above.

3.4. Asymptotic Normality

To establish the asymptotic normality of the proposed estimator, we need to introduce a few additional assumptions. First, we need assumptions that are commonly made whenever a central limit theorem for triangular arrays is invoked (see, e.g., Härdle and Linton, 1994, Theorem 2; Andrews, 1991, Assumption A).

Assumption 11. There exists C > 0 such that E [|x|^2+δ|z] ≤ C, E [|y|^2+δ|z] ≤ C, Var[x|z] ≥ C, and Var[y|z] ≥ C for all z.⁶

The familiar condition E [|K(x*)|^2+δ] < ∞, which is helpful to show the asymptotic normality of standard kernel estimators, is of no use in establishing the asymptotic normality of our more complex estimator. In any case, Assumption 4 implies that E [|K(x*)|^2+δ] < ∞.

The remaining assumptions are used to ensure that the condition

in Lemma 2 holds, so that the higher order remainder terms are asymptotically negligible relative to the standard deviation of the linearized estimator

. The main obstacle to overcome is the necessity to find a lower bound for the variance

of the estimator. The difficulty of obtaining such a result is noted by Fan (1991a) in his study of the limiting distribution of the kernel deconvolution estimator. Fan's solution to this problem is simply to assume that the tails of the various Fourier transforms entering the estimator are not only bounded by some function of the form ζ^γ exp(α|ζ|^β) but are asymptotically equal (as |ζ| → ∞) to such functional form, thereby limiting the set of allowed functions. Our solution to this problem is similar in spirit to Fan's but considerably expands the range of possible behavior toward infinity by employing the concept of functions that are “well behaved at infinity,” as described by Lighthill (1962). The following definition formalizes this notion.

⁷

We expand Lighthill's definition by allowing for exponential tails, which is essential to handle supersmooth functions.

DEFINITION 4. Let

be the set of all functions

such that (i) ψ(ζ) is absolutely integrable in every finite interval and (ii) ∫_|ζ|≥T|ψ(ζ) − Ψ(ζ)| dζ < ∞ for some

and some function Ψ(ζ) that can be written as a finite linear combination of finite products of functions of the form |ζ|^c+, sgn(ζ)|ζ|^c+, ln|ζ|, sin(cζ), cos(cζ), exp(cζ^γ) with

.

Assumption 12. For a given

, the functions

, for k = 0,1 and l = 1,x,y and for

given in equation (26), belong to

.

For simplicity, we do not state Assumption 12 in terms of elementary quantities such as m₁(ζ) and φ_k(ζ), but it is clear that Assumption 12 is only a few algebraic manipulations away from being a primitive condition. We need to constrain the derivative of

to rule out counterexamples where the density of z arbitrarily far away from the point of evaluation

could have a nonvanishing influence on

asymptotically, making it difficult to characterize the behavior of the variance as n → ∞.

The following condition requires the distribution of z to be supported on

, which is usually the case in deconvolution problems because distributions that have a nonvanishing characteristic function (as imposed by equation (41) in Assumption 8) rarely have compact support.

Assumption 13. f (z) > 0 for all

.

Finally, we need to impose a few constraints that would be very difficult to state in a more primitive fashion. However, these assumptions are not very restrictive because the counterexamples violating them are somewhat contrived.

Assumption 14.

.

This assumption merely states that the variance of the estimator is of an order no less than any term in its asymptotic representation. This constraint can only be violated if two or more of the terms

happen to cancel out asymptotically, which is unlikely because each term depends on different random quantities.

Assumption 15. For

as in equation (27),

for k = 0,1, for all

and all

.

This assumption requires that

be of the same order. It precludes

from having an oscillatory behavior (as ξ varies) such that a precise cancellation would occur between the values of

at different ξ during the integration. The cancellation would have to occur for all ζ and n sufficiently large and be such that the order of

would be affected.

Assumptions 12–15 imply condition

in Lemma 2, thus establishing the required asymptotic negligibility of the nonlinear remainder terms. If it is possible to calculate

directly and verify that it goes to 0 asymptotically, then Assumptions 12–15 can be avoided altogether.

⁸

And the term n^{1/(3+2γ_r−2γ_m)} in equation (64) can be replaced by n^{1/(2+2γ_r−2γ_m)}.

We are now ready to state our asymptotic normality result.

THEOREM 3. Under Assumptions 1–8 and 11–15, for any given

and any sequence h_n satisfying

for some η > 0, we have

where

are given in Lemma 1.

4. EXAMPLES

Section 3.3 derives the convergence rates of the proposed estimator under very general conditions. We now focus on specific examples that will allow us to compare these convergence rates with those derived for the estimator proposed by Fan and Truong (1993), which is the most closely related to ours. Fan and Truong's estimator extends the standard kernel deconvolution estimators used for density estimation in the presence of a measurement error drawn from a known distribution to the case of nonparametric regressions. The estimator presented here accomplishes a more difficult task than Fan and Truong's because it considers the density of the measurement error unknown, relying instead on two error-contaminated measurements of the unobserved regressor. Hence, it would come as no surprise if the kernel deconvolution rates were better. The comparison is nevertheless instructive, because it quantifies the precision loss incurred by relaxing the distributional assumptions regarding the measurement error.

We consider four examples. We first study the “difficult” deconvolution problem that consists of estimating an ordinarily smooth conditional expectation (E [y|x*]) when the density of both the true regressor x* and the measurement error Δz are supersmooth. This problem is difficult because a supersmooth measurement error strongly damps out the high-frequency components of E [y|x*] and of the density of x*. Inverting this operation involves the amplification of these damped-out components, an operation that necessarily causes a substantial magnification of the statistical noise. In standard kernel deconvolution estimators, this situation gives rise to extremely slow convergence rates, and it is instructive to verify that the situation does not degrade further when the distribution of the measurement error is unknown. The second example shows that this slow convergence problem is avoided when the conditional expectation E [y|x*] is supersmooth as well. The third example assumes the density of the measurement error is ordinarily smooth, a situation that avoids the slow convergence problem for the kernel deconvolution estimator but, as we will see, not for our estimator. The final example completes the analysis by showing that when all quantities are ordinarily smooth, the slow convergence problem is avoided.

Table 1 summarizes the assumptions made in each of the four cases considered. A few remarks are in order. In each case, we assume that the order of the kernel is sufficiently large so that the smoothness of E [y|x*] and of the density of x* (and not the order of the kernel) is the factor limiting the rate at which the bias goes to zero. We also assume that equation (39) holds with γ_r = 1. Table 1 also summarizes the convergence rates obtained by applying Theorem 2 in each of the four examples considered. We will now discuss the significance of these results.

Convergence rates obtained under given regularity assumptions

In Example 1, the rates are entirely comparable to those obtained by Fan and Truong (1993) for kernel deconvolution estimators. They found rates of the form (ln n)^k/β where k is the number of continuous derivatives that g(x*) possesses. Because a function whose Fourier transform behaves asymptotically as ζ^−(k+1+ε) necessarily has k continuous derivatives, it is clear that the rates are comparable. The rates differ by ε, because Fan and Truong formulate their regularity conditions in terms of derivatives whereas we formulate them in terms of the asymptotic behavior of Fourier transforms. Formulating our regularity conditions in terms of derivatives would yield results identical to Fan and Truong's. It is remarkable that under the assumptions leading to the worst-case convergence rates for kernel deconvolution estimators, the assumption of a known measurement error distribution can be relaxed without bringing the convergence rate down further.

Example 2 shows that the slow convergence rate problem can be alleviated if the unknown regression function g(x*) is supersmooth and if an “infinite-order” kernel is used. This situation ensures that the bias term goes to zero faster than any power of h, which is sufficient to convert a convergence rate of the form (ln n)^γ to a rate of the form n^γ for γ < 0. More generally, relatively fast convergence rates can be achieved with infinite-order kernels whenever case 3 of Section 3.3 applies. Caution is, however, advised when using high-order kernels. They are known not to perform as well in finite samples as their asymptotic properties would suggest (see Härdle and Linton, 1994). The origin of the problem is that a high-order kernel must necessarily take negative values over a portion of its support, which makes it likely for the denominator of the Nadaraya–Watson kernel estimator to approach zero, even at a point where the true density is bounded away from zero.

In Example 3, making the density of the measurement error Δz ordinarily smooth instead of supersmooth does not improve the convergence rates relative to Example 1. This is in sharp contrast to the behavior of kernel deconvolution estimators, whose convergence rates are of the form n^k under the same assumptions. The reason for this distinction is that the only characteristic function appearing in the denominator of a kernel deconvolution estimator is that of the measurement error Δz, whereas in our estimator, it is the characteristic function of z that appears in the denominator. The density of z is supersmooth if either the density of the true regressor x* or of the measurement error Δz is supersmooth. Hence, a supersmooth density for x* will also cause our estimator to converge slowly.

In Example 4, it is seen that when the density of x* is made ordinarily smooth as well, the slow convergence problem is avoided, as expected. The resulting rates are not necessarily identical to those of Fan and Truong's kernel deconvolution estimator, but the rates at least take the form of a negative power of n, indicating that the distributional assumptions regarding the measurement error can be relaxed without an undue increase in the statistical noise.

5. MONTE CARLO SIMULATIONS

We now investigate the finite-sample properties of the proposed estimator through various Monte Carlo simulations. The designs are chosen so as to illustrate the examples of Section 4, summarized in Table 1, which cover the most common combinations of smooth and supersmooth distributions and conditional expectations. As an example of a supersmooth distribution, the normal distribution with variance σ² naturally comes to mind. Its characteristic function has a tail of the form exp(−(σ²/2)|ζ|²). As an example of an ordinarily smooth distribution, we consider the Laplace (or double exponential) distribution with mean μ and variance σ² denoted by L(μ,σ²) and defined as

for any

. The tail of the characteristic function of a Laplace density is of the form |ζ|⁻².

Our example of a supersmooth regression function is the error function

having a Fourier transform decaying at the rate |ζ|⁻¹ exp(−¼|ζ|²)as |ζ| → ∞. Finally, our example of an ordinarily smooth regression function is a piecewise linear continuous function with a discontinuous first derivative

whose Fourier transform decays as |ζ|⁻². To simplify comparisons, both functions are normalized to have the same range and a similar general shape, so that any difference in the results can be attributed to their difference in smoothness. All simulations proceed by drawing 500 samples of 1,000, 2,000, or 8,000 observations from the distributions given in Table 2. Table 2 also provides the theoretical convergence rate in each case, obtained by substituting the appropriate smoothness parameters in the expressions of Table 1. The distribution of Δy is never altered, because it has little impact on the asymptotic properties of the estimator except for a trivial scaling of some of the components of the asymptotic variance. For each sample, the variables x,y,z are constructed through

The variables (y,x,z) are used as an input for our estimator, and the variables (y,x) are fed into the Nadaraya–Watson estimator. We also construct an (infeasible) Nadaraya–Watson estimator from the variables (y,x*) for comparative purposes. For all three estimators, an infinite-order kernel whose Fourier transform is given by equation (23) with ξ = ½ is used. In this fashion, the kernel is never the factor limiting the convergence rate. For each sample, we keep track of the value of the estimated function at a given point (here, x* = 1) and use it to calculate the bias squared, the variance, and the sum of the two, the mean square error. A set of bandwidths ranging from 1.0 to 2.5 is scanned in increments of 0.05 in search of the bandwidth minimizing the mean square error.⁹

Monte Carlo simulation designs

Table 3 compares the bias squared, the variance, and the mean square error of the three estimators considered as a function of bandwidth for a sample size of 1,000. For conciseness, only a subset of the bandwidths considered is shown. The rightmost column gives all quantities evaluated at the optimal bandwidth (which may lie between two of the bandwidths listed in the previous columns). A few important features can be consistently observed throughout the four examples considered.

Monte Carlo simulation results for the examples

In comparison with the Nadaraya–Watson estimator, our estimator is clearly very effective at reducing the bias. More specifically, it is clear that the bias of the Nadaraya–Watson estimator does not converge to zero with decreasing bandwidth but instead settles to a nonzero value. In contrast, the bias of our estimator decreases by orders of magnitude over the range of bandwidths sampled, as the bandwidth decreases. Our estimator's residual bias is attributable to the fact that we are performing a nonparametric estimation, so that a fully unbiased estimation is impossible. In fact, it can readily be seen that, at a given bandwidth, the bias of our estimator is very close to the bias of the infeasible Nadaraya–Watson estimator using the uncontaminated regressor x*, thus indicating that our estimator does not appear to introduce additional bias at the sample size considered. Of course, because the variance of our estimator is larger than the infeasible Nadaraya–Watson estimator, a larger bandwidth must be used, and the resulting bias, evaluated at the optimal bandwidth, is slightly larger than in the error-free case.

The bias reduction made possible by the proposed estimator comes at the expense of an increased variance relative to the Nadaraya–Watson estimator based on mismeasured regressors. However, the decrease in the bias more than offsets the increase in the variance, so that the mean square error we obtain is still better than for the Nadaraya–Watson estimator.

It is instructive to observe the estimator's behavior as a function of the smoothness of the various densities and conditional expectations considered. The asymptotic theory presented earlier predicts the convergence rate, which can be directly compared with the change in the mean square error at the optimal bandwidth, as a function of sample size for each of the examples considered (see Table 4). The fifth column of Table 4, labeled “MSE₈₀₀₀/MSE₂₀₀₀,” reports the ratio of mean square error at a sample size of 8,000 relative to the mean square error at sample size 2,000. We focus on these sample sizes because the differences between the various examples are more readily seen at large sample sizes. In Examples 1 and 3, where the convergence rate should be slow (i.e., a negative power of the log of sample size), convergence is indeed much slower than for Examples 2 and 4, where the convergence rate should be fast (i.e., a negative power of sample size). Moreover, the decrease in mean square error predicted by asymptotic theory (obtained by squaring the rates given in Table 2 and shown in the last column of Table 4) is an excellent predictor of the actual decrease in three out of the four examples. Note that the systematic changes in bandwidth as a function of sample size are difficult to distinguish from the inherent simulation noise, because bandwidth variations are much smaller than the changes in mean square error, as predicted by Theorem 2.

Monte Carlo simulation results as a function of sample size

Monte Carlo simulations can also be used to verify the applicability of the asymptotic distribution in a finite sample. The designs described in Table 2 are again used, with the mean square minimizing bandwidths given in Table 3 and a sample size of 1,000. For each sample, we keep track of the value of the estimated function at a given point (x* = 1.0) and the estimated variance at that point obtained with equations (31) and (32) by replacing all expected values by sample averages. The point estimates are then standardized, that is, demeaned by the average of the point estimates and normalized by the average of the estimated pointwise variance. Figure 1 shows the empirical cumulative distribution function (c.d.f.) of the standardized point estimates p_i for i = 1,…,500, obtained by sorting the p_i in increasing order and by joining the points (p_i,(i − 1)/499) by lines. The resulting empirical c.d.f. (jagged lines in Figure 1) agrees very well with the normal c.d.f. predicted by asymptotic theory (shown as a smooth line in Figure 1).

Comparison between the finite-sample and the asymptotic distributions of the estimator. The abscissa is .