1. MOTIVATION AND RESULTS

The kernel method is the most popular technique used in the estimation of nonparametric/semiparametric models, and it is well known that the selection of smoothing parameters in nonparametric kernel estimation is of crucial importance. In the context of a regression model, Clarke (1975) proposes the leave-one-out least squares cross-validation method for selecting the smoothing parameters. The asymptotic optimality of this approach is studied by Härdle and Marron (1985) and Härdle, Hall, and Marron (1988) in the context of a univariate regression model, and Fan and Gijbels (1995) have studied bandwidth selection in the context of local polynomial kernel regression. For a regression model with a single (univariate) continuous regressor, Härdle and Marron (1985) and Härdle et al. (1988) show that the cross-validation function has the following expression:

where

is the leave-one-out local constant kernel estimator of g(X_i) ≡ E(Y_i|X_i), k(.) is a second-order kernel function, h is the smoothing parameter, w(.) is a weight function, C₁ = ∫{κ₂ /2[g′′(x) f (x) + 2g′(x) f′(x)]}²w(x) f (x)⁻¹ dx, C₂ = κ ∫σ²(x)w(x) dx, κ₂ = ∫k(v)v² dv, κ = ∫k(v)² dv, g′(.) and g′′(.) denote first- and second-order derivative functions, and σ²(x) = Var(Y_i|X_i = x).

The terms of C₁ h⁴ and C₂ /(nh) in (1.1) are the leading squared bias and variance of CV(h), respectively. Let

denote the cross-validation selected smoothing parameter that minimizes CV(h); then from (1.1) it is easy to show that the

, where h₀ = [C₂ /(4C₁)]^1/5n^−1/5. Note that C₁ is nonnegative and C₂ > 0. Therefore, a necessary and sufficient condition for the existence of the unique benchmark nonstochastic optimal smoothing parameter h₀ is that C₁ > 0. The assumption that C₁ > 0 puts some restrictions on g(.); for example, g(.) cannot be a constant function. A similar necessary and sufficient condition exists that guarantees an asymptotically uniquely defined cross-validation selected smoothing parameter in estimating a conditional probability density function (p.d.f.) with an univariate continuous conditional variable.

The cross-validation procedure can be easily extended to the multivariate (regression or p.d.f. estimation) settings for selecting the smoothing parameters. However, the conditions that ensure the uniqueness of cross-validation selected smoothing parameters become more complex. Recently, Hall, Racine, and Li (2004), Hall, Li, and Racine (2004), and Li and Racine (2003, 2004) have considered the problem of nonparametric estimation of conditional density and regression functions with mixed discrete and continuous data. They propose to use the data-driven cross-validation (CV) methods to select the smoothing parameters, and they have shown that the CV selected smoothing parameters are asymptotically equivalent to the nonstochastic optimal smoothing parameters that minimize the asymptotic weighted estimation mean square error. However, when discussing the existence of the asymptotically uniquely defined optimal smoothing parameters, Hall, Racine, and Li (2004) and Li and Racine (2004) impose overly strong conditions. In this note we provide substantially weaker sufficient conditions that guarantee the existence of the uniquely defined CV selected optimal smoothing parameters. We show that when all covariates are continuous random variables, the condition becomes necessary and sufficient for the existence of uniquely defined optimal smoothing parameters.

We consider a nonparametric regression model with mixed discrete and continuous covariates:

where g(·) has an unknown functional form, E(u_i|X_i) = 0, X_i = (X_i^c,X_i^d), X_i^d is a q × 1 vector of regressors that assume discrete values, and X_i^c ∈ R^p are the remaining continuous regressors. We use X_ij^d to denote the jth component of X_i^d, and we assume that X_ij^d takes c_j ≥ 2 different values, that is, X_ij^d ∈ {0,1,…,c_j − 1} for j = 1,…,q. We use

to denote the range assumed by x^d. We are interested in estimating g(x) = E(Y_i|X_i = x) by the nonparametric kernel method. We use f (x) = f (x^c,x^d) to denote the joint density function. For x^c = (x₁^c,…,x_p^c) we use the product kernel:

, where k is a symmetric, univariate density function and 0 < h_j < ∞ is the smoothing parameter for x_j^c. For a discrete regressor we define, for 1 ≤ j ≤ q,

where 0 ≤ λ_j ≤ 1 is the smoothing parameter for x_j^d. Therefore, the product kernel for x^d = (x₁^d,…,x_q^d) is given by

. The kernel function for the mixed regressors x = (x^c,x^d) is simply the product of K^c and K^d, that is,

. The nonparametric estimate of g(x) is given by

. We choose (h,λ) = (h₁,…,h_p,λ₁,…,λ_q) by minimizing the following CV function:

where

is the leave-one-out local-constant (LC) kernel estimator of g(X_i) and 0 ≤ w(.) ≤ 1 is a weight function that serves to avoid difficulties caused by dividing by zero, or by the slow convergence rate for when X_i is near the boundary of the support of X.

Define an indicator function

. Note that I_j(v^d,x^d) = 1 if and only if v^d and x^d differ only in their jth component. Letting m_j(x) and m_jj(x) (m = g or m = f) denote the first-order and second-order partial derivatives of m(x^c,x^d) with respect to x_j^c, Hall, Li, and Racine (2004) have shown that (∫dx = [sum ]_x^d∈D ∫dx^c, D is the support of X^d)

The preceding results are based on the LC kernel estimation result. Li and Racine (2004) have considered the local linear (LL) CV method. The CV objective function is the same as given in (1.4) but with

replaced by a leave-one-out LL kernel estimator. Li and Racine (2004) have shown that the resulting CV function has the same form as (1.5) with the term 2g_j(x) f_j(x) removed.

Define z_j = n^−2/(4+p)h_j² for j = 1,…,p, and z_p+j = n^−2/(4+p)λ_j for j = 1,…,q; then both the leading terms of CV_LC(h,λ) and CV_LL(h,λ) can be written in the form of c₀ n^−4/(p+4)χ(z₁,…,z_p,z_p+1,…,z_p+q), where c₀ = κ^p∫σ²(x)w(x) dx > 0 is a constant, and

where z = (z₁,…,z_p+q)′ (the prime denotes transpose), A is a (p + q) × (p + q) symmetric positive semidefinite matrix with its (j,s)th element given by A_(j,s) = ∫B_j(x)B_s(x) dx, where B_j(x) = c₀^−1/2(κ₂ /2)[g_jj(x) f (x) + 2g_j(x) f_j(x)]w(x)^1/2f (x)^−1/2 (one removes 2g_j(x) f_j(x) if it is a local linear CV function) for j = 1,…,p, and B_p+j(x) = c₀^−1/2 [sum ]_v^d∈D I_j(v^d,x^d)[g(x^c,v^d) − g(x)] f (x^c,v^d)w(x)^1/2f (x)^−1/2 for j = 1,…,q.

Hall, Racine, and Li (2004) have considered the CV selection of smoothing parameters in a conditional probability (density) estimation framework and show that their CV objective function also has a leading term of the form as given in (1.6) with of course a different definition of B_j(x) for j = 1,…,p + q. Therefore, the leading term of the CV objective function, in either a regression or a conditional probability model, has the expression as given by (1.6). The uniqueness of the CV selected optimal smoothing parameters replies on the uniqueness of a nonnegative vector

that minimizes (1.6), where

. Subsequently we will first focus on the simple case that all covariates are continuous.

When q = 0 (no discrete covariates), all covariates are continuous random variables, and (1.6) becomes

with z = (z₁,…,z_p)′, and A is now of dimension p × p. The uniqueness of the CV selected optimal smoothing parameters of h₁,…,h_p hinges on the uniqueness of a vector

that minimizes (1.7). Let z* denote the vector of z that minimizes χ_c(z) over

; we ask that

If (1.8) holds true, then the CV selected smoothing parameters are all well defined asymptotically. In fact, it follows from Hall, Li, and Racine (2004) and Hall, Racine, and Li (2004) that

, or equivalently,

, where

is the benchmark nonstochastic optimal smoothing parameter (j = 1,…,p). The next theorem gives a simple necessary and sufficient condition for (1.8) to hold.

THEOREM 1.1. Assume that q = 0 so that z = (z₁,…,z_p)′; define

Then χ(z) has a unique minimizer

with 0 < z_j* < ∞ for all j = 1,…,p if and only if

Next, we discuss the general case with a mixture of continuous and discrete covariates. Now, z = (z₁,…,z_p+q)′ and A is a (p + q) × (p + q) symmetric positive semidefinite matrix. Let

where

and let

denote a minimizer of χ(z₁,…,z_p+q). We seek conditions that ensure the following result:

Condition (1.10) will lead to asymptotically uniquely defined CV selected smoothing parameters of

. We partition the A matrix as

where A₁₁ is of dimension p × p, and A₂₂ is of dimension q × q, and A₁₂ has a comfortable dimension. The following theorem gives the existence and uniqueness of a minimizer for χ(z).

THEOREM 1.2. Let

If μ > 0, then χ has a minimizer

with χ(z*) < +∞, and a necessary and sufficient condition for a point

to be a minimizer of χ is that z₍₁₎ = z₍₁₎* and z₍₂₎ = z₍₂₎* + z₍₂₎⁰ for some

, the null space of A₂₂.

The null space of A₂₂ is defined as

.

of χ is positive definite at every point

with χ(z) < +∞. Thus χ has a unique minimizer z* satisfying (1.10).