Hostname: page-component-74656f9884-c52kr Total loading time: 0 Render date: 2026-11-07T04:38:46.621Z Has data issue: false hasContentIssue false
  • English
  • Français

AN ALTERNATIVE TO MAXIMUM LIKELIHOOD BASED ON SPACINGS

Published online by Cambridge University Press:  31 March 2005

Stanislav Anatolyev  and
Grigory Kosenok
Stanislav Anatolyev
Affiliation:
New Economic School, Moscow
Grigory Kosenok
Affiliation:
New Economic School, Moscow
Rights & Permissions [Opens in a new window]

Abstract

In the statistics literature, asymptotic properties of the Maximum Product of Spacings estimator are derived from first principles. We propose an alternative derivation based on the comparison between its objective function and that of the Maximum Likelihood estimator.We thank the co-editor Paolo Paruolo for his patience and an anonymous referee for providing references to the statistics literature.

Information

Type
NOTES AND PROBLEMS
Information
Econometric Theory , Volume 21 , Issue 2 , April 2005 , pp. 472 - 476
Copyright
© 2005 Cambridge University Press

1. MOTIVATION

Consider a scalar continuously distributed random variable x with probability density function (p.d.f.) f (x) ≡ f (x,θ) and corresponding cumulative distribution function (c.d.f.) F(x) ≡ F(x,θ) known up to a possibly multidimensional parameter θ ∈ Θ. Denote the true value of the parameter by θ0. Suppose that a random sample x1,…,xn is available. To estimate θ0, one usually uses the maximum likelihood (ML) estimator

where

and x(i) is ith order statistic. The ML procedure adjusts the shape of the density by tuning the parameter θ so that the product of density values at the sample observations is maximized. Suppose that the regularity conditions for consistency and asymptotic normality of the ML estimator hold; in particular, assume that the support of x does not depend on θ and θ0 is a well-separated point of maximum of E [log f (x,θ)] (see, e.g., van der Vaart, 2000, Ch. 5).

The following maximum product of spacings (MPS) estimator has been encountered in the statistics literature but is not widespread among econometricians:

where

and F(x(0)) ≡ 0, F(x(n+1)) ≡ 1. The MPS criterion amounts to tuning the parameter θ so that the product of probabilities of a new observation falling between each two neighboring sample points in the set of order statistics is maximized (a similar interpretation was given by Titterington, 1985). Given that the sum of those probabilities equals one, at the optimum they are as close to each other as possible.

The MPS estimation was introduced independently by Cheng and Amin (1983) and Ranneby (1984). The advantages of MPS over ML were demonstrated when the boundedness of the objective function by construction leads to the consistency of MPS estimates when ML estimates may not exist or may not be unique, and in numerous examples when the density is positive only to the right of a shifted origin, it being one of unknown parameters (Cheng and Amin, 1983). In regular cases MPS estimates are consistent, asymptotically normal, and efficient, which was derived in subsequent papers from first principles (e.g., Shao and Hahn, 1999; Ghosh and Jammalamadaka, 2001).

However, in regular cases when both the ML and MPS estimators are well defined, one can derive the asymptotic properties of the MPS by contrasting the two estimators to the first order as n → ∞.

2. RESULTS AND DISCUSSION

We require that f (x,θ) be twice continuously differentiable in θ and continuous in x for all θ ∈ Θ. We will denote by xL and xU the minimal and maximal points of the support of x (either or both may be infinite). The support is supposed to be independent of the parameter θ and be connected, so that f (x) > 0 for any x ∈ (xL,xU). Denote the true p.d.f. by f0(x) ≡ f (x,θ0) and the true c.d.f. by F0(x) ≡ F(x,θ0). We assume that F(x,θ) and f (x,θ) satisfy the following additional conditions.

Additional Assumption.

(i) limx→xL F0(x)log2 F(x) = limx→xL F0(x)log2 f (x) = 0, limx→xU(1 − F0(x))log2(1 − F(x)) = limx→xU(1 − F0(x))log2f (x) = 0.

(ii)For any θ ∈ Θ, the number of local optima of f (x) is finite.

Part (i) imposes a sort of continuity on the tails as the parameter varies. Part (ii) is not necessary, but it facilitates the proof. It is straightforward to verify that the Additional Assumption is satisfied for frequently used distributions. Our main result follows a helpful lemma.

LEMMA 1. If for some g(x) we have limx→xL F0(x)g(x) = 0, then g(x(1)) = op(n). Similarly, if for some g(x) we have limx→xU(1 − F0(x))g(x) = 0, then g(x(n)) = op(n).

Proof. The distribution of F0(x) is uniform on [0,1], and hence (e.g., Feller, 1971)

so that nF0(x(1)) = Op(1), but not op(1), as n → ∞. Observe that, as

,

It follows that g(x(1)) = op(n). One handles the second implication in a similar manner. █

THEOREM 2. Let the regularity conditions for consistency and asymptotic normality of the ML estimator

hold. Let the Additional Assumption also hold. Then

.

Proof. Denote Δx(i) ≡ x(i) − x(i−1). By the mean value theorem, for i = 2,…,n,

where x(i)* ∈ (x(i−1),x(i)). Then

where

Hence

The first term is independent of θ and thus does not affect the optimizer. By Additional Assumption(i) and Lemma 1 appropriately applied with g(x) = log2 F(x) and g(x) = log2(1 − F(x)), the second and third terms are op(n−1/2).

Let us denote the K local optima of f (x,θ) by

, and define in addition

. For every interval between adjacent points

, denote by x(ikl) and x(ikr) the leftmost and rightmost order statistics that belong to the interval, and by x(ik)* ∈ (x(ikr),x(ik+1l)), k = 1,…,K, those x(i)*-points that do not belong to

. Then

The density f (x,θ) is monotone on any interval [x(ikl),x(ikr)]. Suppose it is monotonically increasing. Then the greatest possible value taken by any f (x(i)*), ikl + 1 ≤ i ≤ ikl, is f (x(i)). Thus Λk is bounded from above by −log f (x(ikl)). Analogously, Λk is bounded from below by −log f (x(ikr)). Therefore, |Λk| ≤ |log f (x(ikl))| + |log f (x(ikr))|. Similarly, the same inequality holds if f (x,θ) is monotonically decreasing on the interval considered. Summing over the intervals and rearranging we obtain

By Lemma 1 applied to g(x) = log2 f (x), we have |log f (x(1))| = op(n1/2) and |log f (x(n))| = op(n1/2). The term with the summation is Op(1) because, for all k = 1,…,K,

and K is finite.

To summarize, QMPS(θ) = QML(θ) + A + op(n−1/2), where A does not depend on θ. Therefore,

. █

The asymptotic equivalence of the two estimators implies that the asymptotic or bootstrap inference about θ0 on the basis of the MPS estimator may be carried out using the ML asymptotics. In small samples the unpleasant behavior of the ML estimator is well known. Using the alternative MPS criterion one may be able to improve the small sample performance of the ML estimator without changing the asymptotic distribution, as the following example attests.

Example

The MPS estimator can be expected to be more efficient than the ML estimator in small samples when the density is skewed and/or the tails are heavy. Consider the exponential distribution

and let the true value λ0 be 1. The mean squared error of λMPS obtained from 10,000 simulations is 65.4%, 73.1%, and 85.2% that of λML for sample sizes of 6, 10, and 20, respectively. Very similar comparisons emerge for the Pareto distribution

with the true value λ0 being 1.

Of course, the efficiency property is not universal: the MPS and ML estimators are equally efficient when a mean of a normal distribution with known variance is estimated, whereas when a standard deviation of a centered normal distribution is estimated, the mean squared error of the MPS estimator exceeds that of the ML estimator by a factor that ranges from 2.7 to 1.4 as the sample size increases from 6 to 50.

As far as extensions of the MPS idea are concerned, Lind (1994) provides a rationale for the MPS in the information theory, Ekström (1997) suggests use of “high order” spacings, Ghosh and Jammalamadaka (2001) expand the class to contain estimators with objective functions based on various divergence criteria (interestingly, the MPS estimator is the only one asymptotically efficient in the extended class), and Ranneby, Jammalamadaka, and Teterukovskiy (2004) propose a generalization to multivariate situations.

References

REFERENCES

Cheng, R.C.H. & N.A.K. Amin (1983) Estimating parameters in continuous univariate distributions with a shifted origin. Journal of the Royal Statistical Society, Series B 45, 394–403.Google Scholar
Ekström, M. (1997) Generalized Maximum Spacing Estimators. Manuscript, Umeå University.
Feller, W. (1971) An Introduction to Probability Theory and Its Applications. Wiley.
Ghosh, K. & S.R. Jammalamadaka (2001) A general estimation method using spacings. Journal of Statistical Planning and Inference 93, 71–82.Google Scholar
Lind, N.C. (1994) Information theory and maximum product of spacings estimation. Journal of the Royal Statistical Society, Series B 56, 341–343.Google Scholar
Ranneby, B. (1984) The maximum spacing method: An estimation method related to the maximum likelihood method. Scandinavian Journal of Statistics 11, 93–112.Google Scholar
Ranneby, B., S.R. Jammalamadaka, & A. Teterukovskiy (2004) The Maximum Spacing Estimate for Multivariate Observations. Manuscript, Swedish University of Agricultural Sciences.
Shao, Y. & M. Hahn (1999) Strong consistency of the maximum product of spacings estimates with applications in nonparametrics and in estimation of unimodal densities. Annals of the Institute of Statistical Mathematics 51, 31–49.Google Scholar
Titterington, D.M. (1985) Comment on “Estimating parameters in continuous univariate distributions.” Journal of the Royal Statistical Society, Series B 47, 115–116.Google Scholar
van der Vaart, A.W. (2000) Asymptotic Statistics. Cambridge University Press.