1. INTRODUCTION
Since the classic papers of Akaike (1973), White (1982), and Vuong (1989), there has existed a growing literature devoted to the study of misspecified models. Furthermore, during the last decade, the “sandwich” variance matrix (also known as the “robust” variance matrix) has been shown to be the proper variance matrix in misspecified models and has been widely used. The sandwich variance matrix estimation procedure was introduced by Huber (1967) and White (1982), and it yields consistent variance matrix estimators, also (and in particular) when the assumed model is misspecified.
The objective of this paper is to derive the analytical closed-form expression of the sandwich variance matrix within the context of the misspecified multivariate regression model. We also derive scalar measures of the asymptotic variance, in particular the trace, determinant, and norm, that play a role in the construction of information criteria. An example of such an application is provided in Bozdogan (2007), where the information complexity (ICOMP) criterion is used to extend the results of Bozdogan and Haughton (1998) from the univariate misspecified regression model to the multivariate case.
2. MULTIVARIATE NORMAL REGRESSION
Consider a set of n vectors y1,…, yn, each of order p × 1, whose first two moments are given by
where B is a k × p matrix of unknown coefficients, X := (x1,…, xn)′ is a nonrandom n × k matrix of full column rank k, and Σ = (σij) is a positive definite unknown p × p matrix. The full set of coefficients is thus θ := ((vec B)′,(vech(Σ))′)′, of order (kp + ½p(p + 1)) × 1, where vech(·) denotes the half-vec operator defined in the Appendix. Assume that yi and yj are uncorrelated for all i ≠ j and let Y := (y1,…, yn)′, of order n × p. Finally, let n ≥ p + k; this is a necessary condition without which the estimator
in (3), which follows, would be singular. These assumptions imply that
If, in addition, we assume normality, then the log-likelihood function of the sample y1,…, yn is given by
see, for example, Magnus and Neudecker (1988, p. 321). The first differential of the log-likelihood is
leading to the first-order conditions
and hence to the maximum likelihood estimators
where M := In − X(X′X)−1X′ is the usual idempotent matrix.
Taking the differential of (2), we obtain the second differential of the log-likelihood as
Then, using the facts that E(Y − XB) = 0 and E(Y − XB)′(Y − XB) = nΣ, we find
where Dp denotes the p2 × ½p(p + 1) duplication matrix, defined in the Appendix. Hence we obtain the following result.
THEOREM 1. In the correctly specified case, the information matrix is given by
and its inverse by
where Dp+ = (Dp′Dp)−1Dp′. Furthermore,
and
Proof. The information matrix
follows from the fact that we can write (4) as
. Its inverse follows from Magnus and Neudecker (1988, Thm. 3.13(d), p. 50), and the trace and determinant follow from Lemma A1 in the Appendix. █
The inverse
of the information matrix provides the asymptotic variance of the maximum likelihood estimator in the correctly specified case. Its trace and determinant provide scalar measures of the asymptotic variance, and they play a role, inter alia, in the construction of information criteria.
3. MULTIVARIATE REGRESSION UNDER MISSPECIFICATION
We next assume the same model as in Section 2, except that we do not assume normality. The first two moments of Y are still given by E(Y) = XB and var(vec Y) = Σ [otimes ] In, but the third and fourth moments of Y are not necessarily equal to the moments that would have been implied by normality.
We estimate the unknown parameters by pseudo maximum likelihood (PML); that is, we take the normal log-likelihood function (1) as our starting point. The PML estimators are given by (3). The expectation of the first differential is still zero (first-order regularity), but it is no longer true that E(d[ell ])2 = −E d2[ell ] (second-order regularity). This is because the evaluation of E(d[ell ])2 involves third and fourth moments.
Let us standardize Y by defining V := (Y − XB)Σ−1/2, so that
Let us also introduce matrix generalizations of the usual skewness and kurtosis measures by defining
In the special case of correct specification, this specializes to
where Np denotes the p2 × p2 symmetrizer matrix defined in the Appendix. If n = p = 1, the kurtosis further specializes to Γ2 = 3, as expected.
We now evaluate E(d[ell ])2. Squaring equation (2) gives
Then, letting Δ := Dp′(Σ−1/2 [otimes ] Σ−1/2)Dp, we find
Thus we obtain the following result.
THEOREM 2. In the misspecified case, we have
where
is given in Theorem 1,
and Γ2* := Γ2 − n2(vec Ip)(vec Ip)′.
Proof. The expression −E(d2[ell ]) is not affected by the misspecification because it uses the first two moments only. Hence the matrix
is the same as in Theorem 1. In contrast, equation (5) implies the expression for
. █
The matrix
is sometimes called the “outer-product form” of the information matrix because it is based on E(d[ell ])2. The “Hessian form”
is based on −E(d2[ell ]). In the correctly specified case where Γ1 = 0 and Γ2* = 2nNp, one verifies that
.
4. ASYMPTOTIC VARIANCE OF THE PML ESTIMATOR
We have seen that, in the presence of misspecification, second-order regularity does not hold and that therefore
and
are not the same. The asymptotic variance of the PML estimator
is therefore not given by either
or
but rather by
. This important result was implied or proved in papers by Huber (1967), Jennrich (1969), Malinvaud (1970), Gallant and Holly (1980), Burguete, Gallant, and Souza (1982), White (1982), and Gouriéroux, Monfort, and Trognon (1984) and more recently by Gouriéroux and Monfort (1995a, p. 237), Gouriéroux and Monfort (1995b, p. 170), Hendry (1995, p. 391), and White (1996).
Although the sandwich matrix
evaluated at the maximum likelihood estimator
provides a consistent estimator of the variance of
, it is not the only consistent estimator. An alternative would be to evaluate minus the Hessian matrix H (instead of
) and the sample variance of the score contributions R (instead of
) and to use these in constructing V := H−1RH−1, as in White (1982). The estimator
is also consistent and hence an alternative to
. It is difficult to judge, in general, which estimator is to be preferred. In our case, the alternative estimator
allows for heteroskedasticity and “heteroskewness” which is excluded by our model assumptions, and this might be one reason to prefer
over
. Our main result is as follows.
THEOREM 3. The sandwich matrix
is given by
The trace and determinant of
are
and
and the norm of
, defined as
, is the square root of
where
and
. The vectors ei are unit vectors, so that ei denotes the ith column of the identity matrix Ip.
Proof. From Theorems 1 and 2 it follows that the matrix
is equal to
so that the expression for
follows from the properties of Np and Dp and the fact that Dp+′ΔDp+(Σ [otimes ] Σ)Dp+′ = Dp Δ−1. Furthermore,
and
Next we compute
. Denote the four blocks of
by
. Then,
Now,
and, using Lemma A2 in the Appendix,
and
This completes the proof. █
The sandwich matrix
thus provides the asymptotic variance of the PML estimator in the misspecified case. As in Theorem 1, its trace, determinant, and norm provide scalar measures of the asymptotic variance. These measures, together with other scalars such as
, play a crucial role in the construction of information criteria.
An interesting special case is obtained when the true joint distribution belongs to the linear exponential family, giving rise to the well-known quasi-generalized PML estimators; see Gouriéroux et al. (1984, Sect. 5). We do not, however, investigate this avenue in this paper.
We notice, after a little algebra, that
which simplifies to
in the special case of correct specification where Γ1 = 0 and Γ2* = 2nNp.
We also notice that, in the case of correct specification,
APPENDIX: The Duplication Matrix— Some New Results
Let A be a square matrix of order p × p. The two vectors vec A and vec A′ contain the same p2 components but in a different order. Hence there exists a unique permutation matrix that transforms vec A into vec A′. This p2 × p2 matrix is (a special case of) the commutation matrix and is denoted by Kp; it is implicitly defined by the operation Kp vec A = vec A′.
Closely related to the commutation matrix is the p2 × p2 symmetrizer matrix Np with the property Np vec A = ½ vec(A + A′) for every square p × p matrix A. It is easy to see that Np = ½(Ip2 + Kp).
We now introduce the half-vec operator vech(·). For any p × p matrix A, the vector vech(A) denotes the ½p(p + 1) × 1 vector that is obtained from vec A by eliminating all supradiagonal elements of A. For example, for p = 2,
where the supradiagonal element a12 has been removed. Thus, for symmetric A, vech(A) only contains the distinct elements of A. Now, if A is symmetric, the elements of vec A are those of vech(A) with some repetitions. Hence, there exists a unique p2 × ½p(p + 1) matrix Dp, called the duplication matrix, that transforms, for symmetric A, vech(A) into vec A, that is,
The matrices Dp and Np are connected through Dp Dp+ = Np. The duplication matrix was introduced by Magnus and Neudecker (1980). A systematic treatment of Kp, Np, and Dp, among others, is given in Magnus (1988).
We now present two new properties, both of which are used in this note.
LEMMA A1. Let A = (aij) be a square matrix of order p × p. The determinant and trace of the matrix Dp+(A [otimes ] A)Dp+′ are given by
and
Proof. Because
we obtain, from Magnus (1988, Thm. 4.11(i)),
This proves the first result. To prove the second result, let δst denote the Kronecker delta and write uij = vech(ei ej′), where ei denotes the ith column of the identity matrix Ip. Then,
where the second equality follows from the proof of Theorem 4.9 and Theorem 4.4(ii) in Magnus (1988). █
LEMMA A2. Letting
and αk := ½k, we have
a weighted average of two idempotent matrices.
Proof. We prove the result first for k = 1. Let Sij := (ei ej′ + ej ei′)/2. Then, using Theorem 4.6(ii) of Magnus (1988),
because
by Theorem 3.2 in Magnus (1988). This proves the result for k = 1. The general result follows by induction, using the facts that both Np and Ξp are idempotent and that NpΞp = Ξp Np = Ξp. █
