Hostname: page-component-74656f9884-rdgjk Total loading time: 0 Render date: 2026-11-07T13:20:36.647Z Has data issue: false hasContentIssue false
  • English
  • Français

THREE RANK FORMULAS ASSOCIATED WITH THE COVARIANCE MATRICES OF THE BLUE AND THE OLSE IN THE GENERAL LINEAR MODEL

Published online by Cambridge University Press:  22 April 2005

Simo Puntanen ,
George P.H. Styan  and
Yongge Tian
Simo Puntanen
Affiliation:
University of Tampere
George P.H. Styan
Affiliation:
McGill University
Yongge Tian
Affiliation:
University of Alberta
Rights & Permissions [Opens in a new window]

Abstract

In this paper we consider the estimation of the expectation vector Xβ under the general linear model {y,Xβ,σ2V}. We introduce a new handy representation for the rank of the difference of the covariance matrices of the ordinary least squares estimator OLSE(Xβ) (= Hy, say) and the best linear unbiased estimator BLUE(Xβ) (= Gy, say). From this formula some well-known conditions for the equality between Hy and Gy follow at once. We recall that the equality between Hy and Gy can be characterized by the rank-subtractivity ordering between the covariance matrices of y and Hy. This rank characterization suggests a particular presentation for the rank of the difference of the covariance matrices of Hy and Gy. We show, however, that this presentation is valid if and only if the model is connected.

Information

Type
NOTES AND PROBLEMS
Information
Econometric Theory , Volume 21 , Issue 3 , June 2005 , pp. 659 - 663
Copyright
© 2005 Cambridge University Press

1. MOTIVATION AND RESULTS

For the general linear (Gauss–Markov) model {y,Xβ,σ2V}, it is important to know the relationship between the ordinary least squares estimator OLSE(Xβ) and the best (minimum variance) linear unbiased estimator BLUE(Xβ). Here we assume that the model matrix X is n × p with rank rk(X) ≤ min(n,p) and that the covariance matrix V is n × n with rank rk(V) ≤ n.

An extensive overview of the characterizations for the equality between the OLSE(Xβ) and the BLUE(Xβ) is given by Puntanen and Styan (1989); in particular, we have (with probability 1):

where H = XX[dagger] is the orthogonal projector onto the range (column space) of X, and M = I − H. For characterization (1c) see also Baksalary and Puntanen [1990, (c3), p. 366].

The characterizations (1a)–(1c) prompt us to consider the rank of the difference between the two covariance matrices cov[OLSE(Xβ)] and cov[BLUE(Xβ)]. We find that

from which the characterizations (1a) and (1b) follow at once. We note that the commutator HV − VH is skew-symmetric and hence its rank is even (see, e.g., Puntanen, Styan, and Werner, 2001). The special case of the equality in (2b) for V symmetric idempotent was established in Bérubé, Hartwig, and Styan (1993).

In view of (1c), it is also tempting to suggest that

but the equality in (3c) does not hold in general because with

we have rk(HVM) = 1, rk(V − HVH) = 2, rk(V) = 1, rk(HVH) = 1, and so (3b) = 1 but (3c) = 2.

As we will show subsequently

and so equality in (3c) holds if and only if the linear model is connected, i.e.,

(see, e.g., Baksalary and Styan, 1993; Bérubé et al., 1993). The linear model is weakly singular (see, e.g., Zyskind and Martin, 1969), whenever

i.e., the range (column space) of X is contained in the range of V. To see that a weakly singular linear model is connected let

Then

because u ≥ 0 and v ≥ 0. The linear model is weakly singular if and only if w = 0, and connected if and only if u = 0, and clearly w = 0 implies u = 0. For our example (4) we have u = w = 1 and v = 0, and so this example is not connected and hence not weakly singular.

2. PROOFS AND DISCUSSION

Because σ2 plays no role in what follows, we assume that σ2 = 1. Then we recall that (see, e.g., Puntanen and Styan, 1989)

and so

Because M is nonnegative definite, so is MVM, and hence range(MVM) = range(MV) ⊇ range(MVH). Therefore

It was shown by Tian and Styan (2001) that for any matrix A and any pair of idempotent matrices P and Q such that PA − AQ is defined, the following rank equality holds:

Substituting P = Q = H and A = V in (8) gives

which establishes the equality in (2b).

An alternative proof of (9) without using (8) is as follows. Let us assume that H is n × n and has rank h, with 1 ≤ h ≤ n − 1. (When h = 0, then H = 0, and when h = n, then H = I and (1a)–(1c), (2a), and (2b) all trivially hold.) We write

where K = (K1 : K2) is orthogonal and K1 is n × h and K2 is n × (n − h). Hence K2 K2′ = Ih − K1 K1′ = Ih − H and K1′K1 = Ih, K1′K2 = 0 and K2′K2 = In−h, and so

whereas

Combining (10) and (11) yields (9).

To establish the equality in (3c) for a connected model, we introduce the partitioned block matrix

Because the rank of N is invariant under elementary block matrix operations, it follows that

Applying the well-known rank formula (see, e.g., Marsaglia and Styan, 1974),

with PP′ = V and Q = (VH : H) gives

because rk(MV : MVH) = rk(MV(I : H)) = rk(MV). Combining (15) with (12) yields

which is (5b), and so equality in (3c) holds if and only if

i.e., if and only if the linear model is connected (6).

In addition to comparing cov[BLUE(Xβ)] and cov[OLSE(Xβ)], we may also compare the two covariance matrices cov[BLUE(X1 β1)] and cov[OLSE(X1 β1)], where X1 β1 comes from the partition Xβ = X1 β1 + X2 β2 (see, e.g., Puntanen, 1996). We plan to extend the results in this paper to this situation in a further paper.

References

REFERENCES

Baksalary, J.K. & S. Puntanen (1990) Characterizations of the best linear unbiased estimator in the general Gauss–Markov model with the use of matrix partial orderings. Linear Algebra and Its Applications 127, 363–370.Google Scholar
Baksalary, J.K. & G.P.H. Styan (1993) Around a formula for the rank of a matrix product with some statistical applications. In R.S. Rees (ed.), Graphs, Matrices, and Designs: Festschrift in Honor of Norman J. Pullman on His Sixtieth Birthday, Lecture Notes in Pure and Applied Mathematics 139, pp. 1–18. Marcel Dekker.
Bérubé, J., R.E. Hartwig, & G.P.H. Styan (1993) On canonical correlations and the degrees of non-orthogonality in the three-way layout. In Kameo Matusita, Madan L. Puri, & Takesi Hayakawa (eds.), Statistical Sciences and Data Analysis: Proceedings of the Third Pacific Area Statistical Conference, pp. 247–252. VSP International Science Publishers BV.
Marsaglia, G. & G.P.H. Styan (1974) Equalities and inequalities for ranks of matrices. Linear and Multilinear Algebra 2, 269–292.Google Scholar
Puntanen, S. (1996) Some matrix results related to a partitioned singular linear model. Communications in Statistics: Theory and Methods 25, 269–279.Google Scholar
Puntanen, S. & G.P.H. Styan (1989) The equality of the ordinary least squares estimator and the best linear unbiased estimator (with comments by O. Kempthorne and by S.R. Searle and with “Reply” by the authors). American Statistician 43, 153–164.Google Scholar
Puntanen, S., G.P.H. Styan, & H.J. Werner (2001) Determinant of a skew-symmetric matrix: Solution 1 to Problem 00.1.1 (proposed by Steve Lawford). Econometric Theory 17, 277.
Tian, Y. & G.P.H. Styan (2001) Some rank equalities for idempotent and involutory matrices. Linear Algebra and Its Applications 335, 101–117.Google Scholar
Zyskind, G. & F.B. Martin (1969) On best linear estimation and a general Gauss–Markov theorem in linear models with arbitrary nonnegative covariance structure. SIAM Journal on Applied Mathematics 17, 1190–1202.Google Scholar