Hostname: page-component-74656f9884-8r8nv Total loading time: 0 Render date: 2026-11-10T17:15:18.812Z Has data issue: false hasContentIssue false
  • English
  • Français

On Normal Theory Based Inference for Multilevel Models with Distributional Violations

Published online by Cambridge University Press:  01 January 2025

Ke-Hai Yuan  and
Peter M. Bentler
Ke-Hai Yuan*
Affiliation:
University of Notre Dame
Peter M. Bentler
Affiliation:
University of California, Los Angeles
*
Requests for reprints should be sent to Ke-Hai Yuan, Department of Psychology, University of Notre Dame, Notre Dame, IN 46556. E-Mail: kyuan@nd.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Data in social and behavioral sciences are often hierarchically organized though seldom normal, yet normal theory based inference procedures are routinely used for analyzing multilevel models. Based on this observation, simple adjustments to normal theory based results are proposed to minimize the consequences of violating normality assumptions. For characterizing the distribution of parameter estimates, sandwich-type covariance matrices are derived. Standard errors based on these covariance matrices remain consistent under distributional violations. Implications of various covariance estimators are also discussed. For evaluating the quality of a multilevel model, a rescaled statistic is given for both the hierarchical linear model and the hierarchical structural equation model. The rescaled statistic, improving the likelihood ratio statistic by estimating one extra parameter, approaches the same mean as its reference distribution. A simulation study with a 2-level factor model implies that the rescaled statistic is preferable.

Keywords

nonnormal datanormal estimating equationrescaled statisticsandwich-type covariancerandom effect ANOVAhierarchical linear modelsstructural equation models

Information

Type
Articles
Information
Psychometrika , Volume 67 , Issue 4 , December 2002 , pp. 539 - 561
Copyright
Copyright © 2002 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.

References

Aitkin, M., & Longford, N. (1986). Statistical modeling issues in school effectiveness studies. Journal of the Royal Statistical Society, Series A, 149, 1–43.CrossRefGoogle Scholar
Amemiya, Y., & Anderson, T.W. (1990). Asymptotic chi-square tests for a large class of factor analysis models. Annals of Statistics, 18, 1453–1463.CrossRefGoogle Scholar
Anderson, T.W., & Amemiya, Y. (1988). The asymptotic normal distribution of estimators in factor analysis under general conditions. Annals of Statistics, 16, 759–771.CrossRefGoogle Scholar
Arnold, S.F. (1981). The theory of linear models and multivariate analysis. New York, NY: Wiley.Google Scholar
Bentler, P.M., & Liang, J. (2002). Two-level mean and covariance structures: Maximum likelihood via an EM algorithm. In Duan, N., & Reise, S. (Eds.), Multilevel modeling: Methodological advances, issues, and applications (pp. 53–70). Mahwah, NJ: Erlbaum.Google Scholar
Bentler, P.M., & Yuan, K.-H. (1999). Structural equation modeling with small samples: Test statistics. Multivariate Behavioral Research, 34, 181–197.CrossRefGoogle ScholarPubMed
Bock, R.D. (1989). Multilevel analysis of educational data. San Diego, CA: Academic Press.Google Scholar
Box, G.E.P. (1954). Some theorems on quadratic forms applied in the study of analysis of variance problems: I. Effect of inequality of variance in the one-way classification. Annals of Mathematical Statistics, 25, 290–302.CrossRefGoogle Scholar
Browne, M.W. (1982). Covariance structures. In Hawkins, D.M. (Eds.), Topics in multivariate analysis (pp. 72–141). Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
Browne, M.W. (1984). Asymptotic distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62–83.CrossRefGoogle ScholarPubMed
Browne, M.W., & Shapiro, A. (1988). Robustness of normal theory methods in the analysis of linear latent variate models. British Journal of Mathematical and Statistical Psychology, 41, 193–208.CrossRefGoogle Scholar
Bryk, A.S., & Raudenbush, S.W. (1992). Hierarchical linear models. Newbury Park, CA: Sage.Google Scholar
Chung, K.L. (1974). A course in probability theory 2nd ed., New York, NY: Academic Press.Google Scholar
Curran, P.S., West, S.G., & Finch, J.F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16–29.CrossRefGoogle Scholar
Efron, B., & Hinkley, D.V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information. Biometrika, 65, 457–487.CrossRefGoogle Scholar
Efron, B., & Tibshirani, R.J. (1993). An introduction to the bootstrap. New York, NY: Chapman & Hall.CrossRefGoogle Scholar
Fears, T.R., Benichou, J., & Gail, M.H. (1996). A reminder of the fallibility of the Wald statistic. Americian Statistician, 50, 226–227.CrossRefGoogle Scholar
Godambe, V.P., & Kale, B.K. (1991). Estimating functions: An overview. In Godambe, V.P. (Eds.), Estimating functions (pp. 3–20). Oxford, England: Oxford University Press.CrossRefGoogle Scholar
Goldstein, H. (1986). Multilevel mixed linear model analysis using iterative generalized least squares. Biometrika, 73, 43–56.CrossRefGoogle Scholar
Goldstein, H. (1995). Multilevel statistical models 2nd ed., London, England: Edward Arnold.Google Scholar
Goldstein, H., & McDonald, R.P. (1988). A general model for the analysis of multilevel data. Psychometrika, 53, 435–467.CrossRefGoogle Scholar
Hu, L., Bentler, P.M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted?. Psychological Bulletin, 112, 351–362.CrossRefGoogle ScholarPubMed
Kendall, M.G., & Stuart, A. (1979). The advanced theory of statistics 4th ed., New York, NY: Macmillan.Google Scholar
Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. London, England: Sage.CrossRefGoogle Scholar
Lange, K. L., Little, R.J.A., & Taylor, J.M.G. (1989). Robust statistical modeling using the t distribution. Journal of the American Statistical Association, 84, 881–896.Google Scholar
Lee, S.-Y. (1990). Multilevel analysis of structural equation models. Biometrika, 77, 763–772.CrossRefGoogle Scholar
Lee, S.-Y., & Poon, W.-Y. (1994). A distribution free approach for analysis of two-level structural equation model. Computational Statistics & Data Analysis, 17, 265–275.Google Scholar
Lee, S.Y., & Poon, W.Y. (1998). Analysis of two-level structural equation models via EM type algorithms. Statistica Sinica, 8, 749–766.Google Scholar
Liang, K.Y., & Zeger, S.L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13–22.CrossRefGoogle Scholar
Longford, N.T. (1987). A fast scoring algorithm for maximum likelihood estimation in unbalanced mixed models with nested random effects. Biometrika, 74, 817–827.CrossRefGoogle Scholar
Longford, N.T. (1993). Regression analysis of multilevel data with measurement error. British Journal of Mathematical and Statistical Psychology, 46, 301–311.CrossRefGoogle Scholar
Longford, N.T. (2000). On estimating standard errors in multilevel analysis. Statistician, 49, 389–398.CrossRefGoogle Scholar
Longford, N.T., & Muthén, B.O. (1992). Factor analysis for clustered observations. Psychometrika, 57, 581–597.CrossRefGoogle Scholar
McDonald, R.P., & Goldstein, H. (1989). Balanced versus unbalanced designs for linear structural relations in two-level data. British Journal of Mathematical and Statistical Psychology, 42, 215–232.CrossRefGoogle Scholar
Magnus, J.R., & Neudecker, H. (1988). Matrix differential calculus with applications in statistics and econometrics. New York, NY: Wiley.Google Scholar
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.CrossRefGoogle Scholar
Muirhead, R.J., & Waternaux, C.M. (1980). Asymptotic distributions in canonical correlation analysis and other multivariate procedures for nonnormal populations. Biometrika, 67, 31–43.CrossRefGoogle Scholar
Muthén, B. (1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22, 376–398.CrossRefGoogle Scholar
Muthén, B. (1997). Latent variable modeling of longitudinal and multilevel data. In Raftery, A. (Eds.), Sociological methodology 1997 (pp. 453–480). Boston, MA: Blackwell Publishers.Google Scholar
Muthén, B., & Kaplan, D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171–189.CrossRefGoogle Scholar
Muthén, B., & Kaplan, D. (1992). A comparison of some methodologies for the factor analysis of non-normal Likert variables: A note on the size of the model. British Journal of Mathematical and Statistical Psychology, 45, 19–30.CrossRefGoogle Scholar
Muthén, B., & Satorra, A. (1995). Complex sample data in structural equation modeling. In Marsden, P.V. (Eds.), Sociological methodology 1995 (pp. 267–316). Cambridge, MA: Blackwell Publishers.Google Scholar
Nunnally, J.C. (1978). Psychometric theory. New York, NY: McGraw-Hill.Google Scholar
Satorra, A. (2001). Goodness of fit testing of structural equation models with multiple group data and nonnormality. In Cudeck, R., du Toit, S., & Sörbom, D. (Eds.), Structural equation modeling: Present and future (pp. 231–256). Lincolnwood, IL: Scientific Software International.Google Scholar
Satorra, A., & Bentler, P.M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. American Statistical Association 1988 Proceedings of Business and Economics Sections (pp. 308–313). Alexandria, VA: American Statistical Association.Google Scholar
Satorra, A., & Bentler, P.M. (1990). Model conditions for asymptotic robustness in the analysis of linear relations. Computational Statistics & Data Analysis, 10, 235–249.CrossRefGoogle Scholar
Satorra, A., & Bentler, P.M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In von Eye, A. & Clogg, C.C. (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Thousand Oaks, CA: Sage.Google Scholar
Shapiro, A., & Browne, M.W. (1987). Analysis of covariance structures under elliptical distributions. Journal of the American Statistical Association, 82, 1092–1097.CrossRefGoogle Scholar
Strenio, J.L.F., Weisberg, H.I., & Bryk, A.S. (1983). Empirical Bayes estimation of individual growth curve parameters and their relationship to covariates. Biometrics, 39, 71–86.CrossRefGoogle ScholarPubMed
Tyler, D.E. (1982). Radial estimates and the test for sphericity. Biometrika, 69, 429–436.CrossRefGoogle Scholar
Tyler, D.E. (1983). Robustness and efficiency properties of scatter matrices. Biometrika, 70, 411–420.CrossRefGoogle Scholar
White, H. (1981). Consequences and detection of misspecified nonlinear regression models. Journal of the American Statistical Association, 76, 419–433.CrossRefGoogle Scholar
Yuan, K.-H., & Bentler, P.M. (1997). Improving parameter tests in covariance structure analysis. Computational Statistics and Data Analysis, 26, 177–198.CrossRefGoogle Scholar
Yuan, K.-H., & Bentler, P.M. (1998). Normal theory based test statistics in structural equation modeling. British Journal of Mathematical and Statistical Psychology, 51, 289–309.CrossRefGoogle Scholar
Yuan, K.-H., & Bentler, P.M. (1999). On normal theory and associated test statistics in covariance structure analysis under two classes of nonnormal distributions. Statistica Sinica, 9, 831–853.Google Scholar
Yuan, K.-H., & Bentler, P.M. (1999). On asymptotic distributions of normal theory MLE in covariance structure analysis under some nonnormal distributions. Statistics & Probability Letters, 42, 107–113.CrossRefGoogle Scholar
Yuan, K.-H., & Bentler, P.M. (2000). Inferences on correlation coefficients in some classes of nonnormal distributions. Journal of Multivariate Analysis, 72, 230–248.CrossRefGoogle Scholar
Yuan, K.-H., & Jennrich, R.I. (1998). Asymptotics of estimating equations under natural conditions. Journal of Multivariate Analysis, 65, 245–260.CrossRefGoogle Scholar