Hostname: page-component-7b7567c8c-jswmv Total loading time: 0 Render date: 2026-11-15T12:01:49.097Z Has data issue: false hasContentIssue false
  • English
  • Français

Empirical Correction to the Likelihood Ratio Statistic for Structural Equation Modeling with Many Variables

Published online by Cambridge University Press:  01 January 2025

Ke-Hai Yuan,
Yubin Tian and
Hirokazu Yanagihara
Ke-Hai Yuan*
Affiliation:
University of Notre Dame
Yubin Tian
Affiliation:
Beijing Institute of Technology
Hirokazu Yanagihara
Affiliation:
Hiroshima University
*
Requests for reprints should be sent to Ke-Hai Yuan, Department of Psychology, University of Notre Dame, Notre Dame, IN 46556, USA. E-mail: kyuan@nd.edu
Get accessRights & Permissions [Opens in a new window]

Abstract

Survey data typically contain many variables. Structural equation modeling (SEM) is commonly used in analyzing such data. The most widely used statistic for evaluating the adequacy of a SEM model is TML, a slight modification to the likelihood ratio statistic. Under normality assumption, TML approximately follows a chi-square distribution when the number of observations (N) is large and the number of items or variables (p) is small. However, in practice, p can be rather large while N is always limited due to not having enough participants. Even with a relatively large N, empirical results show that TML rejects the correct model too often when p is not too small. Various corrections to TML have been proposed, but they are mostly heuristic. Following the principle of the Bartlett correction, this paper proposes an empirical approach to correct TML so that the mean of the resulting statistic approximately equals the degrees of freedom of the nominal chi-square distribution. Results show that empirically corrected statistics follow the nominal chi-square distribution much more closely than previously proposed corrections to TML, and they control type I errors reasonably well whenever N≥max(50,2p). The formulations of the empirically corrected statistics are further used to predict type I errors of TML as reported in the literature, and they perform well.

Keywords

Bartlett correctionBayesian information criterionmaximum likelihoodtype I errors

Information

Type
Original Paper
Information
Psychometrika , Volume 80 , Issue 2 , June 2015, pp. 379 - 405
Copyright
Copyright © 2013 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Amemiya, Y., & Anderson, T.W. (1990). Asymptotic chi-square tests for a large class of factor analysis models. The Annals of Statistics, 18, 1453–1463.CrossRefGoogle Scholar
Anderson, J.C., & Gerbing, D.W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155–173.CrossRefGoogle Scholar
Anderson, J.C., & Gerbing, D.W. (1988). Structural equation modeling in practice: a review and recommended two-step approach. Psychological Bulletin, 103, 411–423.CrossRefGoogle Scholar
Barndorff-Nielsen, O.E., & Hall, P. (1988). On the level-error after Bartlett adjustment of the likelihood ratio statistic. Biometrika, 75, 374–378.CrossRefGoogle Scholar
Bartlett, M.S. (1937). Properties of sufficiency and statistical tests. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 160, 268–282.Google Scholar
Bartlett, M.S. (1951). The effect of standardization on a χ 2 approximation in factor analysis. Biometrika, 38, 337–344.Google Scholar
Bartlett, M.S. (1954). A note on the multiplying factors in various χ 2 approximations. Journal of the Royal Statistical Society, Series B, 16, 296–298.CrossRefGoogle Scholar
Bentler, P.M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246.CrossRefGoogle ScholarPubMed
Bentler, P.M. (2008). EQS 6 structural equations program manual. Encino, CA: Multivariate Software.Google Scholar
Bentler, P.M., & Bonett, D.G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.CrossRefGoogle Scholar
Bentler, P.M., & Yuan, K.-H. (1999). Structural equation modeling with small samples: test statistics. Multivariate Behavioral Research, 34, 181–197.CrossRefGoogle ScholarPubMed
Beran, R. (1988). Prepivoting test statistics: a bootstrap view of asymptotic refinements. Journal of the American Statistical Association, 83, 687–697.CrossRefGoogle Scholar
Bollen, K.A., & Stine, R. (1993). Bootstrapping goodness of fit measures in structural equation models. In Bollen, K.A., & Long, J.S. (Eds.), Testing structural equation models (pp. 111–135). Newbury Park: Sage.Google Scholar
Boomsma, A. (1982). The robustness of LISREL against small sample sizes in factor analysis models. In Jöreskog, K.G., & Wold, H. (Eds.), Systems under indirect observation: causality, structure, prediction (Part I) (pp. 149–173). Amsterdam: North-Holland.Google Scholar
Box, G.E.P. (1949). A general distribution theory for a class of likelihood criteria. Biometrika, 36, 317–346.CrossRefGoogle ScholarPubMed
Browne, M.W., MacCallum, R.C., Kim, C.-T., Andersen, B.L., & Glaser, R. (2002). When fit indices and residuals are incompatible. Psychological Methods, 7, 403–421.CrossRefGoogle ScholarPubMed
Casella, G., & Berger, R.L. (2002). Statistical inference (2nd ed.). Pacific Grove: Duxbury Press.Google Scholar
Cox, D.R., & Hinkley, D.V. (1974). Theoretical statistics. Boca Raton: Chapman and Hall.CrossRefGoogle Scholar
Curran, P.J., West, S.G., & Finch, J.F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16–29.CrossRefGoogle Scholar
Efron, B., & Tibshirani, R.J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.CrossRefGoogle Scholar
Evans, M., Hastings, N., & Peacock, B. (2000). Statistical distributions (3rd ed.). New York: Wiley.Google Scholar
Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 56–83.CrossRefGoogle Scholar
Fouladi, R.T. (2000). Performance of modified test statistics in covariance and correlation structure analysis under conditions of multivariate nonnormality. Structural Equation Modeling, 7, 356–410.CrossRefGoogle Scholar
Fujikoshi, Y. (2000). Transformations with improved chi-squared approximations. Journal of Multivariate Analysis, 72, 249–263.CrossRefGoogle Scholar
Geweke, J.F., & Singleton, K.J. (1980). Interpreting the likelihood ratio statistic in factor models when sample size is small. Journal of the American Statistical Association, 75, 133–137.CrossRefGoogle Scholar
Hall, P. (1992). The bootstrap and Edgeworth expansion. New York: Springer.CrossRefGoogle Scholar
Herzog, W., & Boomsma, A. (2009). Small-sample robust estimators of noncentrality-based and incremental model fit. Structural Equation Modeling, 16, 1–27.CrossRefGoogle Scholar
Herzog, W., Boomsma, A., & Reinecke, S. (2007). The model-size effect on traditional and modified tests of covariance structures. Structural Equation Modeling, 14, 361–390.CrossRefGoogle Scholar
Holzinger, K.J., & Swineford, F. (1939). A study in factor analysis: the stability of a bi-factor solution. Chicago: University of Chicago Press.Google Scholar
Hu, L.T., Bentler, P.M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted?. Psychological Bulletin, 112, 351–362.CrossRefGoogle ScholarPubMed
Ichikawa, M., & Konishi, S. (1995). Application of the bootstrap methods in factor analysis. Psychometrika, 60, 77–93.CrossRefGoogle Scholar
Jackson, D.L. (2001). Sample size and number of parameter estimates in maximum likelihood confirmatory factor analysis: a Monte Carlo investigation. Structural Equation Modeling, 8, 205–223.CrossRefGoogle Scholar
Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.CrossRefGoogle Scholar
Kano, Y. (1992). Robust statistics for test-of-independence and related structural models. Statistics & Probability Letters, 15, 21–26.CrossRefGoogle Scholar
Lawley, D.N. (1956). A general method for approximating to the distribution of likelihood ratio criteria. Biometrika, 43, 295–303.CrossRefGoogle Scholar
Lawley, D.N., & Maxwell, A.E. (1971). Factor analysis as a statistical method (2nd ed.). New York: Elsevier.Google Scholar
MacCallum, R.C., & Austin, J.T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201–226.CrossRefGoogle ScholarPubMed
Mardia, K.V. (1970). Measure of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530.CrossRefGoogle Scholar
McDonald, R.P., & Marsh, H.W. (1990). Choosing a multivariate model: noncentrality and goodness of fit. Psychological Bulletin, 107, 247–255.CrossRefGoogle Scholar
Mooijaart, A., & Bentler, P.M. (1991). Robustness of normal theory statistics in structural equation models. Statistica Neerlandica, 45, 159–171.CrossRefGoogle Scholar
Moshagen, M. (2012). The model size effect in SEM: inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling, 19, 86–98.CrossRefGoogle Scholar
Nevitt, J., & Hancock, G.R. (2004). Evaluating small sample approaches for model test statistics in structural equation modeling. Multivariate Behavioral Research, 39, 439–478.CrossRefGoogle Scholar
Ogasawara, H. (2010). Asymptotic expansions of the null distributions of discrepancy functions for general covariance structures under nonnormality. American Journal of Mathematical and Management Sciences, 30, 385–422.CrossRefGoogle Scholar
Okada, K., Hoshino, T., & Shigemasu, K. (2007). Bartlett correction of test statistics in structural equation modeling. Japanese Journal of Educational Psychology, 55, 382–392. (in Japanese).Google Scholar
Satorra, A., & Bentler, P.M. (1990). Model conditions for asymptotic robustness in the analysis of linear relations. Computational Statistics & Data Analysis, 10, 235–249.CrossRefGoogle Scholar
Satorra, A., & Bentler, P.M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In von Eye, A., & Clogg, C.C. (Eds.), Latent variables analysis: applications for developmental research (pp. 399–419). Thousand Oaks: Sage.Google Scholar
Schott, J.R. (2007). A test for the equality of covariance matrices when the dimension is large relative to the sample size. Computational Statistics & Data Analysis, 51, 6535–6542.CrossRefGoogle Scholar
Sörbom, D. (1989). Model modification. Psychometrika, 54, 371–384.CrossRefGoogle Scholar
Srivastava, M.S., & Yanagihara, H. (2010). Testing the equality of several covariance matrices with fewer observations than the dimension. Journal of Multivariate Analysis, 101, 1319–1329.CrossRefGoogle Scholar
Swain, A.J. (1975). Analysis of parametric structures for variance matrices. Doctoral dissertation, University of Adelaide, Australia.Google Scholar
Wakaki, H., Eguchi, S., & Fujikoshi, Y. (1990). A class of tests for a general covariance structure. Journal of Multivariate Analysis, 32, 313–325.CrossRefGoogle Scholar
White, H. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1–25.CrossRefGoogle Scholar
Yuan, K.-H. (2005). Fit indices versus test statistics. Multivariate Behavioral Research, 40, 115–148.CrossRefGoogle ScholarPubMed
Yuan, K.-H., & Bentler, P.M. (1998). Normal theory based test statistics in structural equation modeling. British Journal of Mathematical & Statistical Psychology, 51, 289–309.CrossRefGoogle Scholar
Yuan, K.-H., & Bentler, P.M. (1999). On normal theory and associated test statistics in covariance structure analysis under two classes of nonnormal distributions. Statistica Sinica, 9, 831–853.Google Scholar
Yuan, K.-H., & Bentler, P.M. (1999). F-tests for mean and covariance structure analysis. Journal of Educational and Behavioral Statistics, 24, 225–243.CrossRefGoogle Scholar
Yuan, K.-H., & Chan, W. (2002). Fitting structural equation models using estimating equations: a model segregation approach. British Journal of Mathematical & Statistical Psychology, 55, 41–62.CrossRefGoogle ScholarPubMed
Yuan, K.-H., & Chan, W. (2008). Structural equation modeling with near singular covariance matrices. Computational Statistics & Data Analysis, 52, 4842–4858.CrossRefGoogle Scholar
Yuan, K.-H., & Hayashi, K. (2003). Bootstrap approach to inference and power analysis based on three statistics for covariance structure models. British Journal of Mathematical & Statistical Psychology, 56, 93–110.CrossRefGoogle ScholarPubMed
Yung, Y.-F., & Bentler, P.M. (1996). Bootstrapping techniques in analysis of mean and covariance structures. In Marcoulides, G.A., & Schumacker, R.E. (Eds.), Advanced structural equation modeling: techniques and issues (pp. 195–226). Hillsdale: Erlbaum.Google Scholar
Yung, Y.-F., & Chan, W. (1999). Statistical analyses using bootstrapping: concepts and implementation. In Hoyle, R.H. (Ed.), Statistical strategies for small sample research (pp. 81–105). Thousand Oaks: Sage.Google Scholar