Hostname: page-component-7b7567c8c-fvjsw Total loading time: 0 Render date: 2026-11-15T04:34:23.881Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayesian Mixture Model of Extended Redundancy Analysis

Published online by Cambridge University Press:  01 January 2025

Minjung Kyung,
Ju-Hyun Park and
Ji Yeh ChoiOpen the ORCID record for Ji Yeh Choi[Opens in a new window]
Minjung Kyung
Affiliation:
Duksung Women’s UniversityDuksung Women’s University
Ju-Hyun Park
Affiliation:
Dongguk University
Ji Yeh Choi*
Affiliation:
York University
*
Correspondence should be made to Ji Yeh Choi, Department of Psychology, York University, 4700 Keele St., Toronto, ON, Canada. Email: jychoi@yorku.ca
Get accessRights & Permissions [Opens in a new window]

Abstract

Extended redundancy analysis (ERA), a generalized version of redundancy analysis (RA), has been proposed as a useful method for examining interrelationships among multiple sets of variables in multivariate linear regression models. As a limitation of the extant RA or ERA analyses, however, parameters are estimated by aggregating data across all observations even in a case where the study population could consist of several heterogeneous subpopulations. In this paper, we propose a Bayesian mixture extension of ERA to obtain both probabilistic classification of observations into a number of subpopulations and estimation of ERA models within each subpopulation. It specifically estimates the posterior probabilities of observations belonging to different subpopulations, subpopulation-specific residual covariance structures, component weights and regression coefficients in a unified manner. We conduct a simulation study to demonstrate the performance of the proposed method in terms of recovering parameters correctly. We also apply the approach to real data to demonstrate its empirical usefulness.

Keywords

Bayesianextended redundancy analysisfinite mixture modelclustering

Information

Type
Theory & Methods
Information
Psychometrika , Volume 87 , Issue 3 , September 2022, pp. 946 - 966
Copyright
Copyright © 2021 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s11336-021-09809-7.

References

Anderson, T. W., Estimating linear restrictions on regression coefficients for multivariate normal distributions Annals of Mathematical Statistics (1951). 22(3) 327–35110.1214/aoms/1177729580CrossRefGoogle Scholar
Benaglia, T., Chauveau, D., & Hunter, D., Young, D., mixtools: An R package for analyzing mixture models Journal of Statistical Software (2009). 32(6) 1–2910.18637/jss.v032.i06CrossRefGoogle Scholar
Celeux, G., Forbes, F., Robert, C. P., & Titterington, D. M., (2006). Deviance information criteria for missing data models. Bayesian Analysis, 1(4), 651–673. https://10.1214/06-BA122 10.1214/06-BA122CrossRefGoogle Scholar
Choi, J. Y., Kyung, M., Hwang, H., Park, J.-H., (2019). Bayesian extended redundancy analysis: A Bayesian approach to component-based regression with dimension reduction Multivariate Behavioral ResearchGoogle ScholarPubMed
Dauvier, B., Chevalier, N., & Blaye, A., (2012). Using finite mixture of GLMs to explore variability in children’s flexibility in a task-switching paradigm. Cognitive Development, 27(4), 440–454. https://10.1016/j.cogdev.2012.07.004CrossRefGoogle Scholar
Davies, P. T., Tso, M.K.-S., (1982). Procedures for reduced-rank regression Applied Statistics 31 244–25510.2307/2347998CrossRefGoogle Scholar
Dempster, A. P., Laird, N. M., Rubin, D. B., (1977). Maximum likelihood from incomplete data via the EM algorithm Journal of the Royal Statistical Society: Series B (Methodological) 39(1) 1–22CrossRefGoogle Scholar
Diebolt, J., Robert, C. P., (1994). Estimation of finite mixture distributions through Bayesian sampling Journal of the Royal Statistical Society: Series B (Methodological) 56(2) 363–37510.1111/j.2517-6161.1994.tb01985.xCrossRefGoogle Scholar
Drton, M., Plummer, M., (2017). A Bayesian information criterion for singular models Journal of the Royal Statistical Society: Series B (Methodological) 79(2) 323–38010.1111/rssb.12187CrossRefGoogle Scholar
Ferguson, T., (1973). A Bayesian analysis of some nonparametric problems Annals of Statistics 1(2) 209–230CrossRefGoogle Scholar
Fritsch, A., Ickstadt, K., (2009). Improved criteria for clustering based on the posterior similarity matrix Bayesian Analysis 4 367–392CrossRefGoogle Scholar
Fraley, C., Raftery, A. E., (2002). Model-based clustering, discriminant analysis, and density estimation Journal of the American Statistical Association 458 611–63110.1198/016214502760047131CrossRefGoogle Scholar
Frühwirth-Schnatter, S., Finite Mixture and Markov Switching Models, Springer Series in Statistics (2006). New York: SpringerGoogle Scholar
Frühwirth-Schnatter, S., Malsiner-Walli, G., (2019). From here to infinity: Sparse finite versus Dirichlet process mixtures in model-based clustering Advances in Data Analysis and Classification 13 33–64CrossRefGoogle ScholarPubMed
Hwang, H., DeSarbo, S. W., Takane, Y., (2007). Fuzzy clusterwise generalized structured component analysis Psychometrika 72 181–198CrossRefGoogle Scholar
Hwang, H., Suk, H. W., Lee, J-H., Moskowitz, D. S., Lim, J., (2012). Functional extended redundancy analysis Psychometrika 77(3) 524–54210.1007/s11336-012-9268-2CrossRefGoogle ScholarPubMed
Hwang, H., Suk, H. W., Takane, Y., Lee, J-H., Lim, J., (2015). Generalized functional extended redundancy analysis Psychometrika 80 101–125CrossRefGoogle ScholarPubMed
Hotelling, H., (1957). The relations of the newer multivariate statistical methods to factor analysis British Journal of Mathematical and Statistical Psychology 10(2) 69–7910.1111/j.2044-8317.1957.tb00179.xCrossRefGoogle Scholar
Ishwaran, H., Zarepour, M., (2000). Markov chain Monte Carlo in approximate Dirichlet and beta two-parameter process hierarchical models Biometrika 87 371–390CrossRefGoogle Scholar
Jolliffe, I. T., (1982). A note on the use of principal components in regression Applied StatisticsCrossRefGoogle Scholar
Kamakura, W. A., Kim, B. D., Lee, J., (1996). Modeling preference and structural heterogeneity in consumer choice Marketing Science 15(2) 152–172CrossRefGoogle Scholar
Kamakura, W. A., Russell, G., (1989). A probabilistic choice model for market segmentation and elasticity structure Journal of Marketing Research 26 379–39010.1177/002224378902600401CrossRefGoogle Scholar
Kok, B. C., Choi, J. S., Oh, H., Choi, J. Y., (2019). Sparse extended redundancy analysis: Variable selection via the exclusive lasso Multivariate Behavioral ResearchGoogle ScholarPubMed
Liu, J., Wong, W., Kong, A., (1994). Covariance structure of the Gibbs sampler with application to the comparisons of estimators and sampling schemes Biometrika 81(1) 27–4010.1093/biomet/81.1.27CrossRefGoogle Scholar
Lovaglio, P. G., Vittadini, G., (2014). Structural equation models in a redundancy analysis framework with covariates Multivariate Behavioral Research 49(5) 486–50110.1080/00273171.2014.931798CrossRefGoogle Scholar
Malsiner-Walli, G., Frühwirth-Schnatter, S., Grün, B., (2016). Model-based clustering based on sparse finite Gaussian mixtures Statistics and Computing 26 303–32410.1007/s11222-014-9500-2CrossRefGoogle ScholarPubMed
McCullagh, P., Nelder, J. A., Generalized linear models London: Chapman and HallGoogle Scholar
McLachlan, G. J., Peel, D., Finite mixture models (2000). New York: WileyCrossRefGoogle Scholar
Mengersen, K. L., Robert, C. P., Titterington, D. M., Mixtures: Estimation and applications (2011). New York: WileyCrossRefGoogle Scholar
Park, J-H., Choi, J. Y., Lee, J., Kyung, M., (1983). Bayesian approach to multivariate component-based logistic regression: analyzing correlated multivariate ordinal data Multivariate Behavioral Research (2021).Google Scholar
Pouwels, J. L., Cillessen, A. H. N., (2013). Correlates and outcomes associated with aggression and victimization among elementary-school children in a low-income urban context Journal of Youth and Adolescence 42 190–20510.1007/s10964-012-9875-3CrossRefGoogle Scholar
Raftery, A. E., (1995). Bayesian model selection in social research Sociological Methodology 25 111–16310.2307/271063CrossRefGoogle Scholar
Ramsay, J., Hooker, G., Graves, S., Functional data analysis with R and Matlab Berlin: Springer 10.1007/978-0-387-98185-7CrossRefGoogle Scholar
Richardson, S., Green, P. J., (2009). On Bayesian analysis of mixtures with an unknown number of components (with discussion) Journal of the Royal Statistical Society: Series B (Statistical Methodology) (1997). 59 731–79210.1111/1467-9868.00095CrossRefGoogle Scholar
Rousseau, J., Mengersen, K., (2011). Asymptotic behaviour of the posterior distribution in overfitted mixture models Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73(5) 689–71010.1111/j.1467-9868.2011.00781.xCrossRefGoogle Scholar
Schwarz, G., (1978). Estimating the dimension of a model Annals of Statistics 6(2) 461–46410.1214/aos/1176344136CrossRefGoogle Scholar
Scrucca, L., Fop, M., Murphy, T. B., Raftery, A. E., (2016). mclust 5: Clustering, classification and density estimation using Gaussian finite mixture models The R Journal 8(1) 289–31710.32614/RJ-2016-021CrossRefGoogle ScholarPubMed
Steele, R. J., Raftery, A. E., (2010). Performance of Bayesian model selection criteria for Gaussian mixture models Frontiers of Statistical Decision Making and Bayesian Analysis 2 113–130Google Scholar
Stephens, M., (2000). Bayesian analysis of mixture models with an unknown number of components—An alternative to reversible jump methods The Annals of Statistics 28 40–7410.1214/aos/1016120364CrossRefGoogle Scholar
Swait, J., Sweeney, J. C., (2000). Perceived value and its impact on choice behavior in a retail setting Journal of Retailing and Consumer Services 7(2) 77–88CrossRefGoogle Scholar
Takane, Y., Hwang, H., (2005). An extended redundancy analysis and its applications to two practical examples Computational Statistics & Data Analysis 49(3) 785–808CrossRefGoogle Scholar
Tanner, M. D., Wong, W., (1987). The calculation of posterior distribution by data augmentation (with discussion) Journal of the American Statistical Association 82 528–55010.1080/01621459.1987.10478458CrossRefGoogle Scholar
van den Wollenberg, A. L., (1977). Redundancy analysis an alternative for canonical correlation analysis Psychometrika 42(2) 207–219CrossRefGoogle Scholar
Vehtari, A., Gelman, A. & Gabry, J., (2017). Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Statistics and Computing, 27, 1413–1432. https://10.1007/s11222-016-9696-4 10.1007/s11222-016-9696-4CrossRefGoogle Scholar
Viele, K., Tong, B., (2002). Modeling with mixtures of linear regressions Statistics and Computing 12 315–33010.1023/A:1020779827503CrossRefGoogle Scholar
Wang, J. M., Duong, M., Schwartz, D., Chang, L., Luo, T., (2014). Interpersonal and personal antecedents and consequences of peer victimization across middle childhood in Hong Kong Journal of Youth and Adolescence 43 1934–1945CrossRefGoogle ScholarPubMed
Watanabe, S., (2013). A widely applicable Bayesian information criterion Journal of Machine Learning Research 14 867–897Google Scholar
Watanabe, S., (2021). WAIC and WBIC for mixture models Behaviormetrika 48 5–2110.1007/s41237-021-00133-zCrossRefGoogle Scholar
Wedel, M., Market segmentation: Conceptual and methodological foundations Berlin: SpringerGoogle Scholar
Wold, H., Krishnaiah, P. R., (2000). Estimation of principal components and related methods by iterative least squares Multivariate analysis (1966). New York: Academic Press 391–420Google Scholar
Wold, H., (1973). Nonlinear iterative partial least squares (NIPALS) modeling: some current developments. In: Krishnaiah, P.R., (Ed.), Multivariate Analysis (pp. 383–487). New York: Academic Press. https://10.1016/B978-0-12-426653-7.50032-6.CrossRefGoogle Scholar
Supplementary material: File

Gische and Voelkle supplementary material

Gische and Voelkle supplementary material
Download Gische and Voelkle supplementary material(File)
File 990.2 KB