Hostname: page-component-7b7567c8c-wxvwp Total loading time: 0 Render date: 2026-11-15T08:45:56.911Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayesian Inference for Finite Mixtures of Generalized Linear Models with Random Effects

Published online by Cambridge University Press:  02 January 2025

Peter J. Lenk and
Wayne S. DeSarbo
Peter J. Lenk*
Affiliation:
The University of Michigan Business School
Wayne S. DeSarbo
Affiliation:
Pennsylvania State University
*
Requests for reprints should be sent to Peter J. Lenk, University of Michigan Business School, 701 Tappan Street, Ann Arbor MI 48109-1234.
Get accessRights & Permissions [Opens in a new window]

Abstract

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.

Keywords

Bayesian inferenceconsumer behaviorfinite mixturesgeneralized linear modelsheterogeneitylatent class analysisMarkov chain Monte Carlo

Information

Type
Original Paper
Information
Psychometrika , Volume 65 , Issue 1 , March 2000, pp. 93 - 119
Copyright
Copyright © 2000 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Petrov, N., & Csadki, F. (Eds.), Proceedings of the Second International Symposium of Information Theory (pp. 267–281). Budapest: Akademiai KiadoGoogle Scholar
Albert, J. H., & Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of American Statistical Association, 88, 669–679CrossRefGoogle Scholar
Allenby, G. M., Arora, N., & Ginter, J. L. (1998). On the Heterogeneity of Demand. Journal of Marketing Research, 37, 384–389CrossRefGoogle Scholar
Bozdogan, H. (1987). Model selection and Akaike's information criterion (AIC): The general theory and its analytical extension. Psychometrika, 52, 345–370CrossRefGoogle Scholar
Breslow, N. E., & Clayton, D. G. (1993). Approximate inference in generalized linear mixed models. Journal of American Statistical Association, 88, 8–25CrossRefGoogle Scholar
Carlin, B. P., & Chib, S. (1995). Bayesian model choice via Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series B, 57, 473–484CrossRefGoogle Scholar
Chib, S. (1995). Marginal likelihood from the Gibbs output. Journal of American Statistical Association, 90, 1313–1321CrossRefGoogle Scholar
Chib, S., & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm. The American Statistician, 49, 327–335CrossRefGoogle Scholar
Damien, P., Wakefield, J. C., & Walker, S. (1999). Gibbs sampling for Bayesian nonconjugate and hierarchical models using auxiliary variables. Journal of the Royal Statistical Society, Series B, 61, 331–334Google Scholar
Diebolt, J., & Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, Series B, 56, 362–375CrossRefGoogle Scholar
Dempster, A. P., Laird, N. M., & Rubin, R. B. (1977). Maximum likelihood from incomplete data via the EM-algorithm. Journal of the Royal Statistical Society, Series B, 39, 1–38CrossRefGoogle Scholar
DeSarbo, W. S., Cron, W. L. (1988). A maximum likelihood methodology for clusterwise linear regression. Journal of Classification, 5, 248–282CrossRefGoogle Scholar
DeSarbo, W. S., Wedel, M., Vriens, M., & Ramaswamy, V. (1992). Latent class metric conjoint analysis. Marketing Letters, 3, 273–288CrossRefGoogle Scholar
DeSarbo, W. S., Ramaswamy, V., Reibstein, D. J., Robinson, W. T. (1993). A latent pooling methodology for regression analysis with limited time series of cross sections: A PIMS data application. Marketing Science, 12, 103–124Google Scholar
De Soete, G., DeSarbo, W. S. (1991). A latent class probit model for analyzing pick any/n data. Journal of Classification, 8, 45–63CrossRefGoogle Scholar
Everitt, B. S., Hand, D. J. (1981). Finite mixture distributions. London: Chapman and HallCrossRefGoogle Scholar
Gelfand, A. E., Dey, D. K. (1994). Bayesian model choice: asymptotics and exact calculations. Journal of the Royal Statistical Society, 56, 501–514CrossRefGoogle Scholar
Gelfand, A. E., Smith, A. F. M. (1990). Sampling-based approaches to calculating marginal densities. Journal of American Statistical Association, 85, 398–409CrossRefGoogle Scholar
Gelman, A., Roberts, G. O., Gilks, W. R. (1996). Efficient metropolis jumping rules. In Bernardo, J. M., Berger, J. O., Dawid, A. P., Smith, A. M. F. (Eds.), Bayesian statistics 5 (pp. 599–607). Cambridge: Oxford University PressCrossRefGoogle Scholar
Gelman, A., Rubin, D. B. (1992). Iterative simulation using single and multiple sequences. Statistical Science, 7, 457–511CrossRefGoogle Scholar
Geyer, C. J. (1992). Practical Markov chain Monte Carlo. Statistical Science, 7, 473–511Google Scholar
Goodman, L. A. (1974). Exploratory latent structure analysis using both identified and unidentified models. Biometrika, 61, 215–231CrossRefGoogle Scholar
Hastings, W. K (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97–109CrossRefGoogle Scholar
Hosmer, D. W. (1974). Maximum likelihood estimates of the parameters of a mixture of two regression lines. Communications in Statistics, 3, 995–1006CrossRefGoogle Scholar
Jeffreys, H. (1961). Theory of probability 3rd. ed., Cambridge: Oxford University PressGoogle Scholar
Jones, P. N., McLachlan, G. J. (1992). Fitting finite mixture models in a regression context. Australian Journal of Statistics, 43, 233–240CrossRefGoogle Scholar
Kamakura, W. A. (1991). Estimating flexible distributions of ideal-points with external analysis of preference. Psychometrika, 56, 419–448CrossRefGoogle Scholar
Kamakura, W. A., Russell, G. J. (1989). A probabilistic choice model for market segmentation and elasticity structure. Journal of Marketing Research, 26, 379–390CrossRefGoogle Scholar
Kass, R. E., Raftery, A. E. (1995). Bayes factors. Journal of American Statistical Association, 90, 773–795CrossRefGoogle Scholar
Lenk, P. J., DeSarbo, W. S., Green, P. E., Young, M. R. (1996). Hierarchical Bayes conjoint analysis: Recovery of partworth heterogeneity from reduced experimental designs. Marketing Science, 15(2), 173–191CrossRefGoogle Scholar
Lewis, S. M., Raftery, A. E. (1997). Estimating Bayes factors via posterior simulation with the Laplace—Metropolis estimator. Journal of American Statistical Association, 92, 648–655Google Scholar
Luce, R. D. (1959). Individual choice behavior: A theoretical analysis. New York: John Wiley & SonsGoogle Scholar
Lwin, T., Martin, P. J. (1989). Probits of mixtures. Biometrics, 45, 721–732CrossRefGoogle ScholarPubMed
McCullagh, P., Nelder, J. A. (1983). Generalized linear models. New York: Chapman and HallCrossRefGoogle Scholar
McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In Zarembda, P. (Eds.), Frontiers in econometrics (pp. 105–142). New York: Academic PressGoogle Scholar
Newcomb, S. (1886). A generalized theory of the combination of observations so as to obtain the best result. American Journal of Mathematics, 8, 343–366CrossRefGoogle Scholar
Pearson, K. (1894). Contribution to the mathematical theory of evolution. Philosophical Transactions, 185, 71–110Google Scholar
Polson, Nicholas G. (1996). Convergence of Markov chain Monte Carlo algorithms. In Bernardo, J. M., Berger, J. O., Dawid, A. P., Smith, A. M. F. (Eds.), Bayesian statistics 5 (pp. 297–312). Cambridge: Oxford University PressCrossRefGoogle Scholar
Quandt, R. E. (1972). A new approach to estimating switching regressions. Journal of American Statistical Association, 67, 306–310CrossRefGoogle Scholar
Quandt, R. E., Ramsey, J. B. (1978). Estimating mixtures of normal distributions and switching regression. Journal of American Statistical Association, 73, 730–738CrossRefGoogle Scholar
Roberts, G. O., Polson, N. G. (1994). On the geometric convergence of the Gibbs sampler. Journal of the Royal Statistical Society, 56, 377–384CrossRefGoogle Scholar
Seber, G. A. F. (1984). Multivariate observations. New York: John Wiley & SonsCrossRefGoogle Scholar
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464CrossRefGoogle Scholar
Smith, A. F. M., Roberts, G. O. (1993). Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, 55, 3–23CrossRefGoogle Scholar
Tanner, M. A. (1993). Tools for statistical inference. New York: Springer-VerlagCrossRefGoogle Scholar
Titterington, D. M., Smith, A. F. M., Makov, U. E. (1985). Statistical analysis of finite mixture distributions. New York: John Wiley & SonsGoogle Scholar
Verdinelli, I., Wasserman, L. (1995). Computing Bayes factors using a generalization of the Savage-Dickey density ratio. Journal of American Statistical Association, 90, 614–618CrossRefGoogle Scholar
Wang, P. M., Cockburn, I. M., Puterman, M. L. (1998). ”Analysis of Patent Data—A Mixed Poisson Regression Model. Journal of Business and Economic Statistics, 16, 27–41Google Scholar
Wang, P. M., & Puterman, M. L. (in press). Mixed logistic regression models. Journal of Agricultural, Biological, and Environmental Statistics.Google Scholar
Wang, P. M., Puterman, M. L., Le, N., Cockburn, I. (1996). Mixed Poisson regression with covariate dependent rates. Biometrics, 52, 381–400CrossRefGoogle ScholarPubMed
Wedel, M., DeSarbo, W. S. (1993). A latent class binomial logit methodology for the analysis of paired comparison data: An application reinvestigating the determinants of perceived risk. Decision Science, 24, 1157–1170CrossRefGoogle Scholar
Wedel, M., DeSarbo, W. S. (1994). A Review of recent developments in latent structure regression models. In Bagozzi, R. (Eds.), Advanced methods of marketing research (pp. 352–388). London: Blackwell PublishingGoogle Scholar
Wedel, M., DeSarbo, W. S. (1995). A mixture likelihood approach for generalized linear models. Journal of Classification, 12, 21–55CrossRefGoogle Scholar
Wedel, M., DeSarbo, W. S., Bult, J. R., Ramaswamy, V. (1993). A latent class Poisson regression model for heterogeneous count data with an application to direct mail. Journal of Applied Econometrics, 8, 397–411CrossRefGoogle Scholar
Zeger, S. L., Karim, M. R. (1991). Generalized linear models with random effects: A Gibbs sampling approach. Journal of American Statistical Association, 86, 79–679CrossRefGoogle Scholar