Hostname: page-component-7b7567c8c-mnklg Total loading time: 0 Render date: 2026-11-12T16:59:58.930Z Has data issue: false hasContentIssue false
  • English
  • Français

A Simulated Annealing Methodology for Clusterwise Linear Regression

Published online by Cambridge University Press:  01 January 2025

Wayne S. DeSarbo,
Richard L. Oliver and
Arvind Rangaswamy
Wayne S. DeSarbo*
Affiliation:
Marketing and Statistics Departments, Graduate School of Business, University of Michigan
Richard L. Oliver
Affiliation:
Marketing Department, The Wharton School, University of Pennsylvania
Arvind Rangaswamy
Affiliation:
Marketing Department, The Wharton School, University of Pennsylvania
*
Requests for reprints should be sent to Wayne S. DeSarbo, S. S. Kresge Professor of Marketing and Statistics, Marketing and Statistics Departments, Graduate School of Business, University of Michigan, Ann Arbor, MI 48109-1234.
Get accessRights & Permissions [Opens in a new window]

Abstract

In many regression applications, users are often faced with difficulties due to nonlinear relationships, heterogeneous subjects, or time series which are best represented by splines. In such applications, two or more regression functions are often necessary to best summarize the underlying structure of the data. Unfortunately, in most cases, it is not known a priori which subset of observations should be approximated with which specific regression function. This paper presents a methodology which simultaneously clusters observations into a preset number of groups and estimates the corresponding regression functions' coefficients, all to optimize a common objective function. We describe the problem and discuss related procedures. A new simulated annealing-based methodology is described as well as program options to accommodate overlapping or nonoverlapping clustering, replications per subject, univariate or multivariate dependent variables, and constraints imposed on cluster membership. Extensive Monte Carlo analyses are reported which investigate the overall performance of the methodology. A consumer psychology application is provided concerning a conjoint analysis investigation of consumer satisfaction determinants. Finally, other applications and extensions of the methodology are discussed.

Keywords

cluster analysiscombinatorial optimizationregression analysissimulated annealingconsumer psychology

Information

Type
Original Paper
Information
Psychometrika , Volume 54 , Issue 4 , December 1989, pp. 707 - 736
Copyright
Copyright © 1989 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

The authors wish to thank the editor, associate editor, and three anonymous reviewers for their insightful comments and thorough review of this manuscript.

References

Aarts, E., & Korst, J. (1989). Simulated annealing and Boltzmann machines, New York: Wiley.Google Scholar
Addelman, S. (1962). Orthogonal main effects plans for asymmetrical factorial experiments. Technometrics, 4, 21–46.CrossRefGoogle Scholar
Bohachevsky, I. O., Johnson, M. E., & Stein, M. L. (1986). Generalized simulated annealing for function optimization. Technometrics, 28, 209–217.CrossRefGoogle Scholar
Carroll, J. D. (1972). Individual differences and multidimensional scaling. In Shepard, R. N., Romney, A. K. & Nerlove, S. (Eds.), Multidimensional scaling: Theory and applications in the behavioral sciences (pp. 105–155). New York: Seminar Press.Google Scholar
Davis, L. (1987). Genetic algorithms and simulated annealing, London: Pitman.Google Scholar
DeSarbo, W. S. (1982). GENNCLUS: New models for general nonhierarchical clustering analysis. Psychometrika, 47, 446–469.CrossRefGoogle Scholar
DeSarbo, W. S., & Carroll, J. D. (1985). Three-way metric unfolding via weighted least squares. Psychometrika, 50, 275–300.CrossRefGoogle Scholar
DeSarbo, W. S., Carroll, J. D., Clark, L. A., & Green, P. E. (1984). Synthesized clustering: A method for amalgamating alternative clustering bases with differential weighting of variables. Psychometrika, 49, 57–78.CrossRefGoogle Scholar
DeSarbo, W. S., & Cron, W. L. (1988). A maximum likelihood methodology for clusterwise linear regression. Journal of Classification, 5, 249–282.CrossRefGoogle Scholar
DeSarbo, W. S., & Mahajan, V. (1984). constrained classification: The use of a priori information in cluster analysis. Psychometrika, 49, 187–215.CrossRefGoogle Scholar
DeSarbo, W. S., Oliver, R. L., & De Soete, G. (1986). A probabilistic multidimensional scaling vector model. Applied Psychological Measurement, 10, 79–98.CrossRefGoogle Scholar
De Soete, G., DeSarbo, W. S., & Carroll, J. D. (1985). Optimal variable weighting for hierarchical clustering: An alternating least squares algorithm. Journal of Classification, 2/3, 173–192.CrossRefGoogle Scholar
De Soete, G., DeSarbo, W. S., Furnas, G. W., & Carroll, J. D. (1984). The presentation of nonsymmetric rectangular proximity data by ultrametric and path length tree structures. Psychometrika, 49, 289–310.CrossRefGoogle Scholar
De Soete, G., Hubert, L., & Arabie, P. (1988). On the use of simulated annealing for combinatorial data analysis. In Gaul, W. & Schader, M. (Eds.), Data, expert knowledge, and decisions (pp. 328–340). Berlin: Springer Verlag.Google Scholar
De Soete, G., Hubert, L., & Arabie, P. (1988). The comparative performance of simulated annealing on two problems of combinatorial data analysis. In Diday, E. (Eds.), Data analysis and informatics, V (pp. 489–496). Amsterdam: North-Holland.Google Scholar
Dubes, R. C., & Klein, R. (1987, June). Simulated annealing in data analysis. Handout at a talk given at the 1987 Annual Meeting of the Psychometric Society, Montreal, Canada.Google Scholar
Gabor, A., & Granger, D. W. (1966). On the price consciousness of consumers. Applied Statistics, 10, 170–181.CrossRefGoogle Scholar
Gidas, B. (1985). Nonstationary Markov chains and convergence of the annealing algorithm. Journal of Statistical Physics, 39, 73–131.CrossRefGoogle Scholar
Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and macine learning, Reading: Addison-Wesley.Google Scholar
Green, P. E., & Rao, V. R. (1971). Conjoint measurement for quantifying judgmental data. Journal of Marketing Research, 8, 355–363.Google Scholar
Green, P. E., & Srinivasan, V. (1978). Conjoint analysis in consumer research: Issues and outlook. Journal of Consumer Research, 5, 103–123.CrossRefGoogle Scholar
Haggerty, M. R. (1985). Improving the predictive power of conjoint analysis: The use of factor analysis and cluster analysis. Journal of Marketing Research, 22, 168–184.CrossRefGoogle Scholar
Henderson, J. M., & Quandt, R. E. (1982). Microeconomic theory: A mathematical approach 3rd ed.,, New York: McGraw Hill.Google Scholar
Johnson, M. E. (1988). Simulated annealing and optimization, Syracuse: American Science Press.Google Scholar
Judge, G. G., Griffiths, W. E., Hill, R. C., Lükepohl, H., & Lee, T. (1985). The theory and practice of econometrics, New York: Wiley.Google Scholar
Kamakura, A. W. (1988). A least-squares procedure for benefit segmentation with conjoint experiments. Journal of Marketing Research, 25, 157–167.CrossRefGoogle Scholar
Kirkpatrick, S., Gelatt, C. D., & Vechhi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.CrossRefGoogle ScholarPubMed
Levy, A. V., & Montalvo, A. (1985). The tunneling algorithm for the global minimization of functions. SIAM Journal of Scientific and Statistical Computing, 6, 15–29.CrossRefGoogle Scholar
Lin, S., & Kernigham, B. (1973). An effective heuristic algorithm for the traveling salesman problem. Operations Research, 21, 498–516.CrossRefGoogle Scholar
Locke, E. A. (1967). Relationship of goal level to performance level. Psychological Reports, 20, 1068–1068.CrossRefGoogle Scholar
Locke, E. A. (1982). Relation of goal level to performance with a short work period and multiple goal levels. Journal of Applied Psychology, 67, 512–514.CrossRefGoogle Scholar
Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 1969–1980. Psychological Bulletin, 90, 125–152.CrossRefGoogle Scholar
Lundy, M. (1986). Applications of the annealing algorithm to combinatorial problems in statistics. Biometrika, 72, 191–198.CrossRefGoogle Scholar
MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. 5th Berkeley Symposium on Mathematics, Statistics and Probability, 1, 281–298.Google Scholar
Maddala, G. S. (1976). Econometrics, New York: McGraw-Hill.Google Scholar
Meier, J. (1987). A fast algorithm for clusterwise linear absolute deviations regression. OR Spektrum, 9, 187–189.CrossRefGoogle Scholar
Mitra, D., Romeo, F., & Sangiovanni-Vincentelli, A. (in press). Convergence and finite-time behavior of simulated annealing. Advances in Applied Probability.Google Scholar
Ogawa, K. (1987). An approach to simultaneous estimation and segmentation in conjoint analysis. Marketing Science, 6, 66–81.CrossRefGoogle Scholar
Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research, 17, 460–469.CrossRefGoogle Scholar
Oliver, R. L., DeSarbo, W. S. (1988). Response determinants in satisfaction judgments. Journal of Consumer Research, 14, 495–507.CrossRefGoogle Scholar
Oliver, R. L., & Swan, J. E. (in press). Consumer perceptions of interpersonal equity and satisfaction in transactions: A field survey approach. Journal of Marketing.Google Scholar
Quandt, R. E. (1972). A new approach to estimating switching regressions. Journal of the American Statistical Association, 67, 306–310.CrossRefGoogle Scholar
Sowter, A. P., Gabor, A., & Granger, G. W. (1971). The effect of price on choice. Applied Economics, 3, 167–181.CrossRefGoogle Scholar
Späth, H. (1979). Algorithm 39: Clusterwise linear regression. Computing, 22, 367–373.CrossRefGoogle Scholar
Späth, H. (1981). Correction to Algorithm 39: Clusterwise linear regression. Computing, 26, 275–275.CrossRefGoogle Scholar
Späth, H. (1982). Algorithm 48: A fast algorithm for clusterwise linear regression. Computing, 29, 175–181.CrossRefGoogle Scholar
Späth, H. (1985). Cluster dissection and analysis, New York: Wiley.Google Scholar
Späth, H. (1986). Clusterwise linear least absolute deviations regression. Computing, 37, 371–378.CrossRefGoogle Scholar
Späth, H. (1986). Clusterwise linear least squares versus least absolute deviations regression: A numerical comparison for a case study. In Gaul, W., Schader, M. (Eds.), Classification as a tool of research (pp. 413–422). New York: North Holland.Google Scholar
Späth, H. (1987). Mathematische software zur linearen regression [Mathematical software for linear regression], Munich: R. Oldenbourg.Google Scholar
van Laarhoven, P. J. M., & Aarts, E. H. L. (1987). Simulated annealing: Theory and applications, Boston: D. Reidel.CrossRefGoogle Scholar