Hostname: page-component-7b7567c8c-7pwfn Total loading time: 0 Render date: 2026-11-12T19:15:03.010Z Has data issue: false hasContentIssue false
  • English
  • Français

Principal Component Analysis with External Information on both Subjects and Variables

Published online by Cambridge University Press:  01 January 2025

Yoshio Takane and
Tadashi Shibayama
Yoshio Takane*
Affiliation:
McGill University
Tadashi Shibayama
Affiliation:
McGill University
*
Requests for reprints should be sent to Yoshio Takane, Department of Psychology, McGill University, 1205 Dr. Penfield Avenue, Montreal, Quebec H3A 1B1, CANADA.
Get accessRights & Permissions [Opens in a new window]

Abstract

A method for structural analysis of multivariate data is proposed that combines features of regression analysis and principal component analysis. In this method, the original data are first decomposed into several components according to external information. The components are then subjected to principal component analysis to explore structures within the components. It is shown that this requires the generalized singular value decomposition of a matrix with certain metric matrices. The numerical method based on the QR decomposition is described, which simplifies the computation considerably. The proposed method includes a number of interesting special cases, whose relations to existing methods are discussed. Examples are given to demonstrate practical uses of the method.

Keywords

orthogonal projection operatortrace-orthogonalitygeneralized singular value decomposition (GSVD)QR decompositionvector preference modelstwo-way CANDELINCdual scalingredundancy analysisGMANOVA (growth curve models)

Information

Type
Original Paper
Information
Psychometrika , Volume 56 , Issue 1 , March 1991, pp. 97 - 120
Copyright
Copyright © 1991 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

The work reported in this paper was supported by grant A6394 from the Natural Sciences and Engineering Research Council of Canada to the first author. Thanks are due to Jim Ramsay, Haruo Yanai, Henk Kiers, and Shizuhiko Nishisato for their insightful comments on earlier versions of this paper. Jim Ramsay, in particular, suggested the use of the QR decomposition, which simplified the presentation of the paper considerably.

References

Bechtel, G. G. (1976). Multidimensional preference scaling, The Hague: Mouton.CrossRefGoogle Scholar
Bechtel, G. G., Tucker, L. R., & Chang, W. (1971). A scalar product model for the multidimensional scaling of choice. Psychometrika, 36, 369–387.CrossRefGoogle Scholar
Besse, P., && Ramsay, J. O. (1986). Principal components analysis of sampled functions. Psychometrika, 51, 285–311.CrossRefGoogle Scholar
Bloxom, B. (1978). Constrained multidimensional scaling in N spaces. Psychometrika, 43, 397–408.CrossRefGoogle Scholar
Böckenholt, U., & Böckenholt, I. (1990). Canonical analysis of contingency tables with linear constraints. Psychometrika, 55, 633–639.CrossRefGoogle Scholar
Carroll, J. D. (1972). Individual differences and multidimensional scaling. In Shepard, R. N., Romney, A. K., & Nerlove, S. B. (Eds.), Multidimensional scaling, Vol. I (pp. 105–155). New York: Seminar Press.Google Scholar
Carroll, J. D., Pruzansky, S., & Kruskal, J. B. (1980). CANDELINC: A general approach to multidimensional analysis of many-way arrays with linear constraints on parameters. Psychometrika, 45, 3–24.CrossRefGoogle Scholar
Corsten, L. C. A. (1976). Matrix approximation, a key to application of multivariate methods. Invited paper presented at the 9th Biometric Conference, Boston.Google Scholar
Corsten, L. C. A., & Van Eijnsbergen, A. C. (1972). Multiplicative effects in two-way analysis of variance. Statistica Neelandica, 26, 61–68.CrossRefGoogle Scholar
Critchley, F. (1985). Influence in principal component analysis. Biometrika, 72, 627–636.CrossRefGoogle Scholar
de Leeuw, J. (1984). Fixed rank matrix approximation with singular weights matrices. Computational Statistics Quarterly, 1, 3–12.Google Scholar
DeSarbo, W. S., Carroll, J. D., Lehmann, D. R., & O'Shaughnessy, J. (1982). Three-way multivariate conjoint analysis. Marketing Science, 1, 323–350.CrossRefGoogle Scholar
DeSarbo, W. S., & Rao, V. R. (1984). GENFOLD2: A set of models and algorithms for the GENeral UnFOLDing analysis of preference/dominance data. Journal of Classification, 1, 147–186.CrossRefGoogle Scholar
De Soete, G., & Carroll, J. D. (1983). A maximum likelihood method for fitting the wandering vector model. Psychometrika, 48, 553–566.CrossRefGoogle Scholar
Eastment, H. T., & Krzanowski, W. J. (1982). Cross-validatory choice of the number of components from a principal component analysis. Technometrics, 24, 73–77.CrossRefGoogle Scholar
Efron, B. (1979). Bootstrap methods: Another look at the Jackknife. Annals of Statistics, 7, 1–26.CrossRefGoogle Scholar
Escoufier, Y., & Holmes, S. (1988). Décomposition de la variabilité dans les analyses exploratoires: Un exemple d'analyse en composantes principles en presence de variables qualitatives concomittantes [Descomposition of variabilities in exploratory data analysis: An example of principal component analysis in the presence of qualitative concomitant variables.], Montpellier, France: ENSAM-INRA.Google Scholar
Fisher, R. A. (1948). Statistical methods for research workers 10th ed.,, London: Oliver and Boyd.Google Scholar
Gabriel, K. R. (1978). Least squares approximation of matrices by additive and multiplicative models. Journal of Royal Statistical Society, Series B, 40, 186–196.CrossRefGoogle Scholar
Gabriel, K. R., & Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights. Technometrics, 21, 489–498.CrossRefGoogle Scholar
Gifi, A. (1981). Non-linear multivariate analysis, Leiden: University of Leiden, Department of Data Theory.Google Scholar
Gollob, H. F. (1968). A statistical model which combines features of factor analytic and analysis of variance technique. Psychometrika, 33, 73–115.CrossRefGoogle Scholar
Greenacre, M. J., & Underhill, L. G. (1982). Scaling a data matrix in a low-dimensional euclidean space. In Hawkins, D. M. (Eds.), Topics in applied multivariate analysis (pp. 183–268). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Grizzle, J. E., & Allen, D. M. (1969). Analysis of growth and dose response curves. Biometrics, 25, 357–381.CrossRefGoogle ScholarPubMed
Hayashi, C. (1952). On the prediction of phenomena from qualitative data and the quantification of qualitative data from the mathematico-statistical point of view. Annals of the Institute of Statistical Mathematics, 2, 69–98.CrossRefGoogle Scholar
Heiser, W. J., & de Leeuw, J. (1981). Multidimensional mapping of preference data. Mathematiqué et sciences humaines, 19, 39–96.Google Scholar
Heiser, W. J., & Meulman, J. (1983). Analyzing rectangular tables by joint and constrained multidimensional scaling. Journal of Econometrics, 22, 139–167.CrossRefGoogle Scholar
Heiser, W. J., & Meulman, J. (1983). Constrained multidimensional scaling, including confirmation. Applied Psychological Measurement, 7, 381–404.CrossRefGoogle Scholar
Israëls, A. Z. (1984). Redundancy analysis for qualitative variables. Psychometrika, 49, 331–346.CrossRefGoogle Scholar
Jolliffe, I. T. (1986). Principal component analysis, Berlin: Springer Verlag.CrossRefGoogle Scholar
Khatri, C. G. (1966). A note on a MANOVA model applied to problems in growth curves. Annals of the Institute of Statistical Mathematics, 18, 75–86.CrossRefGoogle Scholar
Kruskal, J. B. (1964). Nonmetric multidimensional scaling: A numerical method. Psychometrika, 29, 115–129.CrossRefGoogle Scholar
Meulman, J. (1982). Homogeneity analysis of incomplete data, Leiden: DSWO Press.Google Scholar
Nishisato, S. (1978). Optimal scaling of paired comparison and rank order data: An alternative to Guttman's formulation. Psychometrika, 43, 263–271.CrossRefGoogle Scholar
Nishisato, S. (1980). Analysis of categorical data: Dual scaling and its applications, Toronto: University of Toronto Press.CrossRefGoogle Scholar
Nishisato, S. (1980). Dual scaling of successive categories data. Japanese Psychological Research, 22, 134–143.CrossRefGoogle Scholar
Nishisato, S. (1982). Quantification of qualitative data, Tokyo: Asakurashoten.Google Scholar
Nishisato, S. (1988). Dual scaling: Its development and comparisons with other quantification methods. In Pressmar, H. D., Jager, K. E., Krallmann, H., Schellhaas, H., & Streitferdt, L. (Eds.), Deutsche Geselleschaft für operations research proceedings (pp. 376–389). Berlin: Springer.Google Scholar
Nishisato, S., & Lawrence, D. R. (1981, May). Dual scaling of multidimensional tables, a comparative study. Paper presented at the annual meeting of the Psychometric Society, Chapel Hill, NC.Google Scholar
Nishisato, S., & Lawrence, D. R. (1989). Dual scaling of multiway data matrices: Several variants. In Coppi, R., Bolasco, S. (Eds.), Multiway data analysis (pp. 317–326). Amsterdam: North Holland.Google Scholar
Nishisato, S., & Sheu, W. (1984). A note on dual scaling of successive categories data. Psychometrika, 49, 493–500.CrossRefGoogle Scholar
Okamoto, M. (1972). Four techniques of principal component analysis. Journal of Japanese Statistical Society, 2, 63–69.Google Scholar
Potthoff, R. F., & Roy, S. N. (1964). A generalized multivariate analysis of variance model useful especially for growth curve problems. Biometrika, 51, 313–326.CrossRefGoogle Scholar
Ramsay, J. O. (1978). Confidence regions for multidimensional scaling analysis. Psychometrika, 43, 145–160.CrossRefGoogle Scholar
Ramsay, J. O. (1980). Joint analysis of direct ratings, pairwise preferences and dissimilarities. Psychometrika, 45, 149–165.CrossRefGoogle Scholar
Ramsay, J. O. (1989). Monotone regression splines in actions. Statistical Science, 4, 425–441.Google Scholar
Ramsay, J. O., ten Berge, J., & Styan, G. P. H. (1984). Matrix correlation. Psychometrika, 49, 403–423.CrossRefGoogle Scholar
Rao, C. R. (1964). The use and interpretation of principal component analysis in applied research. Sankhya A, 26, 329–358.Google Scholar
Rao, C. R. (1965). The theory of least squares when the parameters are stochastic and its application to the analysis of growth curves. Biometrika, 52, 447–458.CrossRefGoogle Scholar
Rao, C. R. (1979). Separation theorems for singular values of matrices and their applications in multivariate analysis. Journal of Multivariate Analysis, 9, 362–377.CrossRefGoogle Scholar
Rao, C. R. (1980). Matrix approximations and reduction of dimensionality in multivariate statistical analysis. In Krishnaiah, P. R. (Eds.), Multivariate analysis (pp. 3–22). Amsterdam: North Holland.Google Scholar
Rumelhart, D. L., & Greeno, J. G. (1971). Similarity between stimuli: An experimental test of the Luce and Restle choice models. Journal of Mathematical Psychology, 8, 370–381.CrossRefGoogle Scholar
Sabatier, R., Lebreton, J. D., & Chessel, D. (1989). Multivariate analysis of composition data accompanied by qualitative variables describing a structure. In Coppi, R., Bolasco, S. (Eds.), Multiway data analysis (pp. 341–352). Amsterdam: North-Holland.Google Scholar
Shibayama, T. (1988). Multivariate analysis of test scores with missing values. Unpublished Doctoral Dissertation, University of Tokyo. (in Japanese)Google Scholar
Siotani, M., Hayakawa, T., & Fujikoshi, Y. (1985). Modern multivariate statistical analysis: A graduate course handbook, Columbus, OH: American Sciences Press.Google Scholar
Slater, P. (1960). The analysis of personal preferences. The British Journal of Statistical Psychology, 13, 119–135.CrossRefGoogle Scholar
Sjöberg, L. (1967). Successive categories scaling of paired comparisons. Psychometrika, 32, 297–308.CrossRefGoogle Scholar
Takane, Y. (1980). Maximum likelihood estimation in the generalized case of Thurstone's model of comparative judgment. Japanese Psychological Research, 22, 188–196.CrossRefGoogle Scholar
Takane, Y. (1987). Analysis of covariance structures and binary choice data. Communication and Cognition, 20, 45–62.Google Scholar
Takane, Y., & Shibayama, T. (1988). Three vector models of pairwise preference ratings and their generalizations. In Kashiwagi, S. (Eds.), Proceedings of the 16th Annual Meeting of the Behaviormetric Society (pp. 131–132). Tokyo: Behaviormetric Society of Japan.Google Scholar
Takane, Y., & Shibayama, T. (1988). Dual scaling with external criteria reconsidered. In Kashiwagi, S. (Eds.), Proceedings of the 16th Annual Meeting of the Behaviormetric Society (pp. 133–134). Tokyo: Behaviormetric Society of Japan.Google Scholar
Takane, Y., Yanai, H., & Mayekawa, S. (in press). Relationships among several methods of linearly constrained correspondence analysis. Psychometrika.Google Scholar
Tanaka, Y. (1988). Sensitivity analysis in principal component analysis: Influence on the subspace spanned by principal components. Communications in Statistics—Theory and Methods, 17, 3157–3175.CrossRefGoogle Scholar
ter Braak, C. J. F. (1986). Canonical correspondence analysis: A new eigenvector technique for multivariate direct gradient analysis. Ecology, 67, 1167–1179.CrossRefGoogle Scholar
Tucker, L. R. (1959). Intra-individual and inter-individual multidimensionality. In Gulliksen, H., Messick, S. (Eds.), Psychological scaling (pp. 155–167). New York: Wiley.Google Scholar
van den Wollenberg, A. L. (1977). Redundancy analysis: An alternative for canonical correlation analysis. Psychometrika, 42, 207–219.CrossRefGoogle Scholar
van der Heijden, P. G. M., de Falguerolles, A., & de Leeuw, J. (1989). A combined approach to contingency table analysis using correspondence analysis and log linear analysis. Applied Statistics, 38, 249–292.CrossRefGoogle Scholar
Weinberg, S. L., Carroll, J. D., & Cohen, H. S. (1984). Confidence regions for INDSCAL using the jackknife and bootstrap techniques. Psychometrika, 49, 475–491.CrossRefGoogle Scholar
Wilkinson, J. H. (1965). The algebraic eigenvalue problem, Oxford: Oxford University Press.Google Scholar
Winsberg, S. (1988). Two techniques: Monotone spline transformations for dimension reduction in PCA and easy-to-generate metrics for PCA of sampled functions. In van Rijckevorsel, J. L. A. & de Leeuw, J. (Eds.), Component and correspondence analysis (pp. 115–135). New York: Wiley.Google Scholar
Winsberg, S., & Ramsay, J. O. (1983). Monotone spline transformations for dimension reduction. Psychometrika, 48, 575–595.CrossRefGoogle Scholar
Yanai, H. (1970). Factor analysis with external criteria. Japanese Psychological Research, 12, 143–153.CrossRefGoogle Scholar
Yanai, H. (1974). Unification of various techniques of multivariate analysis by means of generalized coefficients of determination. (G.C.D.). Journal of Behaviormetrics, 1, 45–54.Google Scholar
Yanai, H. (1990). Some generalized forms of least squares g-inverse, minimum norm g-inverse and Moore-Penrose inverse matrices. Computational Statistics and Data Analysis, 10, 251–260.CrossRefGoogle Scholar
Yanai, H., & Takeuchi, K. (1983). Projection matrices, generalized inverse and singular value decomposition, Tokyo: University of Tokyo Press (in Japanese)Google Scholar