Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Original Article

Analyzing Multitrait-Multimethod Data

A Comparison of Three Approaches

Published Online:July 23, 2009https://doi.org/10.1027/1614-2241.5.3.99

Abstract

Assessing construct validity is a core task in psychology. Since Campbell and Fiske’s (1959) seminal article on multitrait-multimethod (MTMM) analysis, several different methodological approaches for the analysis of convergent and discriminant validity of MTMM data have been developed. In this article, two MTMM approaches are transferred to the general framework of confirmatory factor analysis and compared with the extended version of the correlated trait-correlated method minus one model (Nussbeck, Eid, Geiser, Courvoisier, & Lischetzke, 2009): The multilevel MTMM model (Maas, Lensvelt-Mulders, & Hox, 2009) and the three-mode model (Oort, 2009). Assessing the construct validity of a German Big Five MTMM data set these three MTMM approaches are compared with regard to convergent and discriminant validity estimates and with regard to method effects. Advantages and limitations of each methodological approach will be discussed in detail.

References

  • Arbuckle, J. L. , Wottke, W. (2003). Amos 5.0 Update to the user’s guide. Chicago: Small Waters Corporation. First citation in articleGoogle Scholar

  • Bentler, P. M. , Lee, S. Y. (1979). A statistical development of three-mode factor analysis. British Journal of Mathematical and Statistical Psychology, 32, 87–104. First citation in articleCrossref, Google Scholar

  • Biesanz, J. C. , West, S. G. (2004). Towards understanding assessments of the Big Five: Multitrait-multimethod analyses of convergent and discriminant validity across measurement occasion and type of observer. Journal of Personality, 72, 845–876. First citation in articleCrossref, Google Scholar

  • Bloxom, B. (1968). A note on invariance in three-mode factor analysis. Psychometrika, 33, 347–350. First citation in articleCrossref, Google Scholar

  • Browne, M. W. (1984). The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology, 37, 1–21. First citation in articleCrossref, Google Scholar

  • Browne, M. W. , Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen, J. S. Long, (Eds.), Testing structural equation models (pp. 136–162). Beverly Hills, CA: Sage. First citation in articleGoogle Scholar

  • Bryk, A. S. , Raudenbusch, S. W. (1992). Hierarchical linear models for social and behavioral research: Application and data analysis methods. Newbury Park, CA: Sage. First citation in articleGoogle Scholar

  • Campbell, D. T. , Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. First citation in articleCrossref, Google Scholar

  • Courvoisier, D. S. , Nussbeck, F. W. , Eid, M. , Geiser, C. , Cole, D. A. (2008). Analyzing the convergent validity of states and traits: Development and application of multimethod latent state-trait models. Psychological Assessment, 20, 270–280. First citation in articleCrossref, Google Scholar

  • Dumenci, L. (2000). Multitrait-multimethod analysis. In H. E. A. Tinsley, S. D. Brown, (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 583–611). San Diego, CA: Academic Press. First citation in articleCrossref, Google Scholar

  • Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241–261. First citation in articleCrossref, Google Scholar

  • Eid, M. (2006). Methodological approaches for analyzing multimethod data. In M. Eid, E. Diener, (Eds.), Handbook of multimethod measurement in psychology (pp. 223–230). Washington, DC: APA. First citation in articleCrossref, Google Scholar

  • Eid, M. , Diener, E. (2006). Introduction: The need for multimethod measurement in psychology. In M. Eid, E. Diener, (Eds.), Handbook of multimethod measurement in psychology (pp. 3–9). Washington, DC: APA. First citation in articleCrossref, Google Scholar

  • Eid, M. , Lischetzke, T. , Nussbeck, F. W. (2006). Structural equation models for multitrait-multimethod data. In M. Eid, E. Diener, (Eds.), Handbook of multimethod measurement in psychology (pp. 283–299). Washington, DC: APA. First citation in articleCrossref, Google Scholar

  • Eid, M. , Lischetzke, T. , Nussbeck, F. W. , Geiser, C. (2004). Die Multitrait-Multimethod-Analyse: Entwicklung neuer Modelle und ihre Anwendung in der Differentiellen und Diagnostischen Psychologie [The multitrait-multimethod analysis: development of new models and their application in psychology] Landau in der Pfalz: Universität Koblenz-Landau DFG-Projekt EI 379/5-2. First citation in articleGoogle Scholar

  • Eid, M. , Lischetzke, T. , Nussbeck, F. W. , Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple indicator CT-C(M−1) model. Psychological Methods, 8, 38–60. First citation in articleCrossref, Google Scholar

  • Eid, M. , Nussbeck, F. W. , Geiser, C. , Cole, D. A. , Gollwitzer, M. , Lischetzke, T. (2008). Structural equation modelling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230–253. First citation in articleCrossref, Google Scholar

  • Geiser, C. , Eid, M. , Nussbeck, F. W. (2008). On the meaning of latent variables in the CT-C(M-1) model: A comment on Maydeu-Olivares and Coffman (2006). Psychological Methods, 13, 49–57. First citation in articleCrossref, Google Scholar

  • Hox, J. J. (2002). Multilevel analysis. Techniques and applications. Mahwah, NJ: Lawrence Erlbaum Associates. First citation in articleCrossref, Google Scholar

  • Hox, J. J. , Maas, C. (2006). Multilevel models for multimethod measurements. In M. Eid, E. Diener, (Eds.), Handbook of multimethod measurement in psychology (pp. 269–281). Washington, DC: APA. First citation in articleCrossref, Google Scholar

  • Kenny, D. A. (1976). An empirical application of confirmatory factor analysis to the multitrait-multimethod matrix. Journal of Experimental Social Psychology, 12, 247–252. First citation in articleCrossref, Google Scholar

  • Maas, C. J. M. , Lensvelt-Mulders, G. J. L. M. , Hox, J. J. (2009). A multilevel multitrait-multimethod analysis. Methodology, 5, 72–77. First citation in articleLink, Google Scholar

  • Marsh, H. W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 12, 335–361. First citation in articleCrossref, Google Scholar

  • Marsh, H. W. , Grayson, D. (1995). Latent variable models of multitrait-multimethod data. In R. H. Hoyle, (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 177–198). Thousand Oaks: Sage. First citation in articleGoogle Scholar

  • Marsh, H. W. , Hocevar, D. (1988). A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. Journal of Applied Psychology, 73, 107–117. First citation in articleCrossref, Google Scholar

  • Muthén, L. K. , Muthén, B. O. (2006). Mplus user’s guide. Los Angeles, CA: Muthén & Muthén. First citation in articleGoogle Scholar

  • Neale, M. C. , Boker, S. M. , Xie, G. , & Maes, H. H. (1999). Mx: Statistical modeling (5th ed.). Available at: www.vcu.edu/mx. First citation in articleGoogle Scholar

  • Nussbeck, F. W. (2006). Assessing multimethod association with categorical variables. In M. Eid, E. Diener, (Eds.), Handbook of multimethod measurement in psychology (pp. 231–247). Washington, DC: APA. First citation in articleCrossref, Google Scholar

  • Nussbeck, F. W. , Eid, M. , Geiser, C. , Courvoisier, D. S. , Lischetzke, T. (2009). A CTC(M−1) model for different types of raters. Methodology, 5, 88–98. First citation in articleLink, Google Scholar

  • Oort, F. J. (1999). Stochastic three-mode models for mean and covariances structures. British Journal of Mathematical and Statistical Psychology, 52, 243–272. First citation in articleCrossref, Google Scholar

  • Oort, F. J. (2009). Three-mode models for multitrait-multimethod data. Methodology, 5, 78–87. First citation in articleLink, Google Scholar

  • Podsakoff, P. M. , MacKenzie, S. B. , Lee, J.-Y. , Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903. First citation in articleCrossref, Google Scholar

  • Pohl, S. , Steyer, R. , Kraus, K. (2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society, Series A, 171, 41–63. First citation in articleGoogle Scholar

  • Rost, J. , Carstensen, C. (2002). Multidimensional Rasch measurement via item component models and faceted designs. Applied Psychological Measurement, 26, 42–56. First citation in articleCrossref, Google Scholar

  • Rost, J. , Walter, O. (2006). Multimethod item response theory. In M. Eid, E. Diener, (Eds.), Handbook of multimethod measurement in psychology (pp. 249–268). Washington, DC: APA. First citation in articleCrossref, Google Scholar

  • Schermelleh-Engel, K. , Keith, N. , Moosbrugger, H. , Hodapp, V. (2004). Decomposing person and occasion-specific effects: An extension of latent state-trait theory to hierarchical models. Psychological Methods, 9, 198–219. First citation in articleCrossref, Google Scholar

  • Schermelleh-Engel, K. , Schweizer, K. (2007). Multitrait-Multimethod-Analysen. In H. Moosbrugger, A. Kelava, (Eds.) [Multitrait-multimethod analyses] Testtheorie und Fragebogenkonstruktion [Test theory and questionnaire designing] Heidelberg: Springer. First citation in articleGoogle Scholar

  • Snijders, T. A. B. , Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage. First citation in articleGoogle Scholar

  • Swain, A. J. (1975). Analysis of parametric structures for variance matrices. Australia: University of Adelaide Unpublished doctoral dissertation. First citation in articleGoogle Scholar

  • Widaman, K.-F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1–26. First citation in articleCrossref, Google Scholar