Multimethod Assessement of Time-Stable and Time-Variable Interindividual Differences
Introduction of a New Multitrait-Multimethod Latent State-Trait IRT Model
Abstract
Abstract. The dynamic development of interindividual differences and the temporal interplay between different personality constructs are of major interest to many researchers in the field of personality psychology. Furthermore, the collection of multiple rater-perspectives complementing classical self-report measures in psychological assessment is increasingly applied also in longitudinal research. Nevertheless, models to analyze longitudinal multitrait-multimethod (MTMM) data are scarce. A new Latent State-Trait (LST) Graded Response Model for the analysis of longitudinal MTMM data with ordered categorical response variables is introduced. The model combines advantages of LST theory and MTMM models for different types of raters (interchangeable and structurally different) with an Item Response Theory (IRT) approach. The model allows researchers to analyze the stability and variability of personality constructs, discriminant and convergent validity, as well as rater effects on the item-level. Model application and interpretation are illustrated using subjective well-being data of young adults. Results of an extensive simulation study indicate that the model can be accurately estimated with Bayesian statistics with at least 3 measurement occasions and more than 250 target persons rated by at least 5 interchangeable raters under moderate degrees of convergent validity.
References
1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. https://doi.org/10.1037/h0046016
(2011). Analyzing multitrait-mulitmethod data with multilevel confirmatory factor analysis: An application to the validation of the state-trait cheerfulness inventory. Journal of Research in Personality, 45, 153–164. https://doi.org/10.1016/j.jrp.2010.12.007
(2005). Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model. Psychological Methods, 10, 3–20. https://doi.org/10.1037/1082-989X.10.1.3
(2008). Analyzing the convergent and discriminant validity of states and traits: Development and applications of multimethod latent state-trait models. Psychological Assessment, 20, 270–280. https://doi.org/10.1037/a0012812
(2011). Evaluating interventions with multimethod data: A structural equation modeling approach. Structural Equation Modeling, 18, 497–524. https://doi.org/10.1080/10705511.2011.607068
(1985). The Satisfaction with Life Scale. Journal of Personality Assessment, 49, 71–75. https://doi.org/10.1207/s15327752jpa4901_13
(1996). Longitudinal confirmatory factor analysis for polytomous item responses: Model definition and model selection on the basis of stochastic measurement theory. Methods of Psychological Research Online, 1, 65–85.
(2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241–261. https://doi.org/10.1007/BF02294377
(2004). Global judgments of subjective well-being: Situational variability and long-term stability. Social Indicators Research, 65, 245–277. https://doi.org/10.1023/B:SOCI.0000003801.89195.bc
(2016). Measuring method effects from traditional to design-oriented approaches. Current Directions in Psychological Science, 25, 275–280. https://doi.org/10.1177/0963721416649624
(1998). Measuring variability and change with an item response model for polytomous variables. Journal of Educational and Behavioral Statistics, 23, 193–215. https://doi.org/10.3102/10769986023003193
(2017). On the definition of latent-state-trait models with autoregressive effects: Insights from LST-R theory. European Journal of Psychological Assessment, 33, 285–295. https://doi.org/10.1027/1015-5759/a000435
(2003). Separating stable from variable individuals in longitudinal studies by mixture distribution models. Measurement: Interdisciplinary Research and Perspectives, 1, 179–206. https://doi.org/10.1207/S15366359MEA0103_01
(2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8, 38–60. https://doi.org/10.1037/1082-989X.8.1.38
(2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230–253. https://doi.org/10.1037/a0013219
(2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466–491. https://doi.org/10.1037/1082-989X.9.4.466
(2008). On the meaning of the latent variables in the CT-C(M-1) model: A comment on Maydeu-Olivares and Coffman (2006). Psychological Methods, 13, 49–57. https://doi.org/10.1037/1082-989X.13.1.49
(2012). A comparison of method effects in two confirmatory factor models for structurally different methods. Structural Equation Modeling, 19, 409–436. https://doi.org/10.1080/10705511.2012.687658
(2012). A comparison of four approaches to account for method effects in latent state-trait analyses. Psychological Methods, 17, 255–283. https://doi.org/10.1037/a0026977
(2011). The German version of the Satisfaction with Life Scale (SWLS). European Journal of Psychological Assessment, 27, 127–132. https://doi.org/10.1027/1015-5759/a000058
(2007). The integrated trait–state model. Journal of Research in Personality, 41, 295–315. https://doi.org/10.1016/j.jrp.2006.04.003
(2017). Multilevel graded response models for analyzing longitudinal multitrait-multimethod data (Unpublished doctoral dissertation, Freie Universität Berlin, Germany). Retrieved from https://refubium.fu-berlin.de/handle/fub188/6457
(2017). Bayesian analysis of longitudinal mulitrait-multimethod data with ordinal response variables. British Journal of Mathematical and Statistical Psychology, 70, 42–80. https://doi.org/10.1111/bmsp.12081
(2016). A comparison of ML, WLSMV, and Bayesian methods for multilevel structural equation models in small samples: A simulation study. Multivariate Behavioral Research, 51, 661–680. https://doi.org/10.1080/00273171.2016.1208074
(2014). Recent developments in maximum likelihood estimation of MTMM models for categorical data. Frontiers in Psychology, 5, 269. https://doi.org/10.3389/fpsyg.2014.00269
(1995).
(The multitrait-multimethod matrix: Design, analysis, and conceptual issues . In P. E. ShroutS. T. FiskeEds., Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 111–124). Hillsdale, NJ: Erlbaum.2018).
(Multitrait-multimethod-analysis: The psychometric foundation of CFA-MTMM models . In P. IrwingT. BoothD. HughesEds., The Wiley handbook of psychometric testing (pp. 781–846). London, UK: Wiley.2017). A multimethod latent state-trait model for structurally different and interchangeable methods. Psychometrika, 82, 17–47. https://doi.org/10.1007/s11336-016-9541-x
(2014). An evaluation of WLSMV and Bayesian methods for confirmatory factor analysis with categorical indicators. International Journal of Quantitative Research in Education, 2, 17–38. https://doi.org/10.1504/IJQRE.2014.060972
(2002, September). Unterschiedliche Definitionen von Methodeneffekten in MTMM Modellen und ihre Implikationen für die Analyse der Validität
([Different definitions of method effects in MTMM models and their implications for validity analysis] . Paper presented at the 43rd meeting of the German Psychological Association, Berlin, Germany.2016). I’m lonely, can’t you tell? Convergent validity of self- and informant ratings of loneliness. Journal of Research in Personality, 61, 50–60. https://doi.org/10.1016/j.jrp.2016.02.002
(1999). A measure of subjective happiness: Preliminary reliability and construct validation. Social Indicators Research, 46, 137–155. https://doi.org/10.1023/A:1006824100041
(2019). Two-level dynamic structural equation models with small samples. Structural Equation Modeling, 26, 948–966. https://doi.org/10.1080/10705511.2019.1578657
(1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543. https://doi.org/10.1007/BF02294825
(2002). Latent variable analysis with categorical outcomes: Multiple-Group and growth modeling in Mplus, (Mplus Web Notes Nos. 4, Version 5). Retrieved from https://www.statmodel.com/download/webnotes/CatMGLong.pdf
(2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17, 313–335. https://doi.org/10.1037/a0026802
(1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
(2006). Analysing multitrait–multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: What sample size is needed for valid results? British Journal of Mathematical and Statistical Psychology, 59, 195–213. https://doi.org/10.1348/000711005X67490
(1993). Review of the Satisfaction with Life Scale. Psychological Assessment, 5, 164–172. https://doi.org/10.1037/1040-3590.5.2.164
(2010). Modeling common traits and method effects in multitrait-multimethod analysis. Multivariate Behavioral Research, 45, 45–72. https://doi.org/10.1080/00273170903504729
(2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society: Series A, 171, 41–63. https://doi.org/10.1111/j.1467-985X.2007.00517.x
(2015). Can’t we make it any shorter? Journal of Individual Differences, 35, 212–220. https://doi.org/10.1027/1614-0001/a000141
(2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373. https://doi.org/10.1037/a0029315
(1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34, 1–97. https://doi.org/10.1007/BF03372160
(2004). Decomposing person and occasion-specific effects: An extension of latent state-trait (LST) theory to hierarchical LST models. Psychological Methods, 9, 198–219. https://doi.org/10.1037/1082-989X.9.2.198
(2007).
(Multitrait-multimethod models for longitudinal research . In K. van MontfortJ. OudA. SatorraEds., Longitudinal models in the behavioral and related sciences (pp. 381–401). Mahwah, NJ: Erlbaum.2009). Self-informant agreement in well-being ratings: A meta-analysis. Social Indicators Research, 94, 363–376. https://doi.org/10.1007/s11205-009-9440-y
(1989). Models of classical psychometric test theory as stochastic measurement models: Representation, uniqueness, meaningfulness, identifiability, and testability. Methodika, 3, 25–60.
(1992). States and traits in psychological assessment. European Journal of Psychological Assessment, 8, 79–98.
(2015). A theory of states and traits – revised. Annual Review of Clinical Psychology, 11, 71–98. https://doi.org/10.1146/annurev-clinpsy-032813-153719
(2017). Probability and conditional expectation: Fundamentals for the empirical sciences. Chichester, UK: Wiley.
(1999). Latent state-trait theory and research in personality and individual differences. European Journal of Personality, 13, 389–408. https://doi.org/10.1002/(SICI)1099-0984(199909/10)13:5<389::AID-PER361>3.0.CO;2-A
(2009). Psychometric evaluation of the Tagalog and German Subjective Happiness Scales and a cross-cultural comparison. Social Indicators Research, 93, 393–406. https://doi.org/10.1007/s11205-008-9331-7
(1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393–408. https://doi.org/10.1007/BF02294363
(2018).
(Well-being concepts and components . In E. DienerS. OishiL. TayEds., Handbook of well-being (pp. 1–15). Salt Lake City, UT: DEF. Retrieved from https://nobascholar.com/