Measuring the Big Five in Self-Report and Other Ratings
Abstract
Summary: The aim of the study is to assess the construct validity of two different measures of the Big Five, matching two “response modes” (phrase-questionnaire and list of adjectives) and two sources of information or raters (self-report and other ratings). Two-hundred subjects, equally divided in males and females, were administered the self-report versions of the Big Five Questionnaire (BFQ) and the Big Five Observer (BFO), a list of bipolar pairs of adjectives (Caprara, Barbaranelli, & Borgogni, 1993, 1994). Every subject was rated by six acquaintances, then aggregated by means of the same instruments used for the self-report, but worded in a third-person format. The multitrait-multimethod matrix derived from these measures was then analyzed via Structural Equation Models according to the criteria proposed by Widaman (1985), Marsh (1989), and Bagozzi (1994). In particular, four different models were compared. While the global fit indexes of the models were only moderate, convergent and discriminant validities were clearly supported, and method and error variance were moderate or low.
References
References
Akaike, H. (1987). Factor Analysis and AIC. Psychometrika, 52, 317– 332Bagozzi, R. (1993). Assessing construct validity in personality research: Applications to measures of self esteem. Journal of Research in Personality, 27, 49– 87Bagozzi, P. (1994). The evaluation of structural equation models and hypothesis testing. In R. Bagozzi (Ed.), Principles of marketing research (pp. 386-422). Cambridge, MA: BlackwellBagozzi, R. Yi, Y. (1992). Testing hypotheses about methods, traits and communalities in the direct product model. Applied Psychological Measurement, 16, 373– 380Bentler, P.M. (1990). Comparative fit indexes in structural equations models. Psychological Bulletin, 107, 238– 246Bentler, P.M. (1995). EQS Structural Equations Program Manual . Encino, CA: Multivariate Software Inc.Block, J. (1995). A contrarian view of the five-factor approach to personality description. Psychological Bulletin, 117, 187– 215Bollen, K.A. (1989). Structural equations with latent variables . New York: WileyBond, M.H. Nakazato, H. Shiraishi, D. (1975). Universality and distinctiveness in dimensions of Japanese person perception. Journal of Cross-Cultural Psychology, 6, 346– 357Borkenau, P. (1986). Toward an understanding of trait-interrelations: Acts as instances of several traits. Journal of Personality and Social Psychology, 51, 371– 381Borkenau, P. Ostendorf, F. (1990). Comparing exploratory and confirmatory factor analysis: A study on the 5-factor model of personality. Personality and Individual Differences, 11, 515– 524Boyle, G.J. (1989). Re-examination of the major personality-type factors in the Cattell, Comrey and Eysenck scales: Were the factor solutions by Noller et al. optimal?. Personality and Individual differences, 10, 1289– 1299Briggs, S. (1992). Assessing the Five Factor model of personality description. Journal of Personality, 60, 254– 293Browne, M.W. (1984). The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology, 37, 1– 21Browne, M.W. (1989). Relationships between an additive and a multiplicative model for multitrait-multimethod matrices. In R. Coppi & S. Bolasco (Eds.), Multiway data analysis (pp. 507-520). Amsterdam: North-HollandBrowne, M.W. (1990). MUTMUM PC user's guide . Unpublished manuscript, Department of Statistics, University of South Africa, PretoriaBrowne, M.W. Cudek, R. (1993). Alternative ways to assess model fit. In K.A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: SageBuss, D.M. Craik, K.H. (1983). The act frequency approach to personality. Psychological Review, 90, 105– 126Byrne, B. (1994). Testing the factorial validity, replication, and invariance of a measuring instrument: A paradigmatic application based on the Maslach Burnout Inventory. Multivariate Behavioral Research, 29, 289– 311Byrne, B. Goffin, R. (1993). Modelling MTMM data from additive to multiplicative covariance structure: An audit of construct validity concordance. Multivariate Behavioral Research, 28, 67– 96Campbell, D.T. Fiske, D.W. (1959). Convergent and discriminant validity by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81– 105Campbell, D.T. O'Connell, E.J. (1967). Methods factors in multitrait-multimethod matrices: Multiplicative rather than additive?. Multivariate Behavioral Research, 2, 409– 426Campbell, D.T. O'Connell, E.J. (1982). Methods as diluting trait relationships rather than adding irrelevant systematic variance. In D. Brinberg & L. Kidder (Eds.), Forms of validity in research: New directions for methodology of social and behavioral science (Vol. 12, pp. 93-111). San Francisco: Jossey-BassCaprara, G.V. (1996). Structures and processes in personality psychology. European Psychologist, 1, 13– 25Caprara, G.V. Barbaranelli, C. Borgogni, L. (1993). BFQ: Big Five Questionnaire. Manuale. [BFQ: Big Five Questionnaire. Manual]. Firenze: Organizzazioni SpecialiCaprara, G.V. Barbaranelli, C. Borgogni, L. (1994). BFO: Big Five Observer. Manuale. [BFQ: Big Five Observer. Manual]. Firenze: Organizzazioni SpecialiCaprara, G.V. Barbaranelli, C. Borgogni, L. Perugini, M. (1993). The Big Five Questionnaire: A new questionnaire for the measurement of the five-factor model. Personality and Individual Differences, 15, 281– 288Caprara, G.V. Barbaranelli, C. Borgogni, L. Perugini, M. (1994). Cinque fattori e dieci sottodimensioni per la descrizione della personalita. [Five factors and ten facets for describing personality]. Giornale Italiano di Psicologia, 21, 77– 97Caprara, G.V. Perugini, M. (1994). Personality described by adjectives: Generalizability of the “Big Five” to the Italian lexical context. European Journal of Personality, 8, 357– 369Church, A.T. Katigbak, M.S. (1989). Internal, external, and self-report structure of personality in a non western culture: An investigation of cross-language and cross-cultural generalizability. Journal of Personality and Social Psychology, 57, 857– 872Costa, P.T. McCrae, R.R. (1985). The NEO Personality Inventory manual . Odessa, FL: Psychological Assessment ResourcesDe Raad, B. Hendricks, A.A.J. Hofstee, W.K.B. (1992). Toward a refined structure of personality traits. European Journal of Personality, 6, 301– 319De Raad, B. Van Heck, G.L. (1994). The fifth of the Big Five [Special issue]. European Journal of Personality, 8, 4Digman, J.M. (1990). Personality structure: Emergence of the five factors model. Annual Review of Psychology, 41, 417– 440Dillon, W.R. Goldstein, M. (1985). Multivariate Analysis: Methods and applications . Hillsdale, NJ: ErlbaumFiske, D.W. (1987). Construct invalidity comes from method effects. Educational and Psychological Measurement, 47, 285– 307Gabrielli, A. (1977). Dizionario dei sinonimi e dei contrari [Dictionary of synonyms and antonyms]. Milano: Istituto Editoriale ItalianoGoffin, R.D. Jackson, D.N. (1992). Analysis of multitrait-multirater performance appraisal data: Composite direct product method versus confirmatory factor analysis. Multivariate Behavioral Research, 27, 363– 385Goldberg, L.R. (1983, June). The magical number of five plus or minus two: Some considerations on the dimensionality of personality descriptors . Paper presented at a Research Seminar Gerontology Research Center, NIA/NIH, Baltimore, MDGoldberg, L.R. (1990). An alternative “Description of personality”: The Big Five factor structure. Journal of Personality and Social Psychology, 59, 1216– 1229Goldberg, L.R. (1992). The development of the markers of the Big-Five factor structure. Psychological Assessment, 4, 26– 42Goldberg, L.R. (1993). The structure of phenotypic personality traits. American Psychologist, 48, 26– 34Goldberg, L.R. Kilowski, J.M. (1985). The prediction of semantic consistency in self-descriptions: Characteristics of persons and of terms that affect the consistency of response to synonym and antonym pairs. Journal of Personality and Social Psychology, 48, 82– 98Isaka, H. (1990). Factor analysis of trait terms in everyday Japanese language. Personality and Individual Differences, 11, 115– 124John, O.P. (1990). The “big five” factor taxonomy: Dimensions of personality in natural language and in questionnaires. In L.A. Pervin (Ed.), Handbook of personality: theory and research (pp. 66-100). New York: GuilfordJohn, O.P. Angleitner, A. Ostendorf, F. (1988). The lexical approach to personality: A historical review of trait taxonomic research. European Journal of Personality, 2, 171– 203Jöreskog, K.C. Sörbom, D. (1989). LISREL 7: A guide to the program and applications . Chicago: SPSSJöreskog, K.C. Sörbom, D. (1993). LISREL 8: User's reference guide . Chicago: Scientific SoftwareKenny, D.A. Kashy, D.A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112, 165– 172Mardia, K.V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, 519– 530Marsh, H.W. (1988). Multitrait-multimethod analysis. In J.P. Keeves (Ed.), Educational research, methodology and measurement: An international handbook (pp. 570-580). Oxford: PergamonMarsh, H.W. (1989). Confirmatory factor analysis of multitrait-multimethod data: Many problems and a few solutions. Applied Psychological Measurement, 13, 335– 361Marsh, H.W. Bailey, M. (1991). Confirmatory factor analysis of multitrait-multimethod data: A comparison of alternative models. Applied Psychological Measurement, 15, 47– 70Marsh, H.W. Barnes, J. Hocevar, D. (1985). self-other agreement on multidimensional self-concept ratings: Factor analysis and multitrait-multimethod analysis. Journal of Personality and Social Psychology, 49, 1360– 1377Marsh, H.W. Grayson, D. (1995). Latent variable models of Multitrait-multimethod data. In R.H. Hoyle (Ed.), Structural equation modeling. Concepts, issues, and applications (pp. 177-198). Thousand Oaks, CA: SageMarsh, H.W. Hocevar, D. (1983). Confirmatory factor analysis of multitrait-multimethod matrices. Journal of Educational Measurement, 20, 231– 248McCrae, R.R. (1989). Why I advocate the Five Factor model: Joint Analysis of the NEO-PI and other instruments. In D.M. Buss, & N. Cantor (Eds.), Personality psychology: Recent trends and emerging directions (pp. 237-245). New York: Springer-VerlagMcCrae, R.R. (1990). Traits and trait names: How well is Openness represented in natural languages?. European Journal of Personality, 4, 119– 129McCrae, R.R. Costa, P.T. (1985). Comparison of EPI and Psychoticism scales with measures of the five-factor model of personality. Personality and Individual Differences, 6, 587– 597McCrae, R.R. Costa, P.T. (1987). Validation of the five-factor model of personality across instrument and observers. Journal of Personality and Social Psychology, 52, 81– 90McCrae, R.R. John, O.P. (1992). An introduction to five-factor model and its applications. Journal of Personality, 60, 175– 215Norman, W.T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66, 574– 583Ostendorf, F. (1990). Sprache und Personlichkeitsstruktur: Zur Validität des Funf-Faktoren-Modells der Persönlichkeit. [Language and personality structure: On the structure and validity of the Five-Factor Model of Personality]. Regensburg, Germany: RodererOstendorf, F. Angleitner, A. (1992). On the generality and comprehensiveness of the Five-Factor Model of personality: Evidence of Five Robust Factors in Questionnaire data. In G.V. Caprara & G. Van Heck (Eds.), Modern personality psychology (pp. 73-109). London: Harvester WheatsheafOzer, D.J. (1989). Construct validity in personality assessment. In D.M. Buss, & N. Cantor (Eds.), Personality psychology: Recent trends and emerging directions (pp. 201-209). New York: Springer-VerlagOzer, D.J. Reise, S.P. (1994). Personality assessment. Annual Review of Psychology, 45, 357– 388Pervin, L.A. (1994). A critical analysis of current trait theory. Psychological Inquiry, 5, 103– 113Schmitt, N. Stults, D.M. (1986). Methodological review: Analysis of multitrait-multimethod matrices. Applied Psychological Measurement, 10, 1– 22Steiger, J.H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173– 180Tanaka, J.S. (1993). Multifaceted conceptions of fit in structural equation models. In K.A. Bollen & J.S. Long (Eds.), Testing structural equation models (pp. 10-39). Newbury Park, CA: SageWidaman, K.F. (1985). Hierarchical nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1– 26Wiggins, J.S. Pincus, A.L. (1992). Personality: structure and measurement. Annual Review of Psychology, 43, 473– 504Wothke, W. (1993). Nonpositive definite matrices in structural modeling. In K.A. Bollen & J. Scott Long (Eds.), Testing structural equation models (pp. 256-293). Newbury Park, CA: SageYang, K. Bond, M.H. (1990). Exploring implicit personality theories with indigenous or imported constructs: The Chinese case. Journal of Personality and Social Psychology, 58, 1087– 1095
