Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

The Measurement Invariance of the Short Dark Triad

Implications for High- and Low-Stakes Contexts

Published Online:May 13, 2020https://doi.org/10.1027/1614-0001/a000322

Abstract

Abstract. Given the fact that most of the dark personality measures are developed based on data collected in low-stake settings, the present study addresses the appropriateness of their use in high-stake contexts. Specifically, we examined item- and scale-level differential functioning of the Short Dark Triad (SD3; Paulhus & Jones, 2011) measure across testing contexts. The Short Dark Triad was administered to applicant (N = 457) and non-applicant (N = 592) samples. Item- and scale-level invariances were tested using an Item Response Theory (IRT)-based approach and a Structural Equation Modeling (SEM) approach, respectively. Results show that more than half of the SD3 items were flagged for Differential Item Functioning (DIF), and Exploratory Structural Equation Modeling (ESEM) results supported configural, but not metric invariance. Implications for theory and practice are discussed.

References

  • Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397–438. https://doi.org/10.1080/10705510903008204 First citation in articleCrossref, Google Scholar

  • Baysinger, M., Scherer, K. T., & LeBreton, J. M. (2014). Exploring the disruptive effects of psychopathy and aggression on group processes and group performance. Journal of Applied Psychology, 99, 48–65. https://doi.org/10.1037/a0034317 First citation in articleCrossref, Google Scholar

  • Birkeland, S. A., Manson, T. M., Kisamore, J. L., Brannick, M. T., & Smith, M. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14, 317–335. https://doi.org/10.1111/j.1468-2389.2006.00354.x First citation in articleCrossref, Google Scholar

  • Bott, J. P., O’Connell, M. S., Ramakrishnan, M., & Doverspike, D. (2007). Practical limitations in making decisions regarding the distribution of applicant personality test scores based on incumbent data. Journal of Business and Psychology, 22, 123–134. https://doi.org/10.1007/s10869-007-9053-x First citation in articleCrossref, Google Scholar

  • Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts,applications, and programming. Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48, 1–29. https://doi.org/10.18637/jss.v048.i06 First citation in articleCrossref, Google Scholar

  • Chen, F. F. (2007). Sensitivity of goodness of fit indices to lack of measurement invariance. Structural Equation Modeling, 14, 464–504. https://doi.org/10.1080/10705510701301834 First citation in articleCrossref, Google Scholar

  • Chiorri, C., Garofalo, C., & Velotti, P. (2019). Does the dark triad manifest similarly in men and women? Measurement invariance of the Dirty Dozen across sex. Current Psychology, 38, 659–675. https://doi.org/10.1007/s12144-017-9641-5 First citation in articleCrossref, Google Scholar

  • Choi, S. W., Gibbons, L. E., & Crane, P. K. (2011). Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. Journal of Statistical Software, 39, 1–30. https://doi.org/10.18637/jss.v039.i08 First citation in articleCrossref, Google Scholar

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Crane, P. K., van Belle, G., & Larson, E. B. (2004). Test bias in a cognitive test: Differential item functioning in the CASI. Statistics in Medicine, 23, 241–256. https://doi.org/10.1002/sim.1713 First citation in articleCrossref, Google Scholar

  • Dilchert, S., Ones, D. S., & Krueger, R. F. (2014). Maladaptive personality constructs, measures, and work behaviors. Industrial and Organizational Psychology, 7, 98–110. https://doi.org/10.1111/iops.12115 First citation in articleCrossref, Google Scholar

  • Du, L., & Tang, T. L.-P. (2005). Measurement invariance across gender and major: The love of money among university students in People’s Republic of China. Journal of Business Ethics, 59, 281–293. https://doi.org/10.1007/s10551-004-6395-4 First citation in articleCrossref, Google Scholar

  • Egan, V., Hughes, N., & Palmer, E. J. (2015). Moral disengagement, the dark triad, and unethical consumer attitudes. Personality & Individual Differences, 76, 123–128. https://doi.org/10.1016/j.paid.2014.11.054 First citation in articleCrossref, Google Scholar

  • Eysenck, H. J., & Eysenck, M. W. (1985). Personality and individual differences. New York, NY: Plenum Press. First citation in articleCrossref, Google Scholar

  • Furnham, A., Richards, S. C., & Paulhus, D. L. (2013). The dark triad of personality: A 10 year review: Dark triad of personality. Social and Personality Psychology Compass, 7, 199–216. https://doi.org/10.1111/spc3.12018 First citation in articleCrossref, Google Scholar

  • Henry, M. S., & Raju, N. S. (2006). The effects of trailed and situational impression management on a personality test: An empirical analysis. Psychology Science, 48, 247–267. First citation in articleGoogle Scholar

  • Hogan, R., & Hogan, J. (1992). Hogan Personality Inventory Manual. Tulsa, OK: Hogan Assessment Systems. First citation in articleGoogle Scholar

  • Horan, J. M., Brown, J. L., Jones, S. M., & Aber, J. L. (2015). Assessing invariance across sex and race/ethnicity in measures of youth psychopathic characteristics. Psychological Assessment, 27, 657–668. https://doi.org/10.1037/pas0000043 First citation in articleCrossref, Google Scholar

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossref, Google Scholar

  • Ion, A., & Iliescu, D. (2017). The measurement equivalence of personality measures across high- and low-stake test taking settings. Personality and Individual Differences, 110, 1–6. https://doi.org/10.1016/j.paid.2017.01.008 First citation in articleCrossref, Google Scholar

  • Jones, D. N., & Paulhus, D. L. (2014). Introducing the short dark triad (SD3): A brief measure of dark personality traits. Assessment, 21, 28–41. https://doi.org/10.1177/1073191113514105 First citation in articleCrossref, Google Scholar

  • Jonason, P. K., & Webster, G. D. (2010). The Dirty Dozen: A concise measure of the dark triad. Psychology Assessment, 22, 420–432. https://doi.org/10.1037/a0019265 First citation in articleCrossref, Google Scholar

  • Kaiser, R. B., LeBreton, J. M., & Hogan, J. (2015). The dark side of personality and extreme leader behavior. Applied Psychology: International Review, 64, 55–92. https://doi.org/10.1111/apps.12024 First citation in articleCrossref, Google Scholar

  • Krasikova, D., Green, S., & LeBreton, J. M. (2013). Destructive leadership: A theoretical review, integration, and future research agenda. Journal of Management, 39, 1308–13038. https://doi.org/10.1177/0149206312471388 First citation in articleCrossref, Google Scholar

  • Kubarych, T. S., Aggen, S. H., Kendler, K. S., Torgersen, S., Reichborn-Kjennerud, T., & Neale, M. C. (2013). Measurement non-invariance of DSM-IV narcissistic personality disorder criteria across age and sex in a population-based sample of Norwegian twins. International Journal of Methods Psychiatric Research, 19, 156–166. https://doi.org/10.1002/mpr.318 First citation in articleCrossref, Google Scholar

  • LeBreton, J. M., Shiverdecker, L. K., & Grimaldi, E. M. (2017). The dark triad and workplace behavior. Annual Review of Organizational Psychology and Organizational Behavior, 5, 387–414. https://doi.org/10.1146/annurev-orgpsych-032117-104451 First citation in articleCrossref, Google Scholar

  • Lee, P., Mahoney, K. T., & Lee, S. (2017). An application of the exploratory structural equation modeling framework to the study of personality faking. Personality and Individual Differences, 119, 220–226. https://doi.org/10.1016/j.paid.2017.07.029 First citation in articleCrossref, Google Scholar

  • Maples, J. L., Lamkin, J., & Miller, J. D. (2014). A test of two brief measures of the dark triad: The dirty dozen and short dark triad. Psychological Assessment, 26, 326–331. https://doi.org/10.1037/a0035084 First citation in articleCrossref, Google Scholar

  • Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22, 471–491. https://doi.org/10.1037/a0019227 First citation in articleCrossref, Google Scholar

  • Miller, J. D., Dir, A., Gentile, B., Wilson, L., Pryor, L. R., & Campbell, W. K. (2010). Searching for a vulnerable dark triad: Comparing factor 2 psychopathy, vulnerable narcissism, and borderline personality disorder. Journal of Personality, 78, 1529–1564. https://doi.org/10.1111/j.1467-6494.2010.00660.x First citation in articleCrossref, Google Scholar

  • Muris, P., Merckelbach, H., Otgaar, H., & Meijer, E. (2017). The malevolent side of human nature: A meta-analysis and critical review of the literature on the dark triad (narcissism, machiavellianism, and psychopathy). Perspectives on Psychological Science, 12, 183–204. https://doi.org/10.1177/1745691616666070 First citation in articleCrossref, Google Scholar

  • Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide, Los Angeles, CA: Author. First citation in articleGoogle Scholar

  • O’Boyle, E. H., Forsyth, D. R., Banks, G. C., & McDaniel, M. A. (2012). A meta-analysis of the dark triad and work behavior: A social exchange perspective. The Journal of Applied Psychology, 97, 557–579. https://doi.org/10.1037/a0025679 First citation in articleCrossref, Google Scholar

  • O’Boyle, E. H., Forsyth, D. R., Banks, G. C., Story, P. A., & White, C. D. (2015). A meta-analytic test of redundancy and relative importance of the dark triad and five-factor model of personality: Dark triad and the five-factor model. Journal of Personality, 83, 644–664. https://doi.org/10.1111/jopy.12126 First citation in articleCrossref, Google Scholar

  • O’Brien, E., & LaHuis, D. M. (2011). Do applicants and incumbents respond to personality items similarly? A comparison of dominance and ideal point response models. International Journal of Selection and Assessment, 19, 109–118. https://doi.org/10.1111/j.1468-2389.2011.00539.x First citation in articleCrossref, Google Scholar

  • Paulhus, D. L., & Jones, D. N. (2015). Measures of dark personalities. In G. J. BoyleD. H. SaklofskeG. MatthewsEds., Measures of personality and social psychological constructs (pp. 562–594). San Diego, CA: Academic Press. https://doi.org/10.1016/B978-0-12-386915-9.00020-6 First citation in articleGoogle Scholar

  • Paulhus, D. L., & Jones, D. N. (2011, January). Introducing a short measure of the dark triad. Poster presented at the meeting of the Society for Personality and Social Psychology, San Antonio, TX First citation in articleGoogle Scholar

  • Paulhus, D. L., & Williams, K. M. (2002). The dark side of normal personality: Self-report andbehavioral correlates. Unpublished manuscript, University of British Columbia. First citation in articleGoogle Scholar

  • Pauls, C. A., & Crost, N. W. (2005). Effects of different instructional sets on the construct validity of the NEO-PI-R. Personality and Individual Differences, 39, 297–308. https://doi.org/10.1016/j.paid.2005.01.003 First citation in articleCrossref, Google Scholar

  • Persson, B. N. (2019). Searching for Machiavelli but finding psychopathy and narcissism. Personality Disorders: Theory, Research, and Treatment, 10, 235–245. https://doi.org/10.1037/per0000323 First citation in articleCrossref, Google Scholar

  • Persson, B. N., Kajonius, P. J., & Garcia, D. (2017). Testing construct independence in the short dark triad using item response theory. Personality and Individual Differences, 117, 74–80. https://doi.org/10.1016/j.paid.2017.05.025 First citation in articleCrossref, Google Scholar

  • Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004 First citation in articleCrossref, Google Scholar

  • R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/ First citation in articleGoogle Scholar

  • Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. The Journal of Applied Psychology, 87, 517–529. https://doi.org/10.1037/0021-9010.87.3.517 First citation in articleCrossref, Google Scholar

  • Roulin, N., & Bourdage, J. S. (2017). Once an impression manager, always an impression manager? Antecedents of honest and deceptive impression management use and variability across multiple job interviews. Frontiers in Psychology, 8, 29. https://doi.org/10.3389/fpsyg.2017.00029 First citation in articleCrossref, Google Scholar

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Richmond, VA: Psychometric Society. (Psychometric Monograph No. 17). Retrieved from http://www.psychometrika.org/journal/online/MN17.pdf First citation in articleGoogle Scholar

  • Schyns, B. (2015). Dark personality in the workplace: Introduction to the special issue. Applied Psychology: An International Review, 64, 1–14. https://doi.org/10.1111/apps.12041 First citation in articleCrossref, Google Scholar

  • Schmit, M. J., & Ryan, A. M. (1993). The big five in personnel selection: Factor structure in applicant and nonapplicant populations. Journal of Applied Psychology, 78, 966–974. https://doi.org/10.1037/0021-9010.78.6.966 First citation in articleCrossref, Google Scholar

  • Scherer, K. T., Baysinger, M. J., Zolynsky, D., & LeBreton, J. M. (2013). Predicting counterproductive work behaviors with sub-clinical psychopathy: Beyond the five factor model of personality. Personality & Individual Differences, 55, 300–305. https://doi.org/10.1016/j.paid.2013.03.007 First citation in articleCrossref, Google Scholar

  • Somma, A., Pincus, A. L., Fontana, A., Cianfanelli, B., & Fossati, A. (2019). Measurement invariance of three versions of the pathological narcissism inventory across gender-matched Italian adolescent high school and young adult university students. Journal of Psychopathology and Behavioral Assessment, https://doi.org/10.1007/s10862-019-09758-7 First citation in articleGoogle Scholar

  • Soper, D. S. (2018). Significance of the difference between two correlations calculator [Software]. Retrieved from https://www.danielsoper.com/statcalc/calculator.aspx?id=104 First citation in articleGoogle Scholar

  • Spain, S. M., Harms, P., & LeBreton, J. M. (2013). The dark side of personality at work. Journal of Organizational Behavior, 35, 141–160. https://doi.org/10.1002/job.1894 First citation in articleGoogle Scholar

  • Stark, S., Chernyshenko, O. S., Chan, K. Y., Lee, W. C., & Drasgow, F. (2001). Effects of the testing situation on item responding: Cause for concern. The Journal of Applied Psychology, 86, 943–953. https://doi.org/10.1037/0021-9010.86.5.943 First citation in articleCrossref, Google Scholar

  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. https://doi.org/10.1177/109442810031002 First citation in articleCrossref, Google Scholar

  • Vecchione, M., Alessandri, G., & Barbaranelli, C. (2012). The five factor model in personnel selection: Measurement equivalence between applicant and non-applicant groups. Personality and Individual Differences, 52, 503–508. https://doi.org/10.1016/j.paid.2011.11.014 First citation in articleCrossref, Google Scholar

  • Vize, C. E., Lynam, D. R., Collison, K. L., & Miller, J. D. (2018). Differences among dark triad components: A meta-analytic investigation. Personality Disorders: Theory, Research, and Treatment, 9, 101–111. https://doi.org/10.1037/per0000222 First citation in articleCrossref, Google Scholar

  • Watson, D., Stasik, S. M., Ro, E., & Clark, L. A. (2013). Integrating normal and pathological personality: Relating the DSM-5 trait-dimensional model to general traits of personality. Assessment, 20, 312–326. https://doi.org/10.1177/1073191113485810 First citation in articleCrossref, Google Scholar

  • Webster, G. D., & Jonason, P. K. (2013). Putting the “irt” in “dirty”: Item response theory analyses of the dark triad dirty dozen – An efficient measure of narcissism, psychopathy, and Machiavellianism. Personality and Individual Differences, 54, 302–306. https://doi.org/10.1016/j.paid.2012.08.027 First citation in articleCrossref, Google Scholar

  • Wetzel, E., Lang, F. J., Back, M., Vecchione, M., Rogoza, R., & Roberts, B. (2020). Measurement invariance of three narcissism questionnaires across the US, the UK, and Germany. [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/rf6ep First citation in articleGoogle Scholar

  • Wu, J., & LeBreton, J. M. (2011). Reconsidering the dispositional basis of counterproductive work behavior: The role of aberrant personality. Personnel Psychology, 64, 593–626. https://doi.org/10.1111/j.1744-6570.2011.01220.x First citation in articleCrossref, Google Scholar

  • Żemojtel-Piotrowska, M., Piotrowski, J., Rogoza, R., Baran, T., Hitokoto, H., & Maltby, J. (2019). Cross-cultural invariance of NPI-13: Entitlement as culturally specific, leadership and grandiosity as culturally universal. International Journal of Psychology, 54, 439–447. https://doi.org/10.1002/ijop.12487 First citation in articleCrossref, Google Scholar

  • Ziegler, M., & Buehner, M. (2009). Modeling socially desirable responding and its effects. Educational and Psychological Measurement, 69, 548–565. https://doi.org/10.1177/0013164408324469 First citation in articleCrossref, Google Scholar