Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Open AccessOriginal Article

Measurement Invariance of the Short Dark Tetrad Across Cultures and Genders

Published Online:June 01, 2022https://doi.org/10.1027/1015-5759/a000715

Abstract

Abstract: The last two decades revealed a plethora of scientific examinations on the Dark Triad (narcissism, psychopathy, Machiavellianism) and Dark Tetrad traits (Dark Triad + sadism) in a variety of contexts. Short scales for the assessment of these traits have been very influential and widely used. Building upon previous research, the 28-item Short Dark Tetrad (SD4) was introduced as a measure for the assessment of the Dark Tetrad traits. A recent study found that the SD4 is invariant across genders, but little is known concerning invariance across cultures. Therefore, we tested measurement invariance (MI) between German and US participants. Additionally, we replicated extant findings on MI across genders. The analyses suggested configural MI across cultures, metric MI between genders in a US sample, and scalar MI across genders in a German sample. To address that the SD4 revealed only a modest fit in the samples, we further computed Exploratory Structural Equation Models. Those were mostly consistent with the original model structure and indicated that adding marginal cross-loadings among the factors accounts for enhanced model fit. Possible explanations for the findings related to MI were discussed.

Since the Dark Triad was introduced (Paulhus & Williams, 2002), research on the area of antagonistic personality traits has been burgeoning. The Dark Triad comprises subclinical forms of narcissism (i.e., striving for admiration by others; restoration of the grandiose self after ego threats), Machiavellianism (Mach; i.e., cynical, distrustful, strategic orientation; self-control), and psychopathy (i.e., impulsivity, aggression, and antisocial behavior; Back et al., 2013; Jones, 2017; Paulhus & Williams, 2002, Skeem et al., 2011). Previous work illustrated the importance of these traits in a variety of everyday contexts, such as romantic relationships (Jonason et al., 2012), school (Stellwagen & Kerig, 2013), and work settings (O’Boyle et al., 2012). The Dark Triad was recently expanded by sadism (i.e., deriving feelings of joy from hurting others or seeing others suffer) and thus became a Dark Tetrad (Chabrol et al., 2009; Paulhus, 2014). As the instruments originally proposed by Paulhus and Williams revealed undesired overlaps (especially psychopathy and Mach scales; Grosz et al., 2020) or suffered from unfavorable content coverage (Paulhus & Jones, 2015), a plethora of research dealt with the development and evaluation of measures (e.g., Blötner & Bergold, 2021; Jonason & Webster, 2010; Jones & Paulhus, 2014; Paulhus & Jones, 2015). Specialized short scales focusing on each trait’s specifics became increasingly popular and reduced some of the issues of earlier scales. One of those is the Short Dark Tetrad (SD4; Paulhus et al., 2020). It is the successor of the widely used Short Dark Triad (SD3; Jones & Paulhus, 2014). The SD4 can be validly interpreted concerning central correlates of narcissism, psychopathy, Mach, and sadism (Blötner et al., 2021; Neumann et al., 2021). However, ensuring an instrument’s nomological network is not yet sufficient to assume its usefulness. Beyond being in line with theoretical expectations about the underlying constructs, the structural stability of the measure must be examined across different groups (measurement invariance [MI]). In doing so, users can be sure that differences between groups indicate different manifestations of the latent trait – as opposed to different measurement properties. Testing of MI comprises a hierarchical process of imposing more and more restrictions on a latent model involving two or more independent groups (or two or more measurement occasions within one person in the case of longitudinal MI). The most common tests of equality refer to the item-factor composition (configural MI), loadings (metric MI), and item intercepts (scalar MI) across groups. Numerous measures cannot withstand more severe restrictions (Putnick & Bornstein, 2016).

Thus far, the SD4 has been used in German, US, and Canadian samples (Blötner et al., 2021; Furnham & Horne, 2021; Neumann et al., 2021; Paulhus et al., 2020, 2021), and MI has been demonstrated across genders (Neumann et al., 2021), but the degree of invariance is yet unclear concerning different cultures (Blötner et al., 2021). Manifestations of the Dark Tetrad traits could differ across cultures as cultural norms allow, dictate, or prohibit particular behaviors to members of particular groups (Eagly & Wood, 1991; Hofstede et al., 2010) and, therefore might shape expressions of the Dark Tetrad. To ensure meaningful interpretations of the SD4 subscales across cultures, we tested whether it is invariant between samples from different countries. Furthermore, we replicated the gender-related analyses of MI conducted by Neumann et al. (2021).

Method

Samples

We used two samples, which we derived from extant studies. First, we used data from Blötner et al.’s (2021) study on the German version of the SD4 (N = 594). Second, and with permission from the authors, we used Webster and Wongsomboon’s (2020) SD4 data involving participants from the US (N = 451, complete data available for 428 participants). Since the two samples differed regarding the expected (Webster & Wongsomboon, 2020, suggested that their participants were between 18 and 23 years of age) or observed age distributions (Blötner et al., 2021: Mage = 28.4, SDage = 9.0, ranging from 18 to 79 years), we carried out the culture-related analyses in two different ways. First, we computed the analyses using the total samples. Second, we restricted the German sample to freshmen between ages 18 and 23 (N = 170) so that the samples agree regarding the age ranges (Webster & Wongsomboon, 2020). On the other hand, for the analyses of gender-related MI within each sample, we used the whole datasets, and excluded participants with non-binary gender (n = 7 in Blötner et al., 2021; n = 4 in Webster & Wongsomboon, 2020), and computed sample-wise analyses.

Measures

Dark Tetrad

Webster and Wongsomboon (2020) and Blötner et al. (2021) presented the English and the German version of the SD4 (Paulhus et al., 2020), respectively, to assess narcissism, psychopathy, Machiavellianism, and sadism. Each scale comprises seven items. Five-point Likert scales serve as response scales (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree). Estimations of reliability ranged from Cronbach’s α = .63 to .78 (see Table 1).

Table 1 Cronbach’s α coefficients of the subscales of the Short Dark Tetrad per sample

Analysis Plan

Measurement Invariance Between Cultures

We used the R package semTools (version 0.5-5; Jorgensen et al., 2021) to examine configural, metric, and scalar MI (MLR estimator). We considered particular levels of MI to be ensured if the difference between the fit measures of the respective models were smaller than the cut-offs proposed by Chen (2007; ΔCFI ≤ .010, accompanied by ΔRMSEA ≤ .015). Differences between the models were thereby due to imposing equality restrictions to specific parameters among groups compared to free estimations of these parameters. We deemphasized the Δχ2-test as it is overly sensitive to negligible changes and the SRMR as it lacks sensitivity to detect non-invariance (Chen, 2007). We did not test partial MI because the SD4 subscales are very concise – yielding a potentially high ratio of non-invariant items per factor – and because there is no consensus on an acceptable percentage of non-invariant items (Putnick & Bornstein, 2016).

Measurement Invariance Between Men and Women

To replicate Neumann et al.’s (2021) analyses concerning MI across genders, we carried out analyses of gender-related MI per sample. Therefore, we applied the same procedures as we did in the analyses of MI between cultures.

Results and Discussion

Measurement Invariance Across Cultures

When using the entire German and US samples, the descriptive model fit measures changed substantially when imposing equal loadings to German and US data (ΔCFI = −.013, ΔRMSEA = .001; see Table 2). Accordingly, the SD4 revealed configural MI across cultures and factor structures can be meaningfully compared between German and US participants (Putnick & Bornstein, 2016). However, when we computed the analysis with the restricted German sample (i.e., only freshmen between 18 and 23 years of age), the analyses exhibited metric MI between the cultures so that factor loadings can also be meaningfully compared between the samples (please find the respective results in Table S1 in the supplement; Blötner et al., 2022).

Table 2 Tests of measurement invariance of the SD4 across genders and cultures

Measurement Invariance Across Genders

The SD4 revealed scalar MI across genders in the German sample but not in the US sample – as indicated by acceptable or unacceptable changes of the CFIs when imposing equal item intercepts to men and women (cut-off ΔCFI ≤ .010; Chen, 2007; see Table 2). Thus, comparisons of latent means are appropriate between German men and German women, and factor loadings are comparable between US men and women (Putnick & Bornstein 2016).

Exploratory Structural Equation Model

Note that all CFIs in the analyses of MI indicated less than acceptable fit (< .90), whereas all RMSEAs were acceptable (< .08; Hu & Bentler, 1999). However, these findings are consistent with earlier analyses of the SD4 (Blötner et al., 2021; Neumann et al., 2021; Paulhus et al., 2020). To address that standard CFAs are very restrictive (i.e., constraint of cross-loadings onto items of other factors, neglecting overlaps among the traits), and in line with Neumann et al. (2021), we further computed Exploratory Structural Equation Models (ESEM; Asparouhov & Muthén, 2009). By allowing marginal cross-loadings, ESEMs accounted for acceptable fit, CFIs = .90 and .91 in the US and German samples, respectively, both RMSEAs = .04. Table 3 provides the loadings from the CFAs and ESEMs. Two out of 196 possible cross-loadings were non-trivial (i.e., λ ≥ .30), whereas 10 out of 56 expected main-loadings were trivial (28 loadings each estimated in two samples). As can be seen in Figure S1 in the supplement (Blötner et al., 2022), the empirical structure of the SD4 differed from the intended one (Paulhus et al., 2020), with slight differences concerning the Mach, narcissism, and psychopathy subscales and noticeable differences arising for the sadism scale. The sadism items had substantial cross-loadings with psychopathy (sixth sadism item), or their loadings were smaller than conventional cut-offs (third, fifth, sixth, and seventh sadism items).

Table 3 Standardized factor loadings of the Short Dark Tetrad by model type and sample

General Discussion

This study analyzed the degrees of structural equivalence of the SD4 across cultures and genders. The findings suggest configural MI between German and US cultures. Thus, the factor structure can be meaningfully compared between these cultures, whereas comparisons of unstandardized factor loadings and item intercepts are not advisable. However, when we compared German and US participants from the same age ranges, we found hints on metric MI, suggesting that both the factor structure and unstandardized loadings, but not the item intercepts, can be compared between the cultures. Furthermore, we found hints on metric MI between men and women in the US sample, as well as scalar MI between men and women in the German sample. Accordingly, factor loadings (item intercepts) can be compared between genders in the US (German) sample.

Although both of our samples belong to WEIRD cultures (Western, Educated, Intellectual, Rich, Democratic; Henrich et al., 2010), there are moderate to large differences between those, especially regarding individualism and indulgence (both higher for the USA), uncertainty avoidance, and long-term orientation (both higher for Germany; Hofstede et al., 2010). Individualism and indulgence reflect person-centeredness (vs. society-centeredness) and liberal adoption of norms to promote one’s well-being. Valuing own advantages over societal norms and the bending of rules are two outstanding features of all antagonistic traits (Paulhus, 2014). On the other hand, uncertainty avoidance and long-term orientation are crucial features of Mach (Blötner & Bergold, 2022). We suggest that these differences accounted for limited levels of invariance between the cultures. Likewise, social norms imposing more prosocial expectations on women as opposed to men and distinct expressions of antagonistic behaviors among men and women (Muris et al., 2017) may have accounted for scalar non-invariance between men and women in our US sample. In the German sample, we replicated Neumann et al.’s (2021) finding on scalar MI across genders, whereas the findings from our US sample contradict extant literature, despite stemming from the same culture as Neumann et al.’s sample. We assume that scalar MI in Neumann et al.’s (2021) and Blötner et al.’s (2021) respective total samples was due to higher heterogeneity within these datasets as compared to Webster and Wongsomboon’s (2020) sample that entails only students. Blötner et al. and Neumann et al. included a wider array of individuals from the general population, which also affected our analysis of MI across cultures when we included all German participants. However, our samples were not sufficient to test this assumption any further. Hence, we encourage future research to test the equivalence of the SD4 in student samples and samples from the general population by purposefully recruiting from these populations and considering equivalence regarding potentially confounding variables, for instance, by incorporating propensity score matching.

Limitations

Given that we reanalyzed data from existing studies, the present work exhibits the same limitations as the original studies. First, the studies predominantly (i.e., Blötner et al., 2021) or exclusively recruited students (Webster & Wongsomboon, 2020). Second, the gender ratio of the German sample was strongly imbalanced. The gender-related imbalance of the German sample also affected our total sample (i.e., our combined sample used to examine culture-related MI), limiting the generalizability of our results as men score higher on antagonistic traits and behaviors than women (e.g., Muris et al., 2017). The last limitation is specific to the analytic approach in this study: When we matched the age ranges of the two samples to test culture-related MI, our examination involved a comparatively small German subsample. We restricted it to ensure the best possible comparability between the German and US samples regarding crucial characteristics of sample composition (i.e., age and student status). However, the age range is relatively narrow and limited to students of psychology, affecting the external validity. The Dark Tetrad refers to subclinical samples and may therefore have different properties in clinical or forensic samples (Blötner et al., 2021; Neumann et al., 2021). In summary, we encourage future research to test the SD4 in groups that are more heterogeneous as well as more balanced in terms of gender.

Conclusion

Because the items of the SD4 have relatively unambiguous contents, artifacts from the translation process should be unlikely to account for our findings on MI. Differences might rather be due to social expectations about how men and women or Germans and US Americans should or should not behave. Therefore, sample characteristics – especially national culture – should be an important issue for future research on antagonistic traits and behaviors. However, given stark contrasts between the results obtained in the samples that were (not) matched regarding age, the SD4 should only be used in cultural comparisons if the general characteristics of the samples agree.

References

  • Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397–438. https://doi.org/10.1080/10705510903008204 First citation in articleCrossref, Google Scholar

  • Back, M. D., Küfner, A. C. P., Dufner, M., Gerlach, T. M., Rauthmann, J. F., & Denissen, J. J. A. (2013). Narcissistic admiration and rivalry: Disentangling the bright and dark sides of narcissism. Journal of Personality and Social Psychology, 105(6), 1013–1037. https://doi.org/10.1037/a0034431 First citation in articleCrossref, Google Scholar

  • Blötner, C., & Bergold, S. (2022). To be fooled or not to be fooled: Approach and avoidance facets of Machiavellianism. Psychological Assessment, 34(2), 147–158. https://doi.org/10.1037/pas0001069 First citation in articleCrossref, Google Scholar

  • Blötner, C., Webster, G. D., & Wongsomboon, V. (2022, February 22). Data, script, and supplementary materials for Measurement invariance of the Short Dark Tetrad across cultures and genders. https://osf.io/p8v3k/ First citation in articleCrossref, Google Scholar

  • Blötner, C., Ziegler, M., Wehner, C., Back, M. D., & Grosz, M. P. (2021). The nomological network of the Short Dark Tetrad Scale (SD4). European Journal of Psychological Assessment, Advance online publication. https://doi.org/10.1027/1015-5759/a000655 First citation in articleLink, Google Scholar

  • Chabrol, H., Van Leeuwen, N., Rodgers, R., & Séjourné, N. (2009). Contributions of psychopathic, narcissistic, Machiavellian, and sadistic personality traits to juvenile delinquency. Personality and Individual Differences, 47(7), 734–739. https://doi.org/10.1016/j.paid.2009.06.020 First citation in articleCrossref, Google Scholar

  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504. https://doi.org/10.1080/10705510701301834 First citation in articleCrossref, Google Scholar

  • Eagly, A. H., & Wood, W. (1991). Explaining sex differences in social behavior: A meta-analytic perspective. Personality and Social Psychology Bulletin, 17(3), 306–315. https://doi.org/10.1177/0146167291173011 First citation in articleCrossref, Google Scholar

  • Furnham, A., & Horne, G. (2021). The Tetradic Heart of Darkness: Comparing three dark-side instruments. Personality and Individual Differences, 179, Article 110918. https://doi.org/10.1016/j.paid.2021.110918 First citation in articleCrossref, Google Scholar

  • Grosz, M. P., Harms, P. D., Dufner, M., Kraft, L., & Wetzel, E. (2020). Reducing the overlap between Machiavellianism and subclinical psychopathy: The M7 and P7 scales. Collabra: Psychology, 6(1), Article 17799. https://doi.org/10.1525/collabra.17799 First citation in articleCrossref, Google Scholar

  • Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152X First citation in articleCrossref, Google Scholar

  • Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and organizations: Software of the mind. McGraw Hill. First citation in articleGoogle Scholar

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118 First citation in articleCrossref, Google Scholar

  • Jonason, P. K., Luevano, V. X., & Adams, H. M. (2012). How the Dark Triad traits predict relationship choices. Personality and Individual Differences, 53(3), 180–184. https://doi.org/10.1016/j.paid.2012.03.007 First citation in articleCrossref, Google Scholar

  • Jonason, P. K., & Webster, G. D. (2010). The Dirty Dozen: A concise measure of the Dark Triad. Psychological Assessment, 22(2), 420–432. https://doi.org/10.1037/a0019265 First citation in articleCrossref, Google Scholar

  • Jones, D. N. (2017). The nature of Machiavellianism: Distinct patterns of misbehavior. In V. Zeigler-HillD. K. MarcusEds., The dark side of personality: Science and practice in social, personality, and clinical psychology (pp. 87–107). American Psychological Association. https://doi.org/10.1037/14854-005 First citation in articleCrossref, Google Scholar

  • Jones, D. N., & Paulhus, D. L. (2014). Introducing the Short Dark Triad (SD3). Assessment, 21(1), 28–41. https://doi.org/10.1177/1073191113514105 First citation in articleCrossref, Google Scholar

  • Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., Rosseel, Y., Miller, P., Quick, C., Garnier-Villareal, M., Selig, J., Boulton, A., Preacher, K., Coffman, D., Rhemtulla, M., Robitzsch, A., Enders, C., Arslan, R., Clinton, B., Panko, P., Merkle, E., Chesnut, S., … Johnson, A. R. (2021). semTools: Useful tools for structural equation modeling(R package version 0.5–5) . CRAN. https://cran.r-project.org/web/packages/semTools/index.html First citation in articleGoogle Scholar

  • Muris, P., Merckelbach, H., Otgaar, H., & Meijer, E. (2017). The malevolent side of human nature. Perspectives on Psychological Science, 12(2), 183–204. https://doi.org/10.1177/1745691616666070 First citation in articleCrossref, Google Scholar

  • Neumann, C. S., Jones, D. N., & Paulhus, D. L. (2021). Examining the Short Dark Tetrad (SD4) across models, correlates, and gender. Assessment, 29(4), 651–667. https://doi.org/10.1177/1073191120986624 First citation in articleCrossref, Google Scholar

  • O’Boyle, E. H., Forsyth, D. R., Banks, G. C., & McDaniel, M. A. (2012). A meta-analysis of the Dark Triad and work behavior: A social exchange perspective. Journal of Applied Psychology, 97(3), 557–579. https://doi.org/10.1037/a0025679 First citation in articleCrossref, Google Scholar

  • Paulhus, D. L. (2014). Toward a taxonomy of dark personalities. Current Directions in Psychological Science, 23(6), 421–426. https://doi.org/10.1177/0963721414547737 First citation in articleCrossref, Google Scholar

  • Paulhus, D. L., Buckels, E. E., Trapnell, P. D., & Jones, D. N. (2020). Screening for dark personalities: The Short Dark Tetrad (SD4). European Journal of Psychological Assessment, 37(3), 208–222. https://doi.org/10.1027/1015-5759/a000602 First citation in articleLink, Google Scholar

  • Paulhus, D. L., Gupta, R., & Jones, D. N. (2021). Dark or disturbed? Predicting aggression from the Dark Tetrad and schizotypy. Aggressive Behavior, 47(6), 635–645. https://doi.org/10.1002/ab.21990 First citation in articleCrossref, Google Scholar

  • Paulhus, D. L., & Jones, D. N. (2015). Measures of dark personalities. In G. J. BoyleD. H. SaklofskeG. MatthewsEds., Measures of personality and social psychological constructs (pp. 562–594). Elsevier Academic Press. https://doi.org/10.1016/B978-0-12-386915-9.00020-6 First citation in articleCrossref, Google Scholar

  • Paulhus, D. L., & Williams, K. M. (2002). The Dark Triad of personality: Narcissism, Machiavellianism and psychopathy. Journal of Research in Personality, 36(6), 556–563. https://doi.org/10.1016/S0092-6566(02)00505-6 First citation in articleCrossref, Google Scholar

  • Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004 First citation in articleCrossref, Google Scholar

  • Skeem, J. L., Polaschek, D. L. L., Patrick, C. J., & Lilienfeld, S. O. (2011). Psychopathic personality. Psychological Science in the Public Interest, 12(3), 95–162. https://doi.org/10.1177/1529100611426706 First citation in articleCrossref, Google Scholar

  • Stellwagen, K. K., & Kerig, P. K. (2013). Dark triad personality traits and theory of mind among school-age children. Personality and Individual Differences, 54(1), 123–127. https://doi.org/10.1016/j.paid.2012.08.019 First citation in articleCrossref, Google Scholar

  • Webster, G. D., & Wongsomboon, V. (2020, July 6). The Hateful Eight (H8): An efficient multifaceted approach to the Short Dark Tetrad (SD4). https://doi.org/10.31234/osf.io/pr4u6 First citation in articleCrossref, Google Scholar