A Multilevel Multidimensional Finite Mixture Item Response Model to Cluster Respondents and Countries

Ever since Freud (1917) identified anger at the self as a crucial factor in depression the link between psychopathology and self-criticism has been subject to much theorising and research. In a major systematic review, Werner et al. (2019) outlined a number of different models for self-criticism and its measurement that include both self-report and qualitative studies. They highlight that self-criticism is a transdiagnostic vulnerability factor for mental health problems such as eating disorders, depression, suicidality, anxiety, psychotic symptoms, and interpersonal problems (Werner et al., 2019). Self-criticism also affects susceptibility to and persistence of psychopathology (Bergner, 1995; Blatt & Zuroff, 1992; Falconer et al., 2015) and stress (Kupeli et al., 2017) and influences the response to medical and psychological interventions and treatments (Blatt & Zuroff, 2005; Bulmash et al., 2009; Shahar et al., 2015). Löwet al. (2020) offered a systematic review of self-criticism and psychotherapy outcomes showing that the intensity of self-criticism is linked to poor outcomes, highlighting the importance of improving psychotherapy for these individuals. For these and other reasons, it is important to explore the psychometric properties of measures of self-criticism.

There exist a number of self-report measures assessing self-criticism including: the Depressive Experiences Questionnaire (DEQ; Blatt et al., 1976), the Levels of Self-Criticism Scale (LOSC; Thompson & Zuroff, 2004), the Forms of Self-Criticising/Attacking and Self-Reassuring Scale (FSCRS; Gilbert et al., 2004), the Self-Critical Rumination Scale (Smart et al., 2016), and a scale assessing situational state self-criticism, the Self-Compassion and Self-Criticism Scales (SCCS; Falconer et al., 2015). In a systematic review of measures of five scales of self-criticism, Rose and Rimes (2018) noted that many studies lacked methodological rigour but suggested that two scales, the Self-Critical Rumination Scale (Smart et al., 2016) and the FSCRS (Gilbert et al., 2004) were the most robust in terms of psychometric properties.

The FSCRS is unique in distinguishing between forms and functions of self-criticism. This is important because many psychological processes, for example, anger, anxiety, or caring behaviour, can have different forms and functions and will relate to social difficulties and mental health problems in different ways. Some individuals are self-critical because they experience failure and feel they should and could do better, which may be linked to perfectionism (Curran & Hill, 2019; Shahar, 2015), where others are not so much trying to improve but have a more self-hating attitude to self and wanting to get rid of aspects of the self (Gilbert et al., 2004). In a study of psychiatric patients, Castilho et al. (2017) found that concerns of being inadequate and self-hating forms of self-criticism linked to psychopathology in different ways, with self-hating being particularly linked to shame.

Given the importance of these core processes underpinning vulnerability to mental health and other difficulties, a number of authors have argued there is a need for closer inspection of self-criticism across cultures (Lau et al., 2010; Luyten & Blatt, 2013). Indeed, little is known about cross-cultural aspects of self-criticism. The main obstacle to performing cross-cultural research on self-criticism is the lack of invariant measurement tools with a stable factor structure. Therefore, any findings achieved without previous testing of scales’ measurement invariance must be considered with caution. A measure of self-criticism which has been evaluated across cultures is the FSCRS which, as noted above, has been subjected to meta-analytic reviews. A number of studies have reported on the invariance (Halamová et al., 2019) and factor structure (Halamová et al., 2018) of the FSCRS using data from 13 international samples. Thus, it is necessary to examine this scale further so more cross-cultural research on self-criticism is possible.

In order to advance knowledge of the performance of the FCSRS across cultures, we applied a multilevel mixture model to a multinational sample. Evidence of good fit for such a model would yield a valuable practical outcome – the knowledge of how many discrete classes there on the level of individuals and countries. For individuals, we will know how to discretize sum scores (i.e., how many cut-offs are to be used), and for countries, we can obtain important information on the magnitude of similarities and dissimilarities among them. An alternative modelling approach, the widely used multi-group structural equation model, faces well-known problems (Kim et al., 2017) of how to establish scalar measurement invariance which is necessary for a meaningful comparison of latent means (or more precisely, to display acceptable fit of such a model with the data). On the other hand, a multilevel finite mixture model is very useful when the violation of exact invariance and potential clustering of groups is assumed due to measurement non-invariance, factor mean heterogeneity, or both (Kim et al., 2017). Because a multilevel finite mixture model estimates far fewer parameters than a multi-group structural equation model, it is also more parsimonious (Kim et al., 2017). One of the main advantages of the latent class analysis model is the fact that countries and persons are clustered in classes, and that within a class they are more similar than between classes: this method is able to resolve the problems of measurement non-invariance and factor mean heterogeneity in one unified approach.

All previous studies have assumed that self-criticism is a continuous latent ability (characteristic), and to our knowledge, there is no study investigating potential differences and heterogeneities among groups of respondents. Measurement invariance studies focus on the comparison of latent means across countries, but they cannot reveal in principle how many respondents in each country contribute to these differences. Ignoring this fact could lead to failing to notice that a small number of highly self-critical respondents could have approximately the same impact on the latent factor mean as a large number of moderately self-critical respondents. The assumption of a homogenous population with a continuous distribution of the latent ability that differs from another homogenous population with a continuous distribution of the latent ability with a higher (or lower) level might be untenable in practice. The latent class analysis could provide a more fine-grained picture of how self-criticism is distributed within and across countries taking into account potential heterogeneity.

Materials and Methods

Statistical Models and Data Analyses

This section is organized as follows. First, we will define a latent class (mixture) model for polytomous item responses with the IRT parameterization (Bacci et al., 2014; Bartolucci, 2007). Second, we will present the extension and specify the model taking into account the multidimensional structure of the data (3 dimensions), and subsequently, we will present the extension and specify the multidimensional model taking into account the multilevel structure of the data (11 countries). Then we will provide criteria for model selection. Finally, we will briefly describe the steps in our statistical analysis. Since our aim is to provide a practical guide rather than a very detailed mathematical specification, we will limit our explanation only to basic formulations leaving the exact mathematical expressions to the Electronic Supplementary Material (ESM 2).

Unidimensional Latent Class IRT Models

Following Bacci et al. (2014), let X_j denotes the observed response variable for the jth item of the analysed questionnaire, with j = 1, …, r. This variable has l_j levels, indexed from 0 to l_j − 1. In our case, we have 22 items of the FSCRS; each item is scored from 0 to 4 and therefore there are 5 levels, indexed from 0 to 4. In the unidimensional case, we define the probability that a respondent with latent trait (ability) level θ responds by category x to this particular item as:

(1)

Moreover, we define λ_j(θ) as the probability vector λ_j0(θ), …, λ_j,_lj−1(θ))′. Now, we can specify the IRT model for polytomous responses:

(2) where g_x(·) is a specific link function for the category x, and γ_j and β_jx are IRT item parameters, usually called discrimination indices and difficulty levels. We will observe if the assumption that properly defined constraints on these parameters will result in differently parameterized IRT models.

We can have three different kinds of IRT specifications: (a) the specific link function; (b) constraints on discrimination parameters; (c) constraints on difficulty parameters.

Type of Link Function

As far as the link function is concerned, the most widely used IRT models are the graded response model with global logits (Bacci et al., 2014; Samejima 1969), the generalized partial credit model with local logits (Bacci et al., 2014; Muraki, 1992), and the continuation ratio model with sequential logits (Bacci et al., 2014; Fienberg, 1980). See ESM 2 for their mathematical formulations. Our instrument (FSCRS) is assumed to measure a continuous latent ability (self-criticism); therefore we will adopt the graded response (cumulative) model. It is of little interest to estimate the probability of answering adjacent categories, let alone the probability of a sequential process. The conclusion is that only the graded response models will be analyzed in this paper.

Constraints on Discrimination Parameters

We can specify two different types of IRT model models based on constrained or unconstrained discrimination parameters. First, each item might discriminate differently from the other items, and γ_i parameters are freely estimated. Second, all the items discriminate in the same way, that is, γ_j = 1 for all items (the one-parameter graded response model).

Constraints on Item Difficulty Parameters

Again, we can specify two different types of IRT model models based on constrained or unconstrained versions. First, the difficulty parameters β_jx are unconstrained and freely estimated. Second, the difficulty parameters β_jx are constrained so that the distance between difficulty levels from category to category is the same for all items – τ_x is the (same) difficulty of response category x for all j (the rating scale graded response model).

We can combine different constraints to obtain IRT models with very different parameterizations. For clarity, we reproduce in ESM 1a (slightly modified) Table E1 from Bacci et al. (2014).

Multidimensional Latent Class IRT Models

This unidimensional IRT model for polytomous responses can be easily extended to the multidimensional setting. Following Bacci et al. (2014, chap. 3.1), let us define s as the number of different latent traits measured by the items, Θ = (Θ₁, …, Θ_s)′ as a vector of latent variables corresponding to these latent traits, and θ = (θ₁, …, θ_s)′ as one of its possible realizations. This random vector Θ is assumed to have a discrete distribution with k support points, denoted by ξ₁, …, ξ_k, and probabilities π₁, …, π_k, with π_c = p(Θ = ξ_c). In addition, let us define δ_jd to be a dummy variable equal to 1 if item j measures a latent trait of type d and to zero otherwise, with j = 1, …, r and d = 1, …, s. Now we can redefine equation 2 above as follows:

(3)

Note that from the discreteness of the distribution of the random vector Θ, it immediately follows that the manifest distribution of X = (X₁, …, X_r)′ for all subjects in the cth latent class is:

(4)

Of course, all possibilities of different IRT parameterizations (Table 1) apply for multidimensional IRT models as well, including the option to select a different link function. For the exact formulation of the multidimensional IRT model with global logits (the multidimensional graded response model), see ESM 2a. We must add that some constraints on the parameters must be placed to ensure the identifiability of models. It is required that one discriminant index is equal to 1 for each latent trait, and one difficulty parameter is equal to zero for each latent trait.

Table 1 Standard LC models for detecting the optimal number (k) of latent classes on the level of respondents

	Deterministic input			Random input
k	logLik	# par	BIC (min)	logLik	# par	BIC (min)
Note. Models with the lowest BIC value are highlighted in bold.
1	−242,856.7	88	486,501.1	−242,856.7	88	486,501.1
2	−221,477.3	177	444,539.0	−221,477.3	177	444,539.0
3	−213,941.1	266	430,263.2	−213,941.1	266	430,263.2
4	−210,670.3	355	424,518.2	−210,670.3	355	424,518.2
5	−208,307.7	444	420,589.5	−208,307.7	444	420,589.5
6	−207,111.6	533	418,993.9	−206,719.3	533	418,890.2
7	−206,616.3	622	418,800.1	−206,650.0	622	418,070.8
8	−206,195.5	711	418,755.1	−206,399.1	711	419,911.6
9	−206,385.8	800	419,932.3

Table 1 Standard LC models for detecting the optimal number (k) of latent classes on the level of respondents

View as image HTML

Multidimensional Multilevel Latent Class IRT Models

The multilevel generalization assumes that responses are a multivariate dependent variable and their hierarchical structure is expressed by respondents (units on first level) who are nested in countries (units on second level). This second-level is in itself a discrete latent variable. To comply with previous work in this field (Bacci & Gnaldi, 2015) and to avoid any confusion concerning various indices, we rename and redefine all indices as follows (Bacci & Gnaldi, 2015, p. 931): j is the generic item (j = 1, …, J), i is a respondent or unit on first level of the latent class k (i = 1, …, I; k = 1, …, K), h is a country (or any group) or unit on second level of the latent class c (h = 1, …, H; c = 1, …, C), and Y_ji is the response to item j of respondent i which might assume L ordered categories (l = 0, …, L − 1). We will also define the D-dimensional vector of latent variables, one of its possible realizations being , with a = 1 for units on first level units and a = 2 for units on second level units. Θ⁽¹⁾ has support points and corresponding weight , with k = 1, …, K( > 0, = 1), and Θ⁽²⁾ has support points , corresponding weight , with c = 1, …, C( > 0, = 1). Finally, λ_j(θ) = (λ_j0(θ), …, λ_j,L−1(θ)), with θ = (θ⁽¹⁾θ⁽²⁾) and λ_jl(θ) = p(Y_ji = l|θ). Now, the multidimensional multilevel IRT model is framed as follows:

(5) where j = 1, …, J; k = 1, …, K; c = 1, …, C, γ_j is the discrimination index for item j, β_jl is the difficulty of item j and cut-off point l (l = 1, …, L −1), and δ_jd is an indicator variable assuming value 1 if item j measures dimension (or latent trait) d (d = 1, …, D), and zero otherwise. Additionally, in equation 5, and are the random effects of first- and second-level, or more specifically represents the value of the latent variable corresponding to latent trait d for a respondent who belongs to first-level latent class k, whereas is the value of the latent variable corresponding to latent trait d for a country (group) belonging to second-level latent class c.

Each latent variable has a discrete distribution, characterised by a finite number of support points (or mixture components) ξ and related mass probabilities π. Each support point ξ identifies a latent class of individuals; individuals belonging to the same latent class share the same level of the latent variable. Standardized support points are directly comparable with particular locations in a continuous latent ability from a standard IRT model.

We should add that, in this formula, weights and do not depend on observable covariates, and therefore and . This means that respondents (or countries) come from K (or C) latent classes which are homogenous in terms of the characteristics measured by the questionnaire (self-criticism). This model could be extended to include covariates, but we will not pursue this issue further here (see Bacci & Gnaldi, 2015 for such an extension).

For the sake of brevity, we will not provide here either a matrix formulation or mathematical description of the class of estimation procedures employed to fit models, expected maximised (EM) estimation algorithms (Bacci et al., 2014; Bartolucci, 2007; Vermunt, 2003, 2008 for details).

Criteria of Model Selection

The key issue in any latent class analysis is to select the best fitting model, especially the number of latent classes on both levels. Various methods have been proposed, all of them based on the log-likelihood function (for a review, see McLachlan & Peel, 2000). They can be broadly divided into two categories: parsimony-based (information criteria) or testing-based (likelihood-ratio tests). We will not discuss all their advantages and disadvantages, but rather focus on summarising the most frequently used approach to justify our procedure of model selection.

Likelihood-ratio tests are widely used, but their main problem for finite mixture models is that the parameter vector lies on the boundary of the parameter space under the null hypothesis. The usual asymptotic null distribution of the likelihood-ratio test statistics is therefore not valid (Melnykov & Maitra, 2010). A recently proposed procedure (Maitra & Melnykov, 2010) to overcome this issue is not yet implemented in any available statistical software; therefore we refrain from using the likelihood-ratio testing. On the other hand, of the two most frequently used information criteria, Akaike Information Criterion (AIC) typically substantially overestimates the number of classes (Melnykov & Maitra, 2010, p. 88), while Bayes-Schwarz Information Criterion (BIC) has been shown to demonstrate good performance (Keribin, 2000). However, these come with two caveats: first, the BIC tends to underestimate the number of components of a model when sample sizes are small (Melnykov & Maitra, 2010, p. 88). Fortunately, the sample size used in this article is large enough to minimise this issue. Second, to obtain a meaningful comparison of model fit from one model structure to another, improvements in BIC should be substantive to provide strong evidence; Kass and Raftery (1995) recommend differences greater than 10. To summarise, we will use the BIC for model selection, but only differences greater than 10 will be regarded as constituting strong evidence in favour of the respective model.

Statistical Procedure

We used R package for the statistical analysis, multiLCIRT package (Bartolucci et al., 2014; see ESM 3 for all R-codes).

• (a)
  Descriptive statistics will be provided, namely distributions of the FSCRS item responses (raw percentage frequencies for each item category) and descriptives of raw scores of all three sub-dimensions (Inadequate Self, Reassured Self, Hated Self) of the FSCRS for each country.
• (b)
  Selection of the optimal number k of latent classes on the level of respondents: Following recommendations in Bacci et al. (2014, chap. 4.2.2), we will adopt the standard latent class model, characterized by one dimension for each item; therefore, no choice on the item parameterization is required. We note that we will not test the dimensionality of the FSCRS; it is known that this instrument is three-dimensional, and we will rely on this previous knowledge (Gilbert et al., 2004). Second, we will not test the fit of models with different logit link functions: we have justified theoretically the choice of the global logits (Graded Response Model). Our procedure will consist of fitting models with increasing numbers of latent classes and stopping this procedure when the value of BIC of the tested model is greater (and not lower) than the value of the previously tested model. We will also stop this procedure when the value of BIC of the tested model is less than 10 lower than the value of the previously tested model. In both cases, the previously tested model will be retained.

A crucial problem with standard latent class models is the multimodality of the likelihood function, that is, their tendency to find a local, rather than global, maximum point. In order to avoid such an outcome, we will repeat the estimation process by randomly varying the starting values of the model parameters, in other words, repeating the procedure of testing as a whole with random starting values. Values of log-likelihood, number of parameters, and BIC will be reported, and the final model with the optimal number k of latent classes (the smallest BIC value) will be specified.

• (c)
  Selection of the item discriminating and difficulty parameterization: This step consists of the choice of the possible constraints on the discriminating and difficulty parameters (see above, Table 1). Four different types of models (with k latent classes specified in the previous step) will be fitted by combining free or constrained γ_j parameters with free or constrained β_jx parameters. Again, the final model with the optimal IRT parameterization (the smallest BIC value) will be specified.
• (d)
  Selection of the optimal number u of latent classes on the level of countries: Again, our procedure will consist of fitting three-dimensional models (parameterized in accordance with the best fitting model from the previous step) with increasing numbers of latent classes on the level of countries (starting with the model with an appropriate number of latent classes based on the level of respondents), and stopping this procedure when the value of BIC of the tested model will be greater (and not lower) than the value of the previously tested model. We will also stop this procedure when the value of BIC of the tested model is less than 10 lower than the value of the previously tested model. In both cases, the previously tested model will be retained.
• (e)
  Interpretation and visualization of estimated parameters of the final model: We will report support points and weights (membership in latent classes) for each latent class, both on the level of respondents and countries. Note that to ensure the identifiability of models, one discriminant index was set to 1 for each latent trait, and one difficulty parameter was set to 0 for each latent trait. However, it is possible to express estimates of the model parameters under the alternative identifiability constraint; the latent abilities on the level of respondents could be standardized (with mean 0 and variance 1). As a consequence, all parameters are more easily and directly interpretable in the IRT context; for example, it would be possible to compare standardized support points of finite mixture models with parameters of classical IRT models with continuous latent abilities. Moreover, the standardized latent abilities on the level of respondents will enable us to obtain the distribution of the estimated average abilities for all latent classes on the level of countries; having the standardized latent abilities on the level of respondents, we can easily compute the mean values for each class of countries, taking into account the weights of each respondents’ latent class. We will therefore report the standardized support points on the respondents’ level, the estimated average abilities for all latent classes on the level of countries, together with their weights (membership in latent classes), and their visualization. All IRT parameters for each item (standardized discrimination and difficulty indices) will also be reported.

Results

1. (1)
   Descriptives: after inspecting the distribution of the FSCRS item responses (i.e., raw percentage frequencies, see Table E1b in ESM 1) and raw scores of sub-dimensions of the FSCRS pooled across countries (see Table E1c in ESM 1), we can clearly see that responses are hugely skewed to the right (positive skewness), especially for the Hated Self sub-dimension. We note again that responses for the sub-dimension Reassured Self were reverse-scored. This result justifies the use of the logistic (non-linear item-response theory) method because the obsersssved distribution cannot be approximated by a multivariate normal distribution.
2. (2)
   Selection of the optimal number k of latent classes on the level of respondents: after fitting a series of standard LC models for detecting the optimal number (k) of latent classes on the level of respondents, we can conclude that models with deterministic outputs suggest k = 8 (Table 1). However, after re-fitting these models with random inputs to rule out the possibility that the likelihood function has reached a local rather than global maximum, we had to modify such a conclusion; models with random starting values suggest k = 7. This outcome underlines the importance of careful checking of fitted models to avoid relying on biased results. We will therefore retain the model with 7 latent classes on the level of respondents.
3. (3)
   Selection of the item discriminating and difficulty parameterization: After fitting four three-dimensional models with different IRT parameterizations, it is clear that the most complex model (the Graded Response Model [GRM] with freely estimated discriminations and unconstrained difficulties) fits the data better than its alternatives (Table 2). We will continue with this model in our subsequent testing.
4. (4)
   Selection of the optimal number u of latent classes on the level of countries: After fitting a series of three-dimensional GRM models with an increasing number of latent classes on the level of countries, we can observe that the model with 5 latent classes is the best option (Table 3).
5. (5)
   Parameters of the final model: In Table 4, the values of standardized support points and weights (membership in latent classes) for each latent class on the level of respondents are reported (see previous section for the justification of the standardization procedure). These results reflect the positive skewness in responses; the membership in the first three latent classes is 45%, whereas the membership in the last 3 latent classes is only 37% (see Figure 1). This figure displays the position of support points for individual respondents on the scale of the latent ability (x-axis), and the percentage of respondents in each latent class (y-axis). Furthermore, in the sub-dimension Hated Self, only 3 support points (and corresponding latent classes) have negative values of latent ability (self-criticism) in comparison to 4 support points in the sub-dimension Inadequate Self. This agrees with the theoretical assumption that the Inadequate Self is relatively sensitive to low levels of latent Inadequate Self, whereas the Hated Self is relatively more sensitive to high levels of Hated Self (with potential clinical implications). Support points (and latent classes) of the sub-dimension Hated Self include more highly self-critical respondents than support points of the sub-dimension Inadequate Self. This is clearly reflected in IRT parameters as well (see Table E1d in ESM 1): difficulty indices (thresholds) for the sub-dimension Hated Self are clearly shifted to higher values of the latent ability (self-criticism) which means that endorsing the higher levels of observed responses requires higher levels of the latent ability. Note that responses in the sub-dimension Reassured Self were reverse-scored, so support points in this sub-dimension follow the same scaling.

Table 5 summarizes estimated average abilities, weights, and membership for all latent classes on the level of countries. Also, on this level, the distribution is clearly positively skewed to the right see Figure 2. This figure displays the position of support points for countries on the scale of the latent ability (x-axis) and the percentage of respondents in each latent class (y-axis) on the level of countries. If we inspect the membership of a particular country, the less self-critical class consists of participants from Portugal and Israel (class 1), and the most self-critical classes are composed of participants from Taiwan (class 4) and Japan (class 5).

Table 2 IRT item parameterization selection for the three-dimensional model with 7 latent classes

Model	logLik	# par	BIC (min)
Note. Models with the lowest BIC value are highlighted in bold.
GRM	−208,621.0	131	418,414.6
RS-GRM	−209,726.2	68	420,061.1
1P-GRM	−209,763.0	112	420,528.4
1P-RS-GRM	−210,480.3	49	421,399.2

Table 2 IRT item parameterization selection for the three-dimensional model with 7 latent classes

View as image HTML

Table 3 Multilevel graded response finite mixture models with 7 latent classes for detecting the optimal number (u) of latent classes on the level of countries

u	logLik	# par	BIC (min)
Note. Models with the lowest BIC value are highlighted in bold.
1	−208,621.0	131	418,414.6
2	−208,473.5	138	418,182.3
3	−208,308.1	145	417,914.0
4	−208,175.3	152	417,711.2
5	−208,106.2	159	417,635.6
6	−208,189.9	166	417,865.7

Table 3 Multilevel graded response finite mixture models with 7 latent classes for detecting the optimal number (u) of latent classes on the level of countries

View as image HTML

Table 4 Standardized support points and membership for 7 latent classes (respondents)

Dimension	Class 1	Class 2	Class 3	Class 4	Class 5	Class 6	Class 7
Inadequate Self	−1.497	−1.270	−0.257	−0.157	0.793	1.117	2.263
Reassured Self	−1.860	−0.717	−0.223	−0.059	0.761	1.147	2.116
Hated Self	−1.480	−1.264	−0.445	0.249	0.529	1.185	2.330
Membership	11.16%	12.55%	21.20%	17.92%	18.21%	14.37%	4.59%

Table 4 Standardized support points and membership for 7 latent classes (respondents)

View as image HTML

Table 5 Estimated average abilities, weights, and membership for all latent classes on the level of countries

	Class 1	Class 2	Class 3	Class 4	Class 5
Note. AUS = Australia (N = 319); CAN = Canada (N = 380); CH = Switzerland (N = 230); ISR = Israel (N = 475); ITA = Italy (N = 393); JAP = Japan (N = 263); NL = The Netherlands (N = 363); POR = Portugal (N = 764); SVK = Slovakia (N = 1,326); TAI = Taiwan (N = 417); UK1 = United Kingdom 1 (N = 1,570); UK2 = United Kingdom 2 (N = 883); USA (N = 331).
Support points	−0.195	−0.035	0.026	0.082	0.378
Membership	16.06%	43.36%	31.76%	5.41%	3.41%
Countries	ISR, POR	CAN, ITA, NL, SVK, UK2	AUS, CH, UK1, USA	TAI	JAP

Table 5 Estimated average abilities, weights, and membership for all latent classes on the level of countries

View as image HTML

Discussion

In this study, we tested the multilevel multidimensional finite mixture item response model of the Forms of Self-Criticising/Attacking and Self-Reassuring Scale to cluster respondents and countries. We analysed the relationships between observed characteristics and the latent trait at the different levels (individuals and countries), and across different dimensions of FSCRS.

Results showed that respondents’ scores were dependent on the unobserved (latent class) individual and countries membership and multidimensional structure of the instrument, and justified the use of a multilevel multidimensional finite mixture item response model in comparative psychological assessment. However, a very important question is if latent classes on the individual level and on the country level are meaningfully different classes, or are they merely an arbitrary discretization of a continuum which fits the data? This question is very difficult to answer without including covariates that could predict membership in latent classes (both on the individual and on a country level), for example, gender and age differences, social status, childhood experiences. In fact, we cannot speculate about the sources and causes of the heterogeneity without such data and models, but we do think that these latent classes are not arbitrary; patterns of responses which assign individuals into latent classes are systematic and are probably driven by yet unknown sets of social and psychological sources. Further research is required to resolve this problem.

According to the findings, there are some items of the FSCRS which discriminate better than others. Related to Hated self, items 10 (“I have a sense of disgust with myself”) and 22 (“I do not like being me”) have the best discriminations while items 15 (“I call myself names”) discriminates less well. However, in terms of difficulties, all items from the Hated Self sub-scale are clearly more informative about highly self-critical participants compared with items from the Inadequate Self sub-scale. This may be partly because self-hatred represents a more pathogenic form of self-criticism and is linked to shame (Castilho et al., 2017).

Items from the Inadequate Self sub-scale provide balanced information across the whole range of the latent ability (self-criticism). Therefore, they are informative of participants with high, low as well as average levels of self-criticism. The best discriminating items from the Inadequate Self sub-scale are items 7 (“I feel beaten down by my own self-critical thoughts”) and 6 (“There is a part of me that feels I am not good enough”) and the worst is item 18 (“I think I deserve my self-criticism”).

All items from the Reassured Self subscale have low discriminations in comparison with items from the Inadequate and Hated Self subscales. The Reassured Self sub-scale has low discrimination specifically on the latent trait of (reversed) self-reassurance. This may be because the items in this sub-scale were reverse scored and the content of the items is positively phrased. It may also mean that self-reassurance is not the opposite of self-criticism on a unipolar continuum. Although self-reassurance can be an antidote for self-criticism it operates through completely different neurophysiological systems to those of self-criticism (Longe et al., 2010).

As high self-criticism is strongly linked to psychopathology (Werner et al., 2019), the distribution of self-criticism is positively skewed as various forms of psychopathology are. Consequently, the most self-critical people (class 6 and 7 combined is 18.9%) are less common in the general population than the least self-critical people (class 1 and 2 combined is 33.7%).

Similar to the findings of Halamová et al. (2019), participants from Japan are the most self-critical, followed by Taiwan, while the least self-critical participants are from Israel and Portugal. In addition to their findings, in this article, we found that there are 5 classes of countries according to their level of self-criticism and 7 classes of participants according to their level of self-criticism. In the future these could be used for diagnostic purposes; at the level of individuals, each sub-score for the particular dimension could be discretized into 7 classes, and norms for individuals should be created and used taking into account this latent class structure. For example, it would be inadvisable to distinguish sub-scores from any sub-dimension into “highly self-critical”, “average”, or “low self-critical” individuals without taking into account weighting (see Table 4); for example, “lowest self-critical group” (class 1) should contain only around 11% of individuals, and “highest self-critical” group (class 7) only around 5% of respondents. Secondly, we should be careful when comparing the sum scores from different countries. If countries do not belong to the same latent class, such comparisons are not warranted and could be biased, because measurement invariance is not guaranteed.

Conclusion

In conclusion, the FSCRS is a well-designed self-report measure with good discrimination of highly self-critical participants. Therefore, we can recommend using the FSCRS for measuring self-criticism. From the practical point of view, Inadequate and Hated Self are functionally different and certainly have different distributions. Our results go also in contradiction to the suggestion of some that Inadequate and Hated Self should or may be combined into a global self-criticism score (Halamová et al., 2018).

Electronic Supplementary Material