Abstract: The clinician-rated Yale-Brown Obsessive Compulsive Scale (Y-BOCS) is a widely used assessment tool for obsessive-compulsive disorder (OCD). However, the measurement invariance (MI) properties of the Y-BOCS, a prerequisite for group or time point comparisons in clinical research, have received little attention in previous studies. In this study, we aim to comprehensively investigate the factor structure and MI of the Y-BOCS severity rating and its symptom checklist, utilizing a large sample of OCD patients (N = 1,066). Our analysis considers various MI covariates, including time (pre- and post-therapy), severity, comorbidity, previous treatments, and demographics. Overall, the majority of tests conducted on the Y-BOCS severity rating and its symptom checklist revealed no substantial issues with MI, reinforcing the validity of the Y-BOCS for comparative clinical research. Specifically, we discuss a three-factor model for the severity rating, contrasting with a two-factor model for obsessions and compulsions when excluding the resistance items. Notably, our findings underscore the advantages and validity of employing latent factors rather than sum scores to model OCD severity and symptoms.

For the assessment of obsessive-compulsive disorder (OCD), the Yale-Brown Obsessive Compulsive Scale (Y-BOCS; Goodman et al., 1989) represents the gold standard as the dominant clinician-rated instrument (Deacon & Abramowitz, 2005; Fatori et al., 2020).

The Y-BOCS comprises two distinct parts: (1) an assessment of the severity of OCD symptoms (Y-BOCS-SR), usually viewed as its core section, and (2) an extensive symptom checklist (Y-BOCS-SCL) to gauge the occurrence of specific OCD symptoms both in the present and the past. Due to its widespread use, an impressive body of research dealing with psychometric aspects of the Y-BOCS has accumulated in the last 30 years, even allowing meta-analysis of factor analytical studies (Bloch et al., 2008). Despite this research effort, studies vary considerably in the reported factor structures, hampering replication in subsequent samples. Thus, the factor structure of the Y-BOCS is still a matter of debate and proposed models need validation (for recent overviews regarding severity and symptom scales see e.g., Fatori et al., 2020; Schulze et al., 2018). On top of the issue of factorial structure, another psychometric aspect has broadly been neglected in the literature: measurement invariance (MI). MI refers to the stability of measurement properties over time or in group comparisons. In a clinical setting, MI is especially important when assessing the outcome of a treatment, as valid conclusions on the assessed construct can only be drawn if the measurement properties themselves remain unchanged.

Empirical studies often show surprising violations of MI even in highly standardized and broadly used assessment tools (Fried et al., 2016; Wicherts, 2016). In general, MI properties of the Y-BOCS were rarely reported in the past, even in studies focusing on psychometrics. We, therefore, lack information about MI in OCD assessment.

The present study thus evaluates the baseline factorial structure of the Y-BOCS and provides an extensive investigation into MI for many clinically relevant variables, like comparing before and after therapy, overall impairment, and comorbidity among others.

Measurement Invariance

The conceptual definition of MI as the stability of a measurement instrument translates to the mathematical properties of a measurement model. In factor analysis, such models comprise three item characteristics: loadings, intercepts, and residuals. MI holds if these model parameters are unaffected by a covariate, i.e., multiple time points, a grouping variable like gender, or a continuous variable like age. Global tests for MI thus consist of comparing a model with fixed parameters to another one where parameters depend on the covariate. The three-item parameters yield several stages of invariance tests (Meredith, 1993): Initially, weak MI implies a similar factor structure (configural invariance) and equal factor loadings. These conditions are viewed as prerequisites for comparisons of regression slopes between groups (Chen, 2007) and are therefore sufficient in correlational studies. The second stage, called strong invariance, adds equal item intercepts to the model. Strong MI is a prerequisite if latent mean scores are to be compared between groups or time points (Chen, 2008). Finally, as a third and last step, strict invariance represents the additional equality of the item residuals. This stage assures equality in reliability.

In this paper, we will focus on strong invariance, as comparisons of latent means make up the major portion of clinical research questions, like comparing patients before and after therapy. If strong invariance is violated in such a comparison, we might draw biased conclusions on the treatment effect, that is, finding a falsely inflated or diminished effect. The validity of the measurement instrument thus influences the validity of the treatment study as well as subsequent meta-analyses. For example, in pre-post therapy study designs, lack of longitudinal MI could result from the instrument having unequal sensitivity across the severity range. Assuming that therapy actually changes severity on average, the instrument would measure at different sensitivity levels in the pre-post comparison. It would thus yield a distorted image of change.

Surprisingly little is known about the MI properties of the Y-BOCS as a gold standard tool in OCD assessment. Only two studies to our knowledge explicitly investigated MI, both in the Y-BOCS-SR. Vanhille and colleagues (2018) found no violations of MI for gender, while Garnaat and Norton (2010) assessed ethnicity, for which weak MI was supported but strong invariance was not. In summary, there is a lack of evidence that MI holds true for the Y-BOCS scales with respect to multiple clinically relevant variables.

Nevertheless, such studies build the foundation for the validity of score comparisons across Y-BOCS assessments.

Aim of This Study

In this paper, we will first investigate the factor structure of both, Y-BOCS-SR and Y-BOCS-SCL, in a sample of OCD patients. Measurement models for both parts of the Y-BOCS were selected from the literature and will be tested comparatively (Deacon & Abramowitz, 2005; Fatori et al., 2020; Kim et al., 1994; McKay et al., 1995; Schulze et al., 2018). After establishing sound measurement models, we will move on to test MI. For selecting variables of interest for MI testing, we considered characteristics that are recurrent research questions in OCD research: Demographics, clinical features, and time course over therapy (Osland et al., 2018). In detail, we will thus address gender, age, migration background, and SES for demographics. Clinical characteristics will comprise age of symptom onset, years since onset, previous treatment, motivation for change, general mental health, and the presence of common comorbidities such as anxiety disorders, depression, and obsessive-compulsive personality disorder.

Methods

Sample

We analyzed data collected from 2007 until February 2020 at an academic outpatient clinic of a psychology department. The clinic is specialized in obsessive-compulsive disorder and provides comprehensive diagnostics and cognitive-behavioral therapy (CBT). During the study period, adult patients (18 + years) were typically self-referred or referred to the clinic by psychiatrists or psychotherapists. If an initial clinical interview by a trained clinical psychologist indicated obsessive-compulsive symptoms, the Structured Clinical Interview for DSM-IV (SKID I & SKID II; First et al., 1995; Spitzer et al., 1992) was conducted to review the criteria for OCD as well as possible other diagnoses. Individuals with one of the following features were generally excluded after initial diagnostics: predominant hoarding symptoms, comorbid neurological, psychotic or borderline personality disorder, substance dependence, suicidal ideation, and patients with a verbal IQ below 80. All other persons, including those with other comorbid mental disorders, were offered individual CBT. In the current analysis, we included only patients with a primary diagnosis of OCD as defined by DSM-IV or, depending on the time of inclusion in the sample, DSM-V criteria of OCD (American Psychiatric Association, 2000, 2013). This resulted in a total sample of N = 1,066 patients who were offered individual CBT.

Individual CBT was conducted in accordance with German psychotherapy regulations and paid for by health insurance companies, thus being comparable with routine care provided by private practices. Therapists are paid for up to 80 therapy sessions with a duration of 50 min, depending on the conditions of the individual case. In the outpatient clinic, therapists were strongly encouraged to follow treatment guidelines (Hohagen et al., 2014) and to make use of CBT interventions for OCD, including exposure and response prevention as well as cognitive techniques as core interventions (see Kathmann et al., 2022, for further details). As customary in routine outpatient care, therapies were terminated on the basis of a consensual decision between patient and therapist, resulting in variable total numbers of therapy sessions (Mdn = 45, IQR [31–60] for the subset of completed therapies).

While the general treatment conditions resembled naturalistic routine care, evaluation followed scientific standards and was conducted on a regular basis, that is, at initial diagnostics, start of therapy, every 20 therapy sessions thereafter, and a final assessment after the end of therapy. All time points included the application of the German Y-BOCS (interview and symptom checklist; Hand & Büttner-Westphal, 1991).

The local ethics committee at Humboldt-Universität zu Berlin approved the study (protocol number 2016-33). We did not preregister for the study. Summary statistics (e.g., frequencies and correlations) will be made available upon request from the authors.

Yale-Brown Obsessive Compulsive Scale (Y-BOCS)

The 10 items of the Y-BOCS-SR (Goodman et al., 1989) assess time spent, interference, distress, resistance, and control for obsessions and compulsions. These items were rated on item-specific five-point scales by a trained clinician, with higher values indicating higher severity. The inter-rater reliability of the Y-BOCS-SR is very high (.90; Jacobsen et al., 2003). In our sample, the mean sum score of the Y-BOCS-SR totaled 22.4 (SD = 6.2), which indicates average severity in our sample compared to the world-wide mean (24.0; Hunt, 2020).

We included 60 closed-response items of a German translation of the Y-BOCS-SCL (Ertle, 2012). These items cover a broad range of OCD symptoms, from characteristic symptoms like checking locks and stoves (75% prevalence in our sample) to less common symptoms like sexual obsessions involving children or incest (10% prevalence). In our sample, the checklist was self-rated for current symptoms (present/not present). In diagnostics, the Y-BOCS-SCL is used for identifying symptoms of interest for the subsequent severity rating with the Y-BOCS-SR. However, it can also be used to extract the prevalence of distinct symptom clusters in a sample (Bloch et al., 2008). With this in mind, we applied the 10-factor model by Schulze and colleagues (2018) with factors like Aggressive Impulses and Pure Repetitions. The factors and items are reproduced in the Electronic Supplementary Materials, ESM 1, Table E4.

Measurement Invariance Covariates

The time of assessment for all covariates was the initial diagnostic session T_diag (except for time itself). We thus used this time point’s Y-BOCS data for cross-sectional MI analysis. Detailed information on the sample characteristics in the Y-BOCS and the covariates are given in Table 1 and in the following.

Table 1 Summary of MI covariates as sample characteristics at T_diag

View as HTML

Demographics

Besides gender and age, we measured socioeconomic status (SES). Three seven-point rating scales on education, income, and occupational prestige were summed up to compute SES (Winkler & Stolzenberg, 1999). When considering ethnicity, we rated all patients as having a migration background who themselves or whose parents were not born in Germany (first and second generation immigrants). The five most prevalent countries of origin in the treatment center’s city are Turkey, Poland, Russia, Italy, and Bulgaria.

Obsessive-Compulsive Disorder Characteristics

Patients were asked at what age the first obsessive-compulsive symptom occurred. Additionally, we calculated the latency between the age of symptom onset and first contact with our institution. We collected data on previous treatments due to OCD, which could comprise psychotherapy (n = 754), pharmacological treatment (n = 695), or both. Based on an initial interview the clinician rated the patient’s motivation to change behaviors causing psychological stress on a 5-point scale from not present to very high. As responses were strongly skewed toward the upper end, we dummy-coded this variable with 1 representing high and very high.

General Mental Health

We applied the general severity index of the Brief Symptom Inventory (BSI; Derogatis & Melisaratos, 1983; Franke, 2000) as a self-report measure of a broad range of psychiatric symptoms. Additionally, the diagnostician-rated global assessment of functioning (GAF) is defined by the DSM-IV (American Psychiatric Association, 2000).

Comorbidities

Comorbid anxiety disorders comprised the following: Panic disorder with and without agoraphobia, social phobia, specific phobias, generalized anxiety disorder, and post-traumatic stress disorder (N = 276, 25.9%). For comorbid depressive disorder, we combined single major depression episodes, recurring depressive episodes, and dysthymia. N = 319 (29.9%) patients suffered from a concurrent depressive disorder at the time of initial assessment, while the total count increased to n = 608 (57.0%) when past remitted episodes were included. Additionally, we evaluated MI for two self-report questionnaires on depression: the Montgomery-Åsberg Depression Rating Scale (MADRS; Montgomery & Åsberg, 1979) and the revised Beck’s Depression Inventory (BDI-II; Beck et al., 1996; Hautzinger et al., 2006). Sum scores were calculated in both measures.

Obsessive-compulsive personality disorder (OCPD) was the most frequent personality disorder in our sample and was diagnosed with n = 58 (5.4 %) of our patients.

Time

While all N = 1,066 patients were assessed with the Y-BOCS at initial diagnostics T_diag, there was attrition at subsequent measurement points due to ongoing therapy or dropout. Only n = 423 patients provided complete data for a comparison of T_pre and T_post, at the beginning and end of therapy respectively. The time differences between T_pre and T_post varied across patients, with a median duration of 12 months and IQR [6–18 months].

Analysis

All models besides the longitudinal models were tested with data from the initial diagnostics session T_diag. The Y-BOCS items were five-point ordinal (Y-BOCS-SR) or dichotomous (Y-BOCS-SCL) which was why we applied factor analysis models for non-continuous data (B. Muthén, 1984). These analyses were mostly conducted in R (R Core Team, 2020) using the package lavaan (Rosseel, 2012) with the WLSMV estimator.

For MI models with continuous covariates, we alternatively used MPlus 8.4 (L. K. Muthén & Muthén, 1998–2017) with the MLR estimator assisted by the R package MplusAutomation (Hallquist & Wiley, 2018).

Baseline Models

Initially, we tested measurement models proposed in the literature (Y-BOCS-SR: see Figure 1; Y-BOCS-SCL: Schulze et al., 2018) without any MI covariate in lavaan. Factors were tested one by one with the exception of the Y-BOCS severity models (due to cross-loadings) and the symptom factors of Sexual Obsessions, Somatic Obsessions, and Mental Exactness (for items see ESM 1, Table E4). The latter consisted of only three items prohibiting factor model tests. We thus opted for a three-factor model in the case of the symptom model.

Figure 1 Assessed factor models for the Y-BOCS severity rating. For item description see Goodman et al. (1989). Obs = Obsessions. Comp = Compulsions. Items in grey are dropped in the model of Fatori et al. (2020).

Modeling factors in a joint model poses an even stricter test of model fit compared to unidimensional models as potential cross-loadings decrease fit. Model fit was assessed by the chi-square test complemented by fit indices as proposed by Hu and Bentler (1999). For good fit they recommend the combination of a SRMR ≤ 0.08 with either a RMSEA ≤ 0.06 (or the lower bound of its 90% confidence interval ≤ 0.06, respectively), or a CFI ≥ 0.95.

In cases where the proposed baseline models failed to provide an adequate fit to the data, we turned to modification indices in order to identify possible improvements. Modification indices capture the decrease in chi-square should a single previously constrained model parameter be released (e.g., a correlated error set to zero). We aimed to change the original models as little as possible and incorporated theoretical considerations as well as findings from competing models to justify alterations.

Testing Measurement Invariance

We estimated multigroup and longitudinal models in lavaan and evaluated model fit using the same standards described by Hu and Bentler (1999), as mentioned above. In all longitudinal models, we included pairwise correlated error terms for repeatedly measured items (Vandenberg & Lance, 2000).

For testing strong MI we compared the configurable factor model with a model where strong MI was imposed. In the present case of binary data, strong MI means that item loadings and thresholds were independent of the respective covariate.¹ We tested the change in the chi-square statistic for statistical significance. Chi-square tests are susceptible to sample size in the sense that marginal deviations from MI easily reach statistical significance with typical sample sizes (Brannick, 1995). We thus, secondly, looked at the change of fit indices. Following Chen’s 2007 recommendations, we retained the hypothesis of strong MI compared to configural MI, when a decrease in CFI < .01 was accompanied by an increase in RMSEA < .015 or an increase in SRMR < .01.

In the case of the metric covariates BDI-II, MADRS, BSI, GAF, age, age of symptom onset, years since onset, and SES we used moderated non-linear factor analysis (Bauer, 2017) in order to treat the covariates as continuous. The items of the Y-BOCS-SCL were treated as dichotomous, whereas the ordinal response format of the Y-BOCS-SR had to be treated as a metric in this model class. Currently, MPlus provides only means for model comparison but no common model fit assessments for this novel MI model type. Configural MI was thus not tested explicitly for these covariates. For strong MI we relied on information criteria and chi-square difference tests as modified by Satorra and Bentler (2010). We considered decreases in model fit if the chi-square difference test yielded p < .05, accompanied by an increased Akaike information criterion (AIC), Bayesian information criterion (BIC), and sample size adjusted BIC (Liang & Luo, 2020).

Investigating Non-MI Structure

In those cases where configural or strong MI did not hold, we investigated further to portray possible causes. To address issues with configural MI, we carefully examined modification indices (Saris et al., 2009). Failing strong MI initiated the search for item subsets where strong MI held individually. We applied the approach proposed by Pohl and Schulze (2020) and Schulze and Pohl (2020) in combination with a significance test as described by Bechger and Maris (2015) at a p level of .05. Identification of invariant item subsets is an important step in partial MI modeling, where the assumption that MI holds for all items, is relaxed. Please note that the above-mentioned approach to partial MI does not identify single items that violate MI, but instead breaks the item list into subsets for which MI holds. The researcher then has to decide which subset they use when anchoring the scale. For further details on the rationale, we would like to direct the reader to the sources cited in this paragraph.

Results

Baseline Models

For the Y-BOCS-SR, we tested several measurement models taken from the literature (see Table 2 and Figure 1). Of the previously published models, a good fit was only achieved with a recently proposed two-factor model by Fatori and colleagues (2020) which excludes the item pair measuring resistance to symptoms (see Figure 1). The factors correlated moderately; r = 0.48. In order to provide an alternative model that keeps all items, we modified the measurement model via the use of modification indices in confirmatory factor analysis. Good model fit was only reached when splitting Obsessions and Compulsions, introducing a third Resistance/Control factor as suggested by Deacon and Abramowitz (2005), and allowing cross-loadings on the latter factor (see Figure 1). As this model is related to a model proposed by Kim and colleagues (1994), we labeled it “modified Kim model”. This model fitted the data best. On the other hand, the advantage in model fit is only small compared to the model by Fatori and colleagues (2020), and from a theoretical perspective, the modified Kim model is substantially more complex.

Table 2 Fit of baseline models

View as HTML

Schulze and colleagues (2018) differentiated the Y-BOCS-SCL into ten factors. When using their proposed model, four scales needed small changes (see Table 2). First, we deleted two items concerning hoarding disorder in the Keeping Order factor, as hoarding disorder has been separated from OCD (American Psychiatric Association, 2013). Model fit increased considerably after deletion (see Table 2; fit with hoarding items: χ²(9) = 93.90, p < .001, CFI = 0.926, RMSEA = 0.094, SRMR = 0.093). Secondly, we introduced a few correlated residuals after inspection of modification indices and theoretical considerations for three scales: (1) In the Cleanliness factor the residuals of closely related items “Concerned will get ill because of contaminant” and “Concerned will get others ill by spreading contaminant” were allowed to correlate. (2) The Mental Urges factor was slightly modified by introducing covarying errors for the related items “need to know and remember” and “need to tell, ask, or confess”. (3) For the Responsibility factor, correlated residuals for “Checking locks, stoves, appliances, etc.” with “Checking that did not make mistake” and “Fear will be responsible for something else terrible happening” were allowed due to large modification indices.