Introducing the scr-OLBI

Methodological Considerations

Demerouti and Bakker (2008) administered the OLBI to Dutch employees and found significant factor cross-loadings on four items. The researchers suggested that the content of these items created disturbances to the nomonological network of burnout. Deficiencies regarding the construct validity of the measure are also reported by Campos et al. (2012) due to the factor cross-loadings of certain items. The proposed 2-factor model showed reasonable fit, but a modified version with the deletion of certain items exhibited a much better fit to their data. Reis et al. (2015) examined the factorial structure and measurement invariance of the OLBI across German and Greek students. The researchers used a modified version of the OLBI (focusing on academic burnout) to assess the construct’s equivalence. The results confirmed the proposed 2-factor model for the Greek version. However, discrepancies were found for the German disengagement subscale due to context-specific issues that need further investigation. Mahadi et al. (2018) examined the factor structure of the OLBI in the Malay population and found unacceptable fit for both the proposed 2-factor and 1-factor models. The researchers pointed out items with poor factor loadings on their proposed construct and performed a stepwise removal approach with the deletion of seven items. Their modified version of the OLBI included nine items and showed acceptably better fit to the data compared to the original measure. Another shorter version was examined on an independent sample (Mészáros et al., 2020); their modified version comprised five items on each factor (both positively and negatively keyed). Factor analysis of this modified version of the OLBI gave prominence to certain discrepancies, underscoring the need for the use of more elaborated techniques.

Discrepancies in the factor structure of the OLBI may be due to the measure’s theoretical background. As suggested by Demerouti and Bakker, the OLBI was originally constructed to measure “burnout and its hypothetical opposite state of work engagement” (2008, p. 75). According to the authors, the subscales of exhaustion and disengagement consist of items that refer to their opposite dimensions, namely vigor and dedication at work; their score is assessed by summing responses from the positive-keyed items without recoding them. Research on work engagement confirmed the nomonological distinction from burnout and new measures on work engagement are now available (e.g., Schaufeli et al., 2001).

Standardized questionnaires with multiple items represent the state of the art in psychometrics, as they are valid and reliable measures for assessing the construct under consideration (e.g., Kruyen et al., 2012). However, in surveys and research, long tests can be time-consuming, can increase respondent fatigue, and can frustrate individuals with attention deficits, resulting in potentially unreliable or missing data (Greer & Liu, 2016). In contrast, short instruments can be considered more specific and offer certain advantages for research purposes: They are quick and inexpensive, provide direct and valid estimates of a trait, and are appropriate for participants with limited attention capabilities. While a short instrument is advantageous in terms of cost, time, and applicability, there are also challenges in its construction. Therefore, examination of the item properties of the OLBI is considered imperative.

Research Objectives

This study adds to the existing literature and has two objectives. The first objective is to provide a shorter tool for measuring occupational burnout that uses a more elaborated technique, namely item response theory (IRT). IRT provides item/test information curves and category thresholds that are fundamental in scrutinizing the measure’s items. To our knowledge, no research applied IRT to examine the full spectrum of the OLBI. Gustavsson et al. (2010) used IRT to analyze only the positively keyed items (eight items), with data from Swedish nurses. This study builds on the findings of Gustavsson and colleagues and proposes a shorter yet more concise tool for assessing subclinical burnout. This shorter measure is named scr-OLBI, which is composed of the first letters of the word “screening” and is intended to be a valuable tool for research purposes. Our second objective is to demonstrate the validity and internal reliability of the instrument by examining its relationship to gender, education, and leadership practices with a focus on ethical behaviors and attitudes.

IRT Analysis

IRT is widely applied to evaluate item parameters of unidimensional constructs (Greer & Liu, 2016). There are several different types of IRT models for assessing factor structure and item properties, all of which represent the nonlinear relation between individuals’ trait level and their probability of choosing a certain response category for each item (Embretson & Reise, 2000). We used two different models in our analysis: the Rasch rating scale model (RRSM; Andrich, 1978), which is also a polytomous Rasch model, and the graded response model (GRM; Samejima, 1997). The RRSM is a latent structure model for polytomous responses to a set of test items. This model is considered an extension of the Rasch model (for dichotomous responses). According to the RRSM, each item of the measure is described only by a j threshold difficulty parameter (β_ij), where j is the number of response options-1 per i items. The β_ij specifies the response category difficulty parameter and is interpreted as the trait value (θ) indicated by the probability of responding in or above a given j threshold. The GRM is an extension of the 2-parameter logistic model; each item of the measure is described by both a j threshold difficulty parameter (β_ij) and a discrimination parameter (a) as well, where j is the number of response options-1 per i items. The a signifies the relationship between observed item response and unobserved latent trait and characterizes the slope of the item’s characteristic curve. The discrimination parameter reports the degree to which item response categories distinguish among trait levels; it is constant for all thresholds within an item yet varies across items within the same measure. The GRM was chosen as opposed to alternative polytomous models (e.g., partial credit model) because this model is considered a more natural model for rating scales and provides the best fit to many measures examined (e.g., Khorramdel & von Davier, 2016). Further details about these models can be found in Embretson and Reise (2000). Building on previous studies, the preceding procedures will be used to check the item properties of the OLBI and assist in the construction of a new shorter, yet concise, instrument.

Burnout and Ethical Leadership

Burnout is associated with various occupational behaviors and outcomes. Individuals exhibiting burnout symptoms present deterioration of their job performance, self-esteem, and social interaction and exhibit higher levels of stress and defective conceptions of self and others, e.g., their leaders (e.g., Demerouti et al., 2003). Leadership is an important determinant of employees’ work-related health outcomes, involving work well-being and job satisfaction. Consequently, research on leadership practices and employees’ well-being is considered critical.

The literature has shown that leaders who are perceived as honorable, honest, and true to their occupation and express a genuine concern for their employees are likely to influence employee’s burnout (e.g., Mitropoulou et al., 2019). Ethical leaders are characterized by their ethical practices and by the promotion of their practices to employees through role modeling and observational learning. They show genuine concern about multiple stakeholders, community welfare, and sustainability issues. They are morally obliged to ethical standards, care about the consequences of their actions, and promote work strategies that are fully in line with their ethical values. Ethical leaders act as role models and conduct their occupational and personal life with integrity, honesty, sincerity, trustworthiness, and humility.

Ethical leadership, measured by a general ethical leadership factor, is found to have a mediating effect on employees’ burnout (e.g., Demerouti et al., 2003; Mitropoulou et al., 2019). When employees discern deception, unfairness, and dishonesty, they are more likely to exhibit feelings of exhaustion and depersonalization from work. Conversely, leaders who sustain equal opportunities for advancement, support employees, communicate about ethics, and clarify responsibilities, expectations, and performance goals are more likely to mitigate burnout symptoms (e.g., Kalshoven et al., 2011). Nonetheless, the concept of ethical leadership has proven to be more complicated than originally expected. Ethical leadership is considered a multifaceted construct including both personality traits and behavioral patterns. Therefore, to further examine the association of burnout and ethical leadership, we employ a more elaborated, multidimensional model of ethical leadership, which includes seven dimensions (Kalshoven et al., 2011). We predict that burnout will exhibit a negative association with all ethical leadership dimensions.

Finally, to demonstrate divergent validity, we examined certain demographic variables considered unrelated to burnout, such as gender, education, and organizational tenure (e.g., Demerouti & Bakker, 2008). In general, employees tend to feel disengaged and exhausted from their work, regardless of their age, education, and tenure. Therefore, we expect to find nonsignificant correlations among burnout, demographic variables, and organizational tenure.

Participants and Procedure

In total, 617 Greek employees have participated in this study with a mean age of 47.16 years (SD = 9.01, age range 25–72 years), of whom 66% held a university degree and 49.1% are female. Participants are occupied into three different organizations (manufacture, public services, and sales). From our total sample, 314 employees were also asked to fulfill an ethical leadership instrument. Data were collected between July 2013 and March 2015 as part of a larger research project and are publicly available (Mitropoulou, 2022a). Snowball and cluster sampling procedures were used for data collection, including organizations from various industries (e.g., sale, manufacture, and public services). In regard to the cluster sampling process, medium/large organizations located in Greece were reached via e-mail by the first researcher through the website https://www.greatplacetowork.gr (n = 40). Three organizations accepted participation (response rate 7.5%). The first one is a public service organization, while the other two are private organizations, dealing with manufacturing and marketing; all three organizations employed more than 100 individuals and consisted of several branches. Organizations received the link that included the study’s measures and were asked to distribute it to their employees. Employees’ participation was voluntary; participants were informed about the research purpose prior to the completion and received an electronic or paper-and-pencil copy of the questionnaire. Due to the combinatorial use of the cluster sampling procedure, the response rate could not be determined. Ethical approval was obtained by the ethics committee of the University of Crete.

Measures

Burnout was measured with the Greek version of the OLBI (Demerouti et al., 2010), which consists of 16 items: Eight items assess exhaustion and the other eight assess disengagement from work. Each item is rated by the degree of agreement regarding occupational burnout, with responses ranging from 1 (strongly agree) to 4 (strongly disagree). Internal consistency reliability is α = .78 for the exhaustion subscale and α = .68 for the disengagement subscale.

Table 1 Descriptives, reliability, and validity indices for the scr-OLBI

Variables	M	SD	ω	scr-OLBI
Note. N = 314. ω = McDonald omega reliability index; *p < .001
ELW – people orientation	22.5	6.72	.94	−.43*
ELW – fairness	21.7	5.3	.86	−.38*
ELW – power sharing	19.1	4.12	.75	−.27*
ELW – sustainability	10.2	2.84	.81	−.35*
ELW – ethical guidance	24.0	5.83	.92	−.36*
ELW – role clarification	17.5	4.21	.90	−.28*
ELW – integrity	14.1	4.42	.97	−.45*
scr-OLBI	1.9	0.66	.74	—
Tenure	10	8.37		−.03
Education				−.01
Age				−.07
Gender				.09

Table 1 Descriptives, reliability, and validity indices for the scr-OLBI

View as image HTML

Ethical leadership was measured with the self-report 38-item Ethical Leadership at Work questionnaire (ELW; Kalshoven et al., 2011). Participants rated their agreement regarding their direct leader’s ethical behaviors and attitudes with responses rating from 1 (strongly disagree) to 5 (strongly agree). The ELW consists of seven dimensions, namely people orientation, fairness, power sharing, concern for sustainability, ethical guidance, role clarification, and integrity (see Table 1).

Table 2 Fit indices for the item properties assessment models in Exhaustion and Disengagement subscales

Subscales	−2Log likelihood	df	AIC	BIC
Note. N = 617; Model 1 = 1PL Rasch rating scale model (RRSM), Model 2 = 2PL graded response model (GRM), AIC = Akaike information criterion, BIC = Bayesian information criterion.
Exhaustion
Model 1	−5,716.60	12	11,457.00	11,510.00
Model 2	−5,542.28	32	11,148.56	11,290.16
Disengagement
Model 1	−5,589.97	12	11,204.00	11,257.00
Model 2	−5,423.50	32	10,911.01	11,052.61

Table 2 Fit indices for the item properties assessment models in Exhaustion and Disengagement subscales

View as image HTML

Organizational tenure was measured by asking participants to state how many years they retain their occupation. Research included information about gender, age, and education.

Data Analysis

Data were analyzed with the R (R Core Team, 2021) and the Stata 16 software (StataCorp, 2019). Before starting our analysis, dimensionality was assessed. We compared the fit indices of different models of IRT to assess dimensionality and item properties for the OLBI. In regard to dimensionality, 2-factor solutions were compared: a unidimensional model and a 2-factor polytomous model using the R package mirt (Chalmers, 2012). The mirt package permits the estimation of multidimensional item response theory parameters for confirmatory models by using the maximum likelihood method and begins with the computations of a matrix of quasi-polychoric correlations. For the assessment of the item properties, we estimated the polytomous RRSM using the TAM package (Robitzsch et al., 2020) and the GRM with the mirt package. Differential information function was conducted with DIFAS 5.0 (Penfield, 2005). The R syntax for all analyses is publicly available at https://osf.io/3tskm.

Descriptive Statistics and Item Analysis

Means, SDs, and item correlations are publicly available at https://osf.io/9n6gk. All items exhibit small-to-moderate interitem correlations apart from Item 15, which reveals insignificant correlations with most items. Dimensionality was also assessed. A unidimensional model and a 2-dimensional model were tested, and fit indices were examined. We used five goodness-to-fit indices across models to determine which provides the best fit: The Bayesian information criterion (BIC) and the Akaike information criterion (AIC), with value differences greater than 10 indicating model fit difference and lower values representing a better fit of the model (Kang et al., 2009), the −2-log likelihood (−2LL), where χ² distribution with degrees of freedom equals to the difference in the number of parameters between the most complex model and the model with fewer parameters, the root mean square error of approximation (RMSEA) and the standardized root mean residual (SRMR) fit indices, with values < .05 indicating acceptable fit (Maydeu-Olivares, 2013). The analysis revealed a significantly better fit for the 2-factor model [AIC = 21,497.43, BIC = 21,847.06, −2LL = 10,669.75, df = 15] in comparison to a unidimensional burnout model [AIC = 21,930.37, BIC = 22,213.56, −2LL = 10,901.18, df = 18]. Yet, examination of the absolute fit indices revealed unacceptable fit to the data for the 2-factor model [RMSEA = .07, 95% CI (.06, .08), SRMR = .09] and the unidimensional model [RMSEA = .12, 95% CI (.13, .09), SRMR = .09], with both RMSEA indices being over the recommended threshold (Maydeu-Olivares, 2013). Also, discrepancies of the items’ factor loadings were evident. Specifically, Item 14 loaded to its noncorresponding factor and Items 9, 11, and 15 yielded significant cross-loadings.

Next, we evaluate the measurement quality of the OLBI. One major advantage of IRT is that the assumptions made can be tested by means of goodness-of-fit measures (see Khorramdel & von Davier, 2016). We used two different models in our analysis: the RRSM (Andrich, 1978) and the GRM (Samejima, 1997). The RRSM examines whether the item discrimination parameter α is equal across all items, while the GRM pertains that item discrimination parameter α is not equal across all items and differences between each of the response categories are not the same across all items. Both models were tested separately for each of the two subscales of the OLBI. The analysis revealed a significantly better fit for the less restrictive GRM compared to the restrictive RRSM; items are best defined by both difficulty and discrimination parameters (see Table 2).

Table 3 shows the distribution of responses and the item parameters for the 16 OLBI items. Discrimination parameter α represents the item’s slope; all items have moderate to high α, with Items 12, 8, and 4 (exhaustion) and Items 7, 3, and 11 (disengagement) exhibiting the highest discrimination parameters. The difficulty parameter β represents the part of the latent continuum θ where the thresholds of item categories are located. Difficulty parameters were found to vary across items. Items’ modal values are located in the first three response options, whereas the highest category is rarely chosen, except for Items 2 and 4, which appear to have evenly distributed pattern of responses. In general, the positive parameter thresholds for most items indicate that the measure discriminates in the higher regions of the latent continuum, with the two subscales having similar patterns of difficulty, apart from Item 13, due to its lower (negative values) difficulty parameters thresholds.

Table 3 Parameters and response thresholds for all items

OLBI item	α	β1	β2	β3	1	2	3	4
Note. N = 617. Items in italics are reverse coded; α = discrimination parameter; βj = difficulty parameter; 1 = totally disagree, 2 = disagree, 3 = agree, and 4 = totally agree. Selected items for the scr-OLBI are presented in bold.
Exhaustion
2. Tired before arrive at work	1.34	−1.41	−0.19	1.13	18.6	25.8	31.4	24.2
4. Need more time to relax and feel better	1.74	−1.06	−0.02	1.24	22.5	27.1	31.8	18.6
5. Tolerate pressure of work very well	0.85	−1.73	1.19	3.09	21.4	48.8	20.9	8.9
8. Emotionally drained	2.09	−0.64	0.52	1.67	30.5	34.9	24.9	9.7
10. Enough energy for leisure activities	1.57	−1.07	0.49	1.95	23.3	39.9	27.2	9.6
12. Worn out and weary	2.90	−0.84	0.38	1.51	23.0	39.6	27.9	9.5
14. Manage amount of work well	0.68	0.10	4.25	6.05	51.7	42.2	4.2	1.9
16. Usually feel energized	0.92	−.59	1.91	3.95	38.7	43.3	14.3	3.7
Disengagement
1. New and interesting aspects in work	1.18	−0.29	1.82	3.46	43.3	41.3	12.3	3.1
3. Talk about work in a negative way	1.43	0.28	1.66	3.01	56.7	28.5	11.5	3.3
6. Do job almost mechanically	1.29	−0.27	1.18	2.59	42.8	33.1	17.8	6.3
7. Work is a positive challenge	2.04	0.08	1.29	2.25	52.1	30.9	12.3	4.7
9. Disconnected from work	1.39	−0.79	0.71	2.31	30.1	36.8	25.6	7.5
11. Sickened by work tasks	1.40	0.36	1.51	2.84	58.8	24.2	12.9	4.1
13. Only type of work that can imagine	0.71	−2.65	−1.05	0.72	14.9	18.3	27.9	3.8
15. Engaged in work	0.27	−4.23	2.77	8.69	24.0	43.4	23.8	8.8

Table 3 Parameters and response thresholds for all items

View as image HTML

Development of scr-OLBI

In accordance with our first objective, indicators that discriminate highly among burnout respondents are preferred for inclusion in this short measure, since such indicators will accurately differentiate among individuals. The analysis revealed that Item 12 (α = 2.90) and Item 8 (α = 2.09) from the exhaustion subscale and Item 7 (α = 2.04) and Item 3 (α = 1.43) from the disengagement subscale fulfill these criteria. However, further examination of their wording reveals that only Item 7 is positively keyed while the other items are negatively keyed items. Consequently, to avoid potential method factor bias due to item wording (i.e., Demerouti & Bakker, 2008), we decided to exclude Item 7 from the final selection and include Item 11 (α = 1.40) instead for two reasons. Item 11 exhibits the third highest discrimination parameter in the subscale, and second, it is negatively keyed, hence theoretically more relevant to burnout.

To further assess the suitability of these four items (12, 8, 3, and 11), we decided to examine the information functions at both item and test levels. Figure 1 shows the item information function (IIF) for all subscale items. The selected items exhibit the highest IIF’s in each subscale. Examination of their item characteristic curves also reveals that their category response curves are mostly evenly endorsed. Accordingly, Figure 2 shows the test information function (TIF) for the original 8-item and the modified 2-item subscales of OLBI. Comparison of TIFs for the full spectrum of burnout and the new shorter version of the construct reveals similar patterns of information and standard error. Interestingly, the SE values are very low for both subscales across most trait values.

We also examined the factor structure, reliability, and differential item functioning (DIF) of the four items. Fit indices revealed a better fit to the data [AIC = 5,259.93, BIC = 5,330.73, RMSEA = .07, 95% CI (.02, .12), SRMR = .04], with RMSEA remaining over the recommended threshold of <0.05 (Maydeu-Olivares, 2013). Internal consistency reliability is α = .74. DIF exists when respondents with the same trait show differing response probabilities in a given item category due to any form of group connection that is irrelevant to the construct measured and may lead to systematic measurement error (Kleinman & Teresi, 2016). Since our objective is to assess occupational burnout among male and female employees, we ensure that such measurement bias does not exist. Accordingly, the selected items were analyzed with gender as the grouping differential factor. DIF is assessed by the weighted and unweighted estimates of the DIF effect variance, the standard error estimators of these variances, and the ratio of each DIF effect variance estimate over its respective standard error estimator (Penfield, 2005). Weighted, unweighted, and standard error estimates exhibit nonsignificant differences for each pair of items, revealing nonsignificant variations among male and female employees.

Preliminary Validity Analysis

In accordance with our second objective, we examined the associations between the scr-OLBI, ethical leadership dimensions, and demographic variables (e.g., gender, age, education, and tenure). The correlation results are provided in Table 2. Validation analysis of the scr-OLBI confirmed all hypothesized correlations. In terms of convergent validity, scr-OLBI was found to correlate negatively with all ethical leadership behaviors. Finally, in terms of divergent validity, the scr-OLBI did not correlate significantly with employees’ age, gender, education, and organizational tenure.