Measuring Anxiety Symptoms – Psychometric Properties of the Dutch Version of the Lehrer–Woolfolk Anxiety Scale Questionnaire

Lang’s tripartite model of fear (1971) states that anxiety can be divided into a cognitive, behavioral, and somatic response system. These three dimensions are weakly correlated with each other and may change independent of each other in the course of treatment. Treatments for anxiety and the definitions of anxiety disorders are often based on all three of these modalities. Therefore, to evaluate the progress of anxiety treatments, it is important to have questionnaires that measure these three modalities of fear. This way we can measure if there is progress on all three modalities on their own. The Lehrer–Woolfolk Anxiety Symptom Questionnaire (LWASQ; Lehrer & Woolfolk, 1982), a 36-item self-report questionnaire that is developed to measure general anxiety symptoms, seems suitable for this goal, as it consists of three subscales: somatic complaints, cognitive problems, and behavioral complaints. The somatic subscale measures somatic symptoms of anxiety, the cognitive subscale refers to a tendency to worry and ruminate, and the behavioral subscale measures avoidance of social situations. The three subscales are correlated with each other. Rose and Devine (2022) mention the LWASQ briefly as a self-report questionnaire for generic anxiety, which can be used for screening purposes and outcome assessments. The LWASQ is the only questionnaire mentioned by Rose and Devine (2022) that measures all three modalities as different constructs. However, little is known about the psychometric properties, as only two studies reported on the psychometric qualities of this questionnaire (Lehrer & Woolfolk, 1982; Scholing & Emmelkamp, 1992). In the Netherlands, the Dutch version of the LWASQ is used within the biggest outpatient mental health care provider in the country, PsyQ, for measuring symptoms of anxiety as part of Routine Outcome Measurement. Because of this widespread use of the LWASQ, it is important that the psychometric properties of the Dutch version are studied.

Lehrer and Woolfolk (1982) designed the original questionnaire. They started with two lists of questions derived from different questionnaires, the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1951) and the Spielberger State-Trait Anxiety Inventory (STAI; Spielberger et al., 1970) and the authors own clinical experience. The questions covered anxiety symptoms in the three modalities of somatic complaints, cognitive problems, and behavioral complaints. Each item was rated on a 9-point Likert scale ranging from never to almost always. They tested the first list in their first study with 60 items on 451 undergraduate psychology students. The second study used a list with 112 items that was tested on a mixed population (289 adult night school students, 70 neurotic patients with anxiety problems, and 67 stress workshop participants) using principal component analysis. In this analysis they found three expected factors: a somatic complaints factor, a behavioral complaints factor, and a cognitive complaints factor. They created the LWASQ with 36 items that loaded more than .5 on the rotated factors of Study 2. Examples of items are “I feel dizzy,” “I try to avoid social gatherings,” and “I picture some future misfortune.” The three factors correlated more with each other than expected (range between .47 and .66), which means they are not completely independent of each other. Split-half reliability was between r = .83 and r = .85 for all three factors in Study 1 and between r = .91 and r = .93 for all three factors in Study 2. Other psychometric properties were measured in following studies: Scholing and Emmelkamp (1992) translated the LWASQ and changed the 9-point Likert scale to a 5-point Likert scale to make it easier for patients to score the items. Using a confirmatory factor analysis (CFA), they confirmed the factor structure for the Groninger Angst Schaal (GAS), the Dutch translation of the LWASQ. Correlations between the three factors ranged from .38 to .62 in the different groups of social phobic patients, normal population, and adolescents. They further found that internal consistency ranged from α = .83 to α = .92 for the three subscales in all three groups.

As Scholing and Emmelkamp (1992) state, the results of the construct validity that Lehrer and Woolfolk (1982) found are contradictory. They found results that support that there is no differentiation between the three subscales. For example, the Trait Anxiety Scale of the STAI (Spielberger et al., 1970) and the IPAT Anxiety Inventory (Krug et al., 1976) correlate significantly with all three subscales of the LWASQ, while the somatic subscale of the Symptom Checklist-90 (SCL-90; Arrindell & Ettema, 2005) and the Eysenck (Eysenck & Eysenck, 1963) introversion subscale showed correlations that corroborate a difference between the three subscales. Scholing and Emmelkamp (1992) also studied convergent and divergent validity with other self-report measures such as the Dutch versions of the SCL-90 (Arrindell & Ettema, 2005), Fear Questionnaire (FQ; Arrindell & Emmelkamp, 1984), and the Social Cognitions Inventory (SCI; Van Kamp & Klip, 1981). The subscale somatic complaints of the SCL-90 correlated strongly (.75 and .84) with the somatic subscale of the LWASQ. Correlations with the other subscales of the LWASQ were at least .27 lower. The behavioral subscale was compared with FQ social phobia and the SCI. As expected, the correlation between the FQ social phobia scale and the behavioral subscale was high (.67). Contrary to their hypothesis, the behavioral subscale also correlated substantially with the SCI (.52). However, the cognitive subscale had a relatively low correlation with the SCI (.42). This could be explained by the fact that the SCI primarily measures cognitive complaints linked to social phobia, while the LWASQ cognition subscale also refers to other cognitive complaints as seen in a generalized anxiety disorder or obsessive-compulsive disorder. This can be seen by the strong correlations of the cognitive subscale with the depression and the interpersonal sensitivity subscales of the SCL-90 (.72 and .74). In conclusion, the somatic complaints factor possessed good convergent and divergent validity. The behavioral and cognitive subscales are more ambiguous.

In sum, the psychometric properties are inconsistent. The factor structure of the LWASQ and the Dutch translation seem sound but are not tested on a population with diverse anxiety disorders. Lehrer and Woolfolk (1982) only used a small sample of anxious patients, and it is not clear if the anxious patients who participated were diagnosed with an anxiety disorder, whereas Scholing and Emmelkamp (1992) studied a normal population (n = 103), an adolescent population (n = 650), and a social phobic sample (n = 108). They found that the LWASQ was able to discriminate between social phobic patients and a normal population but suggested further research in a mixed anxiety sample to assess if it is possible to discriminate between different anxiety disorders using this questionnaire. However, the construct validity seems poor. The results found by Lehrer and Woolfolk (1982) contradict each other, and the measures used do not all measure the same construct that the LWASQ measures. Scholing and Emmelkamp (1992) also found ambiguous results. The somatic complaints factor seems to have good divergent and convergent validity. The behavioral complaints and cognitive problems factors show good convergent validity but low divergent validity. This could be explained by the fact that Scholing and Emmelkamp (1992) only used patients with a social phobia while the questionnaires focuses on all kinds of aspects of anxiety that are not always part of social phobia. Furthermore, both studies did not measure test–retest reliability.

The goal of the present study was to evaluate the psychometric properties of the Dutch version of the LWASQ in a mixed anxiety sample. With this in mind, we conducted three sub studies. In the first study, we examined the factor structure using a confirmatory factor analysis. Next to that we examined the internal consistency with McDonald’s ω. Both measures were performed in a clinical sample of patients with an anxiety disorder. We expect to find the same three factors as in both previous studies and expect them to be moderately correlated. The second study measured the test–retest reliability in a normal population. We expect the test–retest reliability to be good (r > .80). The third study measures the convergent and divergent validity in a clinical sample of patients with an anxiety disorder. However, given the small sample size, we could not draw any conclusions for this study and decided not to mention the outcomes in this paper.

Study 1: Factor Structure and Internal Consistency in a Clinical Sample of Patients With an Anxiety Disorder

Subjects

The ROM measurements of 2,165 patients were made available to the researchers by the data processing department of PsyQ. For unknown reasons, 48 patients had filled in two pretreatment measurements, of which the first administered one was included. Of the 2,117 patients who were included, 63.4% were women. Two third of patients had an anxiety disorder (53.3%) or an OCD or OCD-related disorder (13.7%) as main diagnosis. In the remaining one third of the patients, there was a comorbid anxiety or OCD disorder present when the LWASQ was administered. Of this subgroup of patients with a comorbid anxiety or OCD disorders, the main DSM-5 diagnoses were mood disorders (9.9%), personality disorders (6.2%), PTSD or PTSD-related disorders (4.1%), ADHD (1.6%), eating disorders (0.4%), or other disorders (3.4%). Group characteristics are presented in Table 1.

Table 1 Group characteristics of a clinical sample of patients with an anxiety disorder in Study 1

Variable	M (SD)/%
Note. LWASQ = Lehrer–Woolfolk Anxiety Scale Questionnaire, SD = standard deviation.
LWASQ total	102.62 (26.53)
Subscale somatic complaints	40.91 (14.22)
Subscale cognitive problems	36.34 (8.96)
Subscale behavioral complaints	25.38 (8.92)
Gender (% women)	63.40%
DSM-5 main diagnosis
Anxiety disorder (%)	53.30%
Obsessive compulsive disorder (%)	13.70%
Posttraumatic stress disorder (%)	4.10%
ADHD	1.60%
Depressive disorder (%)	9.70%
Bipolar disorder (%)	0.20%
Personality disorder (%)	6.20%
Eating disorder (%)	0.40%
Other (%)	3.40%
Missing (%)	7.40%

Table 1 Group characteristics of a clinical sample of patients with an anxiety disorder in Study 1

View as image HTML

Analysis and Results

We conducted a CFA using maximum likelihood in JASP Team (2023); see Figure 1 for our hypothesized model. We used the following benchmarks: The CFI should be larger than .9 (Hu & Bentler, 1999), the RMSEA point estimate and the upper bound of the 95 percent confidence interval should be smaller than 0.05 (Browne & Cudeck, 1992; Jöreskog & Sörbom, 1993), and the SRMR should be smaller than .08 (Hu & Bentler, 1999). As Heene et al. (2011) and Greiff and Heene (2017) point out, the model fit as described by Hu and Bentler (1999) within social studies might not work, so we also looked at global misfit of the model. Next to that we used the indications of Little (2013), t < .85 is poor, t between .85 and .90 is mediocre, t between .90 and .95 is acceptable, t between .95 and 99 is very good, and t > .99 is outstanding. Factors such as somatic complaints, cognitive problems, and behavioral complaints were based on the studies of Lehrer and Woolfolk (1982) and Scholing and Emmelkamp (1992). We expect a moderate correlation between the three factors. We expect Items 1, 2, 4, 7, 10, 13, 14, 18, 20, 23, 29, 30, 31, 33, 34, and 35 to load on the factor somatic complaints. We expect Items 3, 6, 9, 12, 17, 22, 25, 26, and 28 to load on factor behavioral complaints. Finally, we expect Items 5, 8, 11, 15, 16, 19, 21, 24, 27, 32, and 36 to load on the factor cognitive problems.

First, we checked the assumption of normal distribution. All Q–Q plots showed a normal distribution on all the items. The factor model was significant (χ² = 6,286.95; p < .001). The fit of the factor model was good with respect to RMSEA (.067) and SRMR (.060), but the CFI was too low (.84). We checked the modification indices as suggested by Heene et al. (2011) and Greiff and Heene (2017). These indicated some residual correlations between different items. We checked these correlations on content. When the correlations were logical on clinical level, we added these to the model. For example, Item 8 “I cannot get some thought out of my mind” and Item 21 “I cannot get some pictures or images out of my head” showed residual correlations. If we examine these items on content, we see considerable overlap, or Item 1 “My throat gets dry” and Item 2 “I have difficulty in swallowing.” We can imagine if someone has a dry throat swallowing will also be difficult. Based on content, we added these correlations. In the end, we added nine residual correlations, and these were Items 8–21, 1–2, 24–27, 23–33, 29–33, 18–20, 4–35, 26–28, and 11–27. After adding these correlations, the CFI was 0.9, the RMSEA (.054) and SRMR (.052) still showed an acceptable fit, and the model remained significant (χ² = 4,199.725; p < .001). The three factors correlated moderately with each other and are presented in Table 2. As shown in Table 3, the nonstandardized factor loadings of all items were high on the expected factors, although the modification indices showed that Item 36 (“I have an uneasy feeling”) showed a fit on both the factor cognitive problems and the factor behavioral complaints. We chose to include the item in the cognitive factor for the following studies, following the previous articles. These results show that our hypothesis is probable, but the model with the new residual correlations must be replicated in another sample.

Table 2 Correlation coefficients between the three factors of the Dutch version of the Lehrer–Woolfolk Anxiety Scale Questionnaire

Subscale	Somatic complaints	Cognitive problems (95% CI)	Behavioral complaints (95%CI)
Note. CI = confidence interval.
*p < .001.
Somatic complaints		.57* (.54–.61)	.48* (.45–.52)
Cognitive problems			.69* (.67–.72)
Behavioral complaints

Table 2 Correlation coefficients between the three factors of the Dutch version of the Lehrer–Woolfolk Anxiety Scale Questionnaire

View as image HTML

Table 3 Nonstandardized factor loadings of the Dutch version of the Lehrer–Woolfolk Anxiety Scale Questionnaire

Factor	Item	Factor loadings	95% CI
			LL	UL
Note. LL = lower limit. UL = upper limit. *p < .001.
Somatic complaints	1. My throat gets dry	0.729*	.680	.779
	2. I have difficulty in swallowing	0.662*	.615	.709
	4. My heart pounds	0.787*	.737	.836
	7. My limbs tremble	0.817*	.767	.867
	10. My stomach hurts	0.768*	.714	.822
	13. My neck feels tight	0.784*	.730	.838
	14. I feel dizzy	0.841*	.792	.891
	18. I breathe rapidly	0.790*	.740	.839
	20. I cannot catch my breath	0.756*	.709	.802
	23. My arms or legs feel stiff	0.986*	.934	1.039
	29. I experience muscular aches and pains	1.014*	.963	1.066
	30. I experience a tingling sensation somewhere in my body	1.086*	1.034	1.137
	31. My arms or legs feel weak	0.996*	.947	1.045
	33. My muscles twitch or jump	0.913*	.860	.966
	34. I feel numbness in my face, limbs or tongue	0.774*	.729	.820
	35. I experience chest pains	0.869*	.819	.918
Behavioral complaints	3. I try to avoid starting conversations	0.948*	.900	.996
	6. I avoid talking to people in authority (my boss, policemen)	0.825*	.770	.880
	9. I avoid going into a room by myself where people are already gathered and talking	1.105*	1.053	1.156
	12. I avoid new or unfamiliar situations	1.042*	.993	1.091
	17. I pass by school friends or people I know but have not seen for a long time, unless they speak to me first	0.936*	.881	.990
	22. I try to avoid social gatherings	1.168*	1.118	1.218
	25. I find myself staying home rather than involving myself in activities outside	0.958*	.906	1.009
	26. I prefer to avoid making specific plans for self-improvement	0.641*	.589	.693
	28. I try to avoid challenging jobs	0.810*	.759	.860
Cognitive problems	5. I picture some future misfortune	0.757*	.700	.813
	8. I cannot get some thought out of my mind	0.665*	.614	.717
	11. I dwell on mistakes that I make	0.711*	.661	.760
	15. I think about possible misfortunes to my loved ones	0.686*	.626	.746
	16. I cannot concentrate at a task or job without irrelevant thoughts intruding	0.809*	.758	.859
	19. I keep busy to avoid uncomfortable thoughts	0.592*	.544	.639
	21. I cannot get some pictures or images out of my head	0.804*	.747	.860
	24. I imagine myself appearing foolish with a person whose opinion is important	0.766*	.707	.825
	27. I am concerned that others might not think well of me	0.670*	.615	.726
	32. I have to be careful not to let my real feelings show	0.818*	.764	.872
	36. I have an uneasy feeling	0.779*	.734	.824

Table 3 Nonstandardized factor loadings of the Dutch version of the Lehrer–Woolfolk Anxiety Scale Questionnaire

View as image HTML

To examine the internal consistency, McDonald’s ω was calculated for the different factors using JASP Team (2023). All three subscales had a good reliability: ω = .93 for the somatic complaints subscale, ω = .90 for the behavioral complaints subscale, and ω = .85 for the cognitive problems subscale.

Study 2: Test–Retest Reliability in a Normal Population

Subjects

In total, 49 people completed both measurements. Seven people did not complete the second measurement. Table 4 shows the descriptive and demographic information. M_age of the population was 37.1 (SD = 16.9). Two third (67.3%) of the participants were women. Almost everyone (90.1%) had a minimum education level of high school, and 81.7% were fully employed, 40% were married, 16.7% were living together, 31.7% were single, and 6.7% were divorced at the time of the administration.

Table 4 Group characteristics of the normal population sample in Study 2

Variable	M (SD)/%
Note. SD = standard deviation.
Age	37.1 (16.9)
Gender (% women)	67.3%
Education
Primary school	3.3%
High school	26.7%
Bachelor’s degree	21.7%
University	41.7%
Missing	6.7%
Employment status
Fully employed	81.7%
Fully unemployed (due to sickness)	6.7%
Partly employed	6.7%
Missing	5.0%
Marital status
Married	40.0%
Living together	16.7%
Divorced	6.7%
Single	31.7%
Missing	5.0%

Table 4 Group characteristics of the normal population sample in Study 2

View as image HTML

Analysis and Results

Pearson’s correlation coefficient was estimated to assess the test–retest reliability by examining the correlations between the total scores and the scores on the subscales of the LWASQ. Table 5 presents the M, SDs, McDonald’s ω, and the correlations of all scales. The internal consistency reliability (McDonald’s ω) is good for all scales on both measurements. Pearson’s correlation coefficient for the total score was .88. The subscales of behavioral complaints, somatic complaints, and cognitive problems were .82, .90, and .75, respectively. Despite these correlations, we see a difference in mean scores between both measurements. However, the changes per subscale remain within the norm groups that Scholing and Emmelkamp (1992) found and are not clinical significant changes. We explain these changes as average fluctuations within a person’s normal anxiety level. All scores suggest good test–retest reliability.

Table 5 Mean scores and Pearson’s correlation coefficient of test–retest reliability of the Lehrer–Woolfolk Anxiety Scale Questionnaire

Subscale	First measurement		Second measurement		Pearson’s correlation
	M (SD)	McDonald’s ω	M (SD)	McDonald’s ω
Note. SD = standard deviation.
*p < .001.
Total score	65.15 (19.27)	.95	58.08 (17.42)	.95	.88*
Subscale somatic complaints	27.02 (9.13)	.89	24.63 (7.89)	.89	.90*
Subscale cognitive problems	22.02 (6.48)	.85	19.00 (6.48)	.89	.75*
Subscale behavioral complaints	15.84 (5.64)	.87	14.45 (5.32)	.89	.82*

Table 5 Mean scores and Pearson’s correlation coefficient of test–retest reliability of the Lehrer–Woolfolk Anxiety Scale Questionnaire

View as image HTML

Discussion

In this paper, two studies into the psychometric properties of the LWASQ within both a mixed anxiety population and a healthy sample were described.

First, a CFA was conducted. The results confirmed that the three-factor solution (somatic complaints, behavioral complaints, and cognitive problems), as found by Lehrer and Woolfolk (1982) and Scholing and Emmelkamp (1992), was an acceptable fit. However, one item (Item 36 “I have an uneasy feeling”) showed a fit on more than one factor. Whereas it loaded on all three factors in the Scholing and Emmelkamp (1992) study, this item showed a fit on both cognition and behavior in our findings, indicating that it might represent a general underlying anxiety construct instead of one specific factor. As such, it might be considered to exclude this item. Due to a few modifications in the CFA model, it is important that the factor structure is studied within a new sample to confirm these findings. Furthermore, in the first study, internal reliability was determined with MacDonald’s ω, and in the second study, test–retest reliability was measured. Both measures of all three subscales were found to be good. The correlation in Study 2 was based on a sample of N = 49. While this is enough for a point estimate that is relatively stable, it is not enough to have sufficient power to detect small to medium effects. Therefore, these results need to be replicated in a future study.