Disaster Preparedness Evaluation Tool (DPET): Adaptation and psychometric evaluation for nurses in Germany

Introduction

In the last decade, the number of disasters (e.g., nature or man-made disasters, such as terrorism) in Germany has increased (Centre for Research on the Epidemiology of Disasters, 2022). Internationally, disaster nursing (i.e., nursing tasks in response to a disaster) involves a broad range of activities, such as assessing the need for nursing measures, performing triage and the immediate initiation of life-sustaining measures, and resource planning and coordination (Veenema, 2018). In Germany, the activities of nurses in disaster scenarios are focused on nursing and health care facility settings. For example, nurses are trained to make necessary intervention decisions in life-threatening situations and to initiate immediate life-sustaining measures until the arrival of the physician (Nursing Professions Training and Examination Ordinance, 2018). According to a survey of nurses in Germany, their activities during a disaster include patient care, assisting during triage, on-scene command, prioritization, communication, public relations, clearing rooms for additional patients, and recruiting and deploying personnel (Grochtdreis et al., 2020). Interestingly, despite disaster training required to become a nurse, disaster preparedness among such professionals worldwide was rated as only low to medium (Labrague et al., 2018). For example, personal preparedness, self-protection, emergency response, but also assessments of biological and chemical agents were identified as weak areas of preparedness. In addition, nurses reported difficulties with treating the victims of disasters without a medical presence (Labrague et al., 2018).

So far there are no suitable instruments to assess disaster preparedness in the nursing context in Germany. Across health care systems, the construct of disaster preparedness is defined by the United Nations Office for Disaster Risk Reduction as “the knowledge and capacities developed by governments, response and recovery organizations, communities and individuals to effectively anticipate, respond to and recover from the impacts of likely, imminent or current disasters” (United Nations Office for Disaster Risk Reduction, 2017). However, a unified theory underlying the construct is lacking (Pourvakhshoori et al., 2017). One instrument (Disaster Preparedness Evaluation Tool, DPET) to assess the level of disaster preparedness was developed specifically for nurses in the USA (Tichy et al., 2009).

The DPET is based on disaster preparedness competencies for nurses derived from both the American Association of Colleges of Nursing Essentials of Master’s Education and the National Organization of Nurse Practitioner Faculties (Tichy et al., 2009). In addition, a panel of nurse-experts in disaster nursing advised the authors in the tool development process (Tichy et al., 2009). Therefore, disaster preparedness in the DPET is defined based on a pragmatic and professional rather than theoretical perspective.

The DPET contains 47 items in English divided into three phases of disaster management (preparedness with 25 items, response with 16 items and recovery with 6 items). The three phases focus on knowledge, disaster skills and personal preparedness (preparedness), knowledge and patient management (response), and knowledge and management (recovery). The items are rated on a 6-point Likert scale with either negative ratings (from 1 = strongly disagree to 3 = disagree) or positive ratings (from 4 = agree to 6 = strongly agree), while the middle ratings (neutral, unsure or do-not-know) are not included (Tichy et al., 2009). The overall DPET-score is a sum of answers on all 47 items that can range from 47 points (low disaster preparedness) to 282 points (high disaster preparedness). The DPET is reliable in that it has a high internal consistency with Cronbach’s alpha of 0.93 for all items (Tichy et al., 2009). However, validity of the DPET was not investigated in the original publication (Tichy et al., 2009). It can only be assumed that the DPET consists of three subscales based on the three disaster preparedness phases included in the tool.

Despite the unclear validity of the DPET, it has already been translated into different languages and adapted into seven versions (Table 1). All seven versions confirmed a high internal consistency of the DPET ranging from a Cronbach’s alpha of 0.88 to 0.97 (Table 1). Four versions also showed that the DPET has acceptable content validity based on expert ratings (Table 1). Thus, the DPET items appear to be understandable and relevant for the target population of the tool (i.e., nurses). However, the DPET appears to have a different item and factor structure in different languages and countries. Specifically, out of the seven versions, six versions include less items than the original DPET (i.e., 28 to 45 items; Table 1). Furthermore, five versions investigated the factor structure of the DPET and reported that while the extracted factors explained from 64 to 70% of variance in all items, a different number of factors (i.e., 3 to 6) was extracted in different versions (Table 1). This result is not surprising due to (1) heterogeneous methods used to perform factor analysis and extract factors in the five studies (Table 1) and (2) a lack of internationally acceptable definition of the disaster preparedness construct in the nursing context. In general, the factor structure of the translated DPET versions is likely to depend on the translation quality and the adaptation complexity that needs to consider heterogeneous responsibilities of nurses in different health care systems internationally. For this reason, it is worthwhile to investigate how the DPET performs in different languages and countries. A substantial cultural adaptation involving removing, adding, or rewording of the original items may be required before the DPET can be used to measure disaster preparedness in nurses in different languages and countries. Since a German version of the DPET does not exist so far, the objective of this study was to provide the first version of the German translation and to test the psychometric performance of this version in the nursing context in Germany.

Table 1 Psychometric properties of the Disaster Preparedness Evaluation Tool (DPET) and its adaptations

Instrument version	Items and subscales	Internal consistency (Cronbach's alpha)	Validity	Factor selection method
DPET, English Tichy et al., 2009	47 items with 6-point Likert scale	0.93	–	–
	25 items: Level of preparedness	0.93
	16 items: Level of preparedness for response	0.93
	6 items: Level of preparedness for disaster recovery	0.91
DPET-A, Arabic Al Khalaileh et al., 2010	45 items with 6-point Likert scale	0.90	Face validity: 1 expert; content validity: a panel of experts; EFA with PCA (Promax rotation); total explained variance: 64%	Eigenvalues-greater-than-1 rule, scree plot
	13 items: Knowledge	0.91
	11 items: Skills	0.90
	21 items: Post disaster management	0.91
DPET-C, Chinese (Taiwan) Chen et al., 2015	38 items with 6-point Likert scale	0.97	S-CVI/Ave: 0.98; EFA with PCA (Varimax rotation); total explained variance: 65%	Eigenvalues-greater-than-1 rule
	18 items: Post-disaster management	0.89
	5 items: Skills in bioterrorism/biological attack	0.93
	8 items: Knowledge of self-preparation in disaster	0.83
	4 items: Knowledge to respond in the community	0.79
	3 items: Knowledge to respond in the workplace	0.78
DPET-J, Japanese Öztekin et al., 2016	47 items with 6-point Likert scale	0.88	–	–
	25 items: Nurses’ disaster preparedness (knowledge)	0.88
	16 items: Nurses’ response abilities	0.94
	6 items: Nurses’ evaluation stage	0.91
DPET-I, Indonesian Nugroho & Alim, 2017	45 items with 6-point Likert scale	0.94	–	–
DPET-K, Korean Han & Chun, 2021	28 items with 6-point Likert scale	0.95	S-CVI: 0.80 or higher, 7 academic experts, 7 clinical experts; EFA with PCA (Varimax rotation); total explained variance: 70%; CFA	Eigenvalues-greater-than-1 rule
	4 items: Disaster education and training	0.77
	7 items: Disaster knowledge and information	0.89
	4 items: Bioterrorism and emergency response	0.93
	5 items: Disaster response	0.89
	8 items: Disaster evaluation	0.95
DPET-T, Thai Krongthaeo et al., 2022	45 items with 6-point Likert scale	0.97	S-CVI: 0.93, 5 experts; EFA with PCA (Promax rotation); total explained variance: 67%	Eigenvalues-greater-than-1 rule, scree plot
	19 items: Post disaster management	0.97
	9 items: Disaster skills	0.95
	8 items: Knowledge of preparedness	0.86
	5 items: Knowledge specific to response	0.76
	2 items: Acquiring disaster education	0.88
	2 items: Sources of disaster knowledge	0.80
DPET-MC, (China) Wang et al., 2023	34 items with 6-point Likert scale	0.97	EFA with PCA (Varimax rotation); total explained variance: 64%; CFA	Eigenvalues-greater-than-1 rule, scree plot, parallel analysis
	3 items: Pre-disaster awareness	0.79
	8 items: Pre-disaster knowledge	0.88
	10 items: Pre-disaster management	0.94
	6 items: Knowledge and skills in the workplace	0.90
	7 items: Post-disaster skills and knowledge	0.96

Table 1 Psychometric properties of the Disaster Preparedness Evaluation Tool (DPET) and its adaptations

View as image HTML

Methods

Results

Phase 2: Psychometric evaluation based on survey data

Participant characteristics

The complete data from 317 participants were used in psychometric evaluation (Figure S1 in ESM1). All participants were fullyqualified nurses who were enrolled in academic nursing programs nationwide in Germany (Table S4 in ESM1). The participants were aged from 22 to 65 years (mean = 34, SD = 9 years) and 72.87% were female (Table 2). Over half of the participants reported working part-time (56.47%) and at hospitals (76.97%) as general healthcare nurses (77.60%). The work experience was between 0 and 40 years (mean = 11.20, SD = 8.64 years) that was obtained mostly at hospital departments (e.g., general wards or intensive care units). Most participants (61.20%) completed different types of advanced training in nursing.

Table 2 Characteristics of participants in the online survey (n = 317)

Characteristic	n	%
Gender
Male	86	27.13
Female	231	72.87
Employment status
Full-time	110	34.70
Part-time	179	56.47
Other	28	8.83
Profession
General healthcare nurse	246	77.60
Pediatric nurse	37	11.67
Geriatric nurse	34	10.73
Advanced training in nursing
Advanced specialist training	51	16.09
Bachelor degree	87	27.44
Master degree	15	4.73
Other	41	12.93
None	123	38.80
Working area
Hospital	244	76.97
Preventive or rehabilitation institution	1	.32
Outpatient care facility	9	2.84
Inpatient care facility	21	6.62
Other	42	13.25
Field of professional experience after apprenticeship in nursing
Emergency department	54	17.03
Intensive care unit	96	30.28
Surgery	33	10.41
General ward	147	46.37
Psychiatric ward	12	3.79
Other departments or units within hospitals	36	11.36
Other (not in hospitals)	73	23.03

Table 2 Characteristics of participants in the online survey (n = 317)

View as image HTML

Item analysis

Item discrimination was acceptable for 33/37 items with absolute item-total correlation coefficients from 0.42 to 0.70 and low for 4/37 items with coefficients of less than 0.30 (Table S5 in ESM1). Internal consistency for all 37 items was high with a Cronbach’s alpha of 0.94. Since the removal of low discrimination items, one at a time, did not change the overall internal consistency, all 37 items were used in the EFA.

Factor structure

According to the EFA, five factors were extracted (Table 3). All five factors explained 55% of the variance in the 37 items.

Table 3 Internal consistency and factor structure of the DPET-GER according to the exploratory factor analysis

Factors after Oblimin rotation	Number of items loading onto a factor	Absolute item loadings onto a factor	Eigen-value	explained variance	Cumulative % explained variance	Internal consistency (Cronbach’s alpha)
1: Professional disaster preparedness knowledge	8	0.40–0.76	5.01	14	14	0.87
2: Disaster management skills	12	0.30–0.73	4.45	12	26	0.90
3: Psychosocial emergency skills	6	0.46–0.78	4.37	12	38	0.85
4: Biologicalhazard skills	5	0.41–0.85	3.79	10	48	0.83
5: Personal disaster preparedness knowledge	6	0.41–0.74	2.77	7	55	0.69

Table 3 Internal consistency and factor structure of the DPET-GER according to the exploratory factor analysis

View as image HTML

Following the rotation, between 5 to 12 items loaded onto each factor. Based on item content, the meaning of the five factors was determined as: (1) professional disaster preparedness knowledge, (2) disaster management skills, (3) psychosocial emergency skills, (4) biological hazard skills, and (5) personal disaster preparedness knowledge (Table 3 and Table S6 in ESM1). Internal consistency was high for the items loading onto the factors 1 to 4 (Cronbach’s alpha from 0.83 to 0.90) and moderate for the items loading onto the factor 5 (Cronbach’s alpha of 0.69). The correlations among factors were small to moderate with correlation coefficients from 0.23 to 0.40 (Table S7 in ESM1).

Discussion

In this study the DPET-GER, a 37-item tool for assessing disaster preparedness in nurses was developed by adapting and validating the original DPET tool from the USA for the German nursing context, and psychometrically evaluated. Although DPET-GER has high internal consistency and meaningful factor structure, its low content validity indicates that further adaption is required before it could be used to assess disaster preparedness among nurses in Germany.

Similar to other versions of the DPET (Table 1), the DPET-GER has a high internal consistency for all items and item subscales. Therefore, the target population of the tool (i.e., nurses) can reliably respond to the items. In contrast, evaluation of content and factorial validity shows that some differences exist between the DPET-GER and the other versions of the DPET (Table 1). In general, the construct of disaster preparedness is not uniformly defined meaning that the validity of the DPET is likely to depend on the nursing context in a specific country. This is especially evident because the DPET-GER and further six DPET versions consist of less than the original 47 items (i.e., 28 to 45 items; Table 1). The removed items were deemed as irrelevant for the nursing context based on expert ratings in each country. Thus, each DPET version consists of a different combination of items and possibly factors underlining these items.

Overall, 10 items were removed from the DPET-GER due to their low relevance for the nursing context in Germany. For example, the DPET items 8, 9 and 12 were rated as irrelevant by the experts probably because the nursing training in Germany is less academic-based than in other parts of the world. As a result, only 1 to 2% of graduates in nursing apprenticeship programs in any given year have also completed an academic nursing degree in Germany (Lehmann et al., 2019). Furthermore, DPET items 2, 15, 16 and 33 were rated as irrelevant by the experts probably because nurses in Germany are predominantly involved in disaster management as part of their work in hospitals and inpatient or outpatient nursing facilities but not in managerial positions outside these facilities. Hospitals in Germany are required by law to prepare an alarm and emergency plan (Federal Office of Civil Protection and Disaster Assistance, 2020). Outside these facilities, responsibility for disaster control lies with the municipalities and districts or independent cities, which are considered as the so-called lower disaster control authorities and rely on their own personnel (Federal Ministry of the Interior and Community, 2022).

It is unclear to what extent the overall factor structure of the DPET-GER was affected by the removal of 10 items. While the original DPET (Tichy et al., 2009) consists of three subscales addressing disaster stages (level of preparedness for a disaster, response, and recovery), other DPET versions (Table 1) have three to six factors with partially different meanings to the original three subscales. This is not surprising considering different methods of translation, adaptation and factor derivation used in all DPET studies so far (Table 1). Unlike the three disaster stages, the DPET-GER items appear to cluster around five factors related to different skills in disaster context, including professional, management, psychosocial emergency, biological hazard, and personal skills. These factor names were derived inductively from the content of the 37 items and are partially aligned with the factor structure or factor interpretation of other versions of the DPET (Table 1). Similar nursing skills were also recently described by others (Su et al., 2022). Specifically, a response to natural or man-made disasters requires basic first aid techniques, casualty triage, observation and monitoring (aligned with our factor ’management skills’), psychological care and communication (aligned with our factor ’psychosocial emergency skills’) (Su et al., 2022). Furthermore, different disaster types require different specific skills, such as those covered by our factor ’biological hazard skills’ (Su et al., 2022). In addition, the International Council of Nurses also refers to skills in biohazard situations and a general, family, and professional preparedness plan in the context of disaster nursing (International Council of Nurses, 2019). Future studies should therefore assess if further adaptation of the DPET-GER (e.g., by adding new items, revising the existing items, or removing the irrelevant items) would change or confirm the factor structure of the tool as reported here.

Compared to the other versions of the DPET (Table 1), the DPET-GER has a low content validity (the S-CVI value of 0.53) meaning that there was a low agreement on item relevance among the experts who rated the items. Overall, 19/37 DPET-GER items were consistently rated as relevant or very relevant by most experts despite the lack of any content adaptation and verbatim translation of these items by non-professional translators. However, 18 potentially irrelevant items probably require further adaptation due to differences in the nursing context between Germany and other countries. For example, unlike in Germany, in the USA the nurses also perform disaster management tasks outside of hospitals and inpatient care facilities. These include, for example, assessing community needs, conducting infection control surveillance in collaboration with the health department and hospital, performing on-site triage of victims as needed, and establishing mass prophylaxis centres (Spencer & Spellman, 2013).

Furthermore, the S-CVI values are not computed consistently among studies, making their interpretation and comparability questionable. We included all expert ratings for all 37 items in our computation of S-CVI. However, higher S-CVI values could be obtained if the adaptation is performed in two stages, as done by others (Han & Chun, 2021). During stage one, the irrelevant items are identified based on very low or low ratings assigned to an item by the majority or all experts. During stage two, the irrelevant items from stage one are revised, re-rated by the experts and such revised ratings are used to compute the S-CVI (Han & Chun, 2021). While item revisions were not planned in this study, a computation of the S-CVI for items not requiring revisions (i.e., the 19 relevant or very relevant items) produced a nearly acceptable value of 0.74. The content validity of the DPET-GER could be improved in further adaptation of this tool performed by removing or revising the existing items and by adding new items relevant for the nursing context in Germany. The adaptation process could be supported by the Delphi-methodology to obtain qualitative feedback on item relevance from experts in disaster nursing.

A strength of this study is that it presents an example of a measurement tool, the DPET-GER, to assess disaster preparedness among nurses in Germany. Such tool is required in the nursing practice to determine the need for further training to improve disaster preparedness among groups of nurses or to focus on specific aspects of disaster preparedness (Grochtdreis et al., 2020). The results of psychometric evaluation provide valuable evidence required to further develop and adapt the DPET-GER for the nursing context in Germany. Further research is also required to decide if the total score on the DPET-GER should be computed and interpreted. Since all 37 items are measured on a 6-point Likert scale (i.e., from 1 to 6), a total sum score could indicate a minimum (37 points) to a maximum (222 points) level of disaster preparedness. Alternatively, a total mean score of all items could indicate a low level of disaster preparedness when tending towards 1 to a high level of disaster preparedness when tending towards 6. However, it is unclear if all items should have the same weight when computing a total score. Furthermore, there is no cut-off point in the tool to distinguish, for example, well-prepared from poorly prepared nurses with respect to disaster management that could be addressed in future research. Finally, building on the qualitative study by Grochtdreis et al. (2020), more theoretical research is required to define the construct of disaster preparedness required for further adaptation of the DPET-GER.

There were several limitations in this study. First, due to a lack of funding, professional translators were not involved in the translation procedure. Thus, the translation quality could be further improved by using professional translators in the future. Second, the selection and recruitment of experts was challenging due to difficulties in identifying training courses focusing on disaster management for nurses in Germany. It cannot be ruled out that the small sample size, specific expertise, or indeed inadequate expertise in disaster nursing among the recruited experts affected the relevance ratings of the DPET-GER items required for content validity evaluation. Third, the data for psychometric evaluation of the DPET-GER were obtained in an online survey of predominantly young and female nurses enrolled in academic degrees in Germany. Thus, our results may not be generalizable to other samples of nurses. The sample is also not representative for Germany in terms of the distribution of participants across the federal states. Despite their average work experience of 11 years, some study participants were probably still relatively inexperienced in the nursing profession. In addition to work experience, disaster preparedness can be addressed through education and training. Thus, it can be speculated that nurses in an academic setting are aware of disaster preparedness due to their academic studies. Further psychometric evaluation of the DPET-GER should be done using data from more heterogeneous samples of nurses, in terms of age, gender, and work experience as well as by including nurses without academic degrees, who are typical of the general nursing population in Germany.

Conclusions

A tool for assessing disaster preparedness (DPET) was adapted and validated for the nursing context in Germany using expert ratings and psychometrically evaluated using survey data. Such tool is required to measure disaster preparedness, to determine the need for additional training and thereby to facilitate the response to future disasters. The adapted DPET-GER has acceptable psychometric properties (high internal consistency and meaningful factor structure). However, low content validity indicates that further adaptations of the DPET-GER are required before it could be used to assess disaster preparedness among nurses in Germany. Such adaptations could be performed by removing or revising the DPET-GER items that were rated as irrelevant by at least one expert or by adding new items. The adaptation process could be supported by using the Delphi-methodology to obtain qualitative feedback on item relevance from the experts. More research is also needed to define the construct of disaster preparedness and the role of nurses regarding disaster preparedness in Germany.

Electronic Supplementary Materials

The electronic supplementary material is available with the online version of the article at https://doi.org/10.1024/1012-5302/a000964.

ESM1. Additional results in Tables S1–S7 and Figure S1.