Indicators of outcome quality in peripheral arterial disease revascularisations – a Delphi expert consensus

Introduction

The numbers of invasive treatment of peripheral arterial disease (PAD), especially regarding endovascular revascularisations [1, 2], are continuously rising and it is necessary to monitor quality of treatment to improve patient care and safety. Due to the lack of evidence, especially regarding the treatment of intermittent claudication (IC), the choice of treatment should take patients’ preferences more into consideration and be supported by shared decision-making [3–5]. Evidence-based quality indicators of structure, process, and outcomes would help to align patients’ preferences to enable pragmatic quality improvement in PAD treatment. To date, no broad consensus of indicators evaluating outcome quality of treatment in patients with PAD exists. Indicators available in the literature mainly focus on identifying risk factors for morbidity, mortality, or failure of treatment [6].

In the present study, we used a Delphi approach. A panel of multidisciplinary and inter-professional experts participating within the multimethodological and multistage IDOMENEO study (www.idomeneo.de) achieved consensual agreement and identified clinically relevant and practicable indicators for outcome quality of invasive PAD treatment. The IDOMENEO study aims to collect data on 10,000 consecutive patients from 40 vascular centres in Germany undergoing invasive endovascular or open-surgical treatment for symptomatic PAD with a follow-up of 12 months [7, 8]. Hence, the aim of the study was to create a set of core indicators for quality measurements that can be consistently utilized for outcome research and quality improvement in PAD treatment. The indicators of outcome quality consented in this Delphi study will in the next stage be field-tested on the 10,000 patients enrolled in the IDOMENEO study.

Methods

The Delphi approach is widely accepted and used to gain consensus among a panel of experts [9] and has previously been used in various specialties including vascular surgery [10–14]. An extensive literature review was conducted to identify indicators of quality upon studies on PAD. Medline was searched for meta-analyses, systematic reviews, and guidelines, using keywords referring to PAD and quality measurement. In addition, a grey literature search was performed. Websites concerning PAD or indicators of outcome quality were searched by hand for guidelines, statements and quality indicators. The search was restricted to German and English. No restriction to publication date was made.

Furthermore, data variable definitions of peripheral arterial revascularisation registries participating in the International Consortium of Vascular Registries (ICVR) were reviewed and possible indicators of outcome quality were extracted [15].

Leading vascular specialists from different medical specialties including vascular surgery and interventional internal medicine, participating the IDOMENEO study (ClinicalTrials.gov: NCT03098290; German Clinical Trials Registry: DRKS00014649) were invited to take part in this Delphi study. The data privacy compliant GermanVasc registry is used to collect and to validate study data. Additionally, vascular specialists in the outpatient setting, general practitioners, vascular assistants, wound nurses, and the patients’ representatives of the largest patient-society in Germany were invited to participate. The final number of quality indicators was not specified a priori. All participants in this study agreed to the scope of items identified through the abovementioned process. The panel experts were then invited to participate in web-based anonymised electronic questionnaires. Open source software (LimeSurvey GmbH, Hamburg, Germany) was used to generate the online 43-item questionnaires. In the first round, each participant was asked to indicate whether the item has a high clinical relevance and a high practicability in daily clinical practice. Each item was scored for both parameters on a five-point Likert scale, comprising “Strongly agree”, “Agree”, “Neutral”, “Disagree”, and “Strongly disagree”. Items received a consensus recommendation for the minimal dataset if at least 80 % of the participants voted with “Strongly agree” or “Agree” either for relevance or practicability. Items with less than 80 % of agreement were eliminated from consideration. The 80 % criteria were applied in correspondence to valid practical guidelines and specialist society consensus recommendations [16, 17].

Additionally, space for a free text comment for each item was given in the questionnaire for individual arguments, hints or questions. The participants could only submit one set of answers in each Delphi round.

Following the first round, a structured report about the voting results using bar charts for clinical relevance and practicability was given for each item. The charts included graphical highlighting of items having reached the 80 % consensus criteria and anonymised group comments separated in pro and contra arguments (in green/red) (Figure 1).

In the second Delphi round, the participants voted only for quality indicators in order for outcome to achieve a higher response rate. Subsequently, the voting results are presented with bar charts again.

Statistical analyses were performed with IBM SPSS Statistics software version 25.0 (IBM, Armonk, NY, U. S.)

Results

Forty experts from different medical specialties (vascular surgery, interventional internal medicine) and professions (physicians, vascular nurses with at least six years of clinical practice, physician assistants) as well as the patients’ representative of Germany were contacted and 30 (17 vascular surgeons, eight angiologists, one radiologist, one patients’ representative, three vascular nurses) accepted and completed the first online survey and 24 of them the second one between November 2017 and March 2018. In total, 43 indicators of outcome quality were identified in the literature or submitted by the experts and were included in the panel discussion (Table I and II). All quality indicators were reviewed and validated by the expert panel. After two Delphi rounds, a total of twelve (27.9 %) quality indicators reached the consensual limit of agreement for clinical relevance and four (9.3 %) quality indicators reached the consensual limit of agreement for practicability (Figure 2). Three composite quality indicators capturing major adverse limb events (MALE), major amputation, and major re-intervention (or re-operation), respectively, were considered to be of both high relevance and high practicability (Table I).

Table I Results of the Delphi study after two rounds. Twelve out of 43 quality indicators for outcome measures were consented.

	Relevance after two rounds	Practicability after two rounds
Major adverse cardiovascular events (MACE/MACCE) Including: myocardial infarction, coronary revascularisation, cardiac hospitalization, stroke or TIA, all-cause death or cardiac death	92 %	58 %
Major adverse limb events (MALE) Including: major amputation, major re-intervention or re-operation	96 %	96 %
Myocardial infarction	88 %	58 %
Stroke or transient ischaemic attack (TIA)	83 %	50 %
All-cause death	96 %	63 %
Major amputation (above ankle level)	96 %	92 %
Major re-intervention (bypass, bypass revision, thrombectomy, thrombolysis)	88 %	83 %
Re-intervention or re-operation	96 %	63 %
Wound infection	88 %	75 %
Vascular access-related major complication	92 %	75 %
Increase of maximum walking distance	88 %	54 %
Increase of Rutherford-classification	92 %	71 %
Increase of ankle-brachial-index	79 %	92 %

Table I Results of the Delphi study after two rounds. Twelve out of 43 quality indicators for outcome measures were consented.

View as image HTML

Two out of the twelve consented items can be differentiated into specific quality indicators. Major adverse cardiovascular or cardiac events (MACE, MACCE) are inconsistently used to describe major health events like myocardial infarction, coronary artery revascularisations, re-hospitalization for cardiac events, stroke or transient ischaemic attack, all-cause or cardiac-related death. Corresponding to that, MALE is commonly used to describe major health events including major amputation above ankle level, major re-interventions or re-operations to retain blood perfusion of the index leg.

In addition to the two abovementioned composite parameters and their components, the rate of wound infections, access-related complications requiring treatment, increase of maximum walking distance, improvement in Rutherford-classification, and improvement in ankle-brachial-index (ABI) found consensual agreement for clinical relevance during this Delphi process.

A total of 31 indicators failed to reach the consensual limit of agreement (six patient-reported outcomes) (Table II).

Table II A total of 31 quality indicators were not consented after two rounds of the Delphi study. Patient-reported outcomes in orange colour. Items sorted by relevance.

	Relevance after two rounds	Practicability after two rounds
Minor amputations	79 %	75 %
Primary patency	79 %	63 %
Secondary patency	79 %	67 %
Quality of life	79 %	17 %
Healing of wound infections or wound healing disorders	79 %	33 %
Cardiac death	75 %	25 %
Lymphatic fistula	75 %	54 %
Periprocedural major complications (cardiac, respiratory, renal, neurologic, venous, pulmonary embolic events, allergic complications, nerve lesions)	75 %	38 %
Ambulation	75 %	29 %
Re-stenosis	71 %	50 %
Cardiac events	67 %	46 %
Primary-assisted patency	67 %	58 %
Technical success	63 %	42 %
Patient satisfaction	63 %	25 %
Contrast-volume applied (Endovascular)	58 %	67 %
Treatment of wound infection	54 %	54 %
Dose-area product	54 %	67 %
Unplanned re-transfer to operating room	50 %	42 %
Total length of hospital stay	50 %	67 %
Postoperative length of hospital stay	50 %	71 %
Experience of pain	46 %	21 %
Functional status	46 %	25 %
Coronary revascularisation/interventions	42 %	29 %
Need for pain medication	42 %	8 %
Hospital-acquired infections	42 %	42 %
Catheter-related infections	42 %	21 %
Cardiac-related rehospitalisation	38 %	13 %
Toe-Brachial-Index	29 %	13 %
Pressure/decubitus ulcer	25 %	29 %
Oxygen pressure (tcpO2)	25 %	17 %

Table II A total of 31 quality indicators were not consented after two rounds of the Delphi study. Patient-reported outcomes in orange colour. Items sorted by relevance.

View as image HTML

Discussion

In this Delphi study with experts, we achieved a consensus on a set of indicators of outcome quality to be used in the treatment of patients with PAD. Twelve quality indicators were recommended, including two composite parameters for major cardiovascular and limb events. The Delphi approach is a widely used and accepted method to find a consensual agreement between a panel of experts on variable questions using at least two rounds of questionnaires with at least six to eleven experts [9].

In the light of rising numbers of procedures performed in patients with PAD, the question remains how to measure outcome quality of vascular care to improve patient safety and to evaluate if aims of treatment have been reached. Quality improvement programs have already been implemented in various fields of multidisciplinary vascular medicine worldwide, using specific indicators as a measurable aspect of care. Against this backdrop, quality indicators can cover aspects of process, structure, and outcome. However, in the field of PAD, the 2017 European Society of Cardiology (ESC) guideline in collaboration with the European Society for Vascular Surgery (ESVS) does not contain recommendations regarding quality indicators [5]. Ploeg et al. systematically reviewed the existing literature to provide an insight into quality improvement initiatives in vascular surgery [6]. Besides several structural and process measures, they identified 31 reports on outcome measures as indicators of quality of care published between 1991 and 2007. These reports mainly focused on identifying risk factors for morbidity, mortality or failure of treatment. Conte et al. suggested a set of objective performance goals (OPG) for evaluating catheter-based treatments in critical limb ischaemia (CLI), including MALE and MACE as important outcome measure [18].

To date, there is a paucity of evidence on valid indicators for measuring outcome quality of care in PAD revascularisations. Mortality and morbidity are certainly the most widely used indicators. However, a growing application of endovascular techniques with decreasing mortality and consequently lower variation between hospitals may prompt the emergence of further valid indicators. An increasing number of procedures is performed in patients suffering from intermittent claudication (IC) [19]. To address differences between both groups, one could claim to stratify this Delphi study into IC and CLI. While limb salvage and wound healing plays a major role in treatment of CLI, patient-reported outcomes and maximum walking distance are usually named as aims for invasive treatments in claudicants. However, even if markedly rare, amputations or deaths following elective revascularisations for IC remain fatal events and therefore deserve to be suggested as quality indicators for both groups. All indicators of outcome quality consented in this study can be measured for both IC and CLI but certainly have specific prevalence. Interestingly, two out of the twelve consented indicators (MACE, MALE) are composite indicators that can be further differentiated into specific quality indicators. In particular, the practicability of indicators for outcome quality seems to make the difference.

Interestingly, no patient-reported outcomes reached the limit of consensual agreement in this Delphi study. Although indicators such as quality of life and ambulation reached 79 and 75 % clinical relevance, respectively, their practicability was scored notably low by the panel experts (17 % for quality of life, 29 % for ambulation). To be used in daily clinical practice, a brief quality of life instrument such as the VascuQoL-6 by Nordanstig et al. [20] could be helpful but needs to be translated and validated accordingly.

Limitations

One important limitation of this study is the paucity of commonly accepted definitions for the indicators included into the Delphi process. Some of the expert comments suggested that there are various definitions available in daily clinical practice, making it challenging to compare experts voting. However, all comments and relevant questions have been reported back to the experts before finishing the second Delphi round and we took the most common definition as a basis for panel discussion. Another controversy concerned the duration of follow-up and the exact time-point of measuring the indicators of outcome quality. The fact that especially treatments for IC usually do not exceed two days of hospital stay implies that the in-hospital duration is insufficient. A follow-up duration of 30 days is commonly accepted as minimum but one year of follow-up might bring valid results even for patient-reported outcomes. Furthermore, it was difficult to find a consensual agreement for PAD revascularisations in general, although the scope of quality indicators obviously differs between treatment of IC vs. CLI.

Funding

The IDOMENEO-study is partially funded by the German Joint Federal Committee (01VSF16008).

Conclusions

The consented indicators of the present study may help to harmonise future studies and quality benchmarks increasing their comparability, validity, and efficiency. The indicators will in the next stage be field-tested on the 10,000 patients enrolled in the IDOMENEO study.

IDOMENEO collaborators

Erwin Blessing (Karlsbad), Wolfgang Paul Tigges (Hamburg), Thomas Nolte (Bad Bevensen), Claus Nolte-Ernsting (Mülheim an der Ruhr), Johannes Hoffmann (Essen), Kai Michael Balzer (Bonn), Simon Classen (Bad Nauheim), Kristian Nitschmann (Soest), Peter Heider (München), Friedhelm Beyersdorf (Freiburg im Breisgau), Thomas Zeller (Bad Krozingen), Markus Kleemann (Lübeck), Hendrik Bergert (Erfurt), Sebastian M. Schellong (Dresden), Thomas Bürger (Kassel), Alexander Selch (Neumünster), Patrick Berg (Kevelaer), Manfred Pfeiffer (Sörgenloch/Mainz), Hartmut Görtz (Lingen), Myriam Seifert (Hamburg), Jörg Teßarek (Lingen), Sven Seifert (Chemnitz), Martin Münchow (Hamburg), Bianca Neitzel (Hamburg), Daniela Karbe (Hamburg), Boris Leithäuser (Hamburg), Friederike Baron (Hamburg), Arndt Hribaschek (Magdeburg), Claus Röttger (Rheine).

There are no conflicts of interest existing.