Indicators of outcome quality in peripheral arterial disease revascularisations – a Delphi expert consensus
Abstract.Introduction: Peripheral arterial disease (PAD) affects a continuously increasing number of people worldwide leading to more invasive treatments. Indication to perform invasive revascularisations usually arises from consensus-based recommendations of practice guidelines and from few randomized controlled trials where outcome measures focus mainly on risk factors associated with mortality and morbidity. To date, no broad consensual agreement of experts on valid indicators of outcome quality exists for PAD. Methods: A literature review was conducted to collect indicators of outcome quality from studies of PAD. The Delphi technique was used to achieve a consensual agreement on a set of core indicators. The expert panel of the two-round Delphi approach was formed by leading vascular specialists joining the IDOMENEO study, physician assistants, wound nurses, and patient representatives. Items were scored via a web-based anonymised electronic questionnaire using a five-point Likert-scale. Results: Out of 40 invited experts 30 joined the panel and completed round one. Twenty-four experts completed the second and final round. Forty-three indicators of outcome quality were initially identified and validated by the panel. After two Delphi rounds, 12 indicators (27.9 %) achieved the limit of agreement for relevance and four (9.3 %) for practicability. Major adverse limb events (MALE), major amputation, and major re-intervention (or re-operation) were consented as both highly relevant and practicable. Additionally, major adverse cardiovascular events (MACE), myocardial infarction, stroke or transient ischaemic attack, all-cause death, all re-intervention (or re-operation), wound infection, vascular access-related major complication, walking distance, and Rutherford-classification were consented as highly relevant. Ankle-brachial-index was consented as highly practicable. Conclusions: This Delphi approach of vascular experts identified three indicators as highly relevant and clinically practicable to be recommended as indicators of outcome quality in invasive PAD treatment. Among others, these consented items may help in harmonising future studies and quality benchmarking increasing their comparability, validity, and efficiency.
The numbers of invasive treatment of peripheral arterial disease (PAD), especially regarding endovascular revascularisations [1, 2], are continuously rising and it is necessary to monitor quality of treatment to improve patient care and safety. Due to the lack of evidence, especially regarding the treatment of intermittent claudication (IC), the choice of treatment should take patients’ preferences more into consideration and be supported by shared decision-making [3–5]. Evidence-based quality indicators of structure, process, and outcomes would help to align patients’ preferences to enable pragmatic quality improvement in PAD treatment. To date, no broad consensus of indicators evaluating outcome quality of treatment in patients with PAD exists. Indicators available in the literature mainly focus on identifying risk factors for morbidity, mortality, or failure of treatment .
In the present study, we used a Delphi approach. A panel of multidisciplinary and inter-professional experts participating within the multimethodological and multistage IDOMENEO study (www.idomeneo.de) achieved consensual agreement and identified clinically relevant and practicable indicators for outcome quality of invasive PAD treatment. The IDOMENEO study aims to collect data on 10,000 consecutive patients from 40 vascular centres in Germany undergoing invasive endovascular or open-surgical treatment for symptomatic PAD with a follow-up of 12 months [7, 8]. Hence, the aim of the study was to create a set of core indicators for quality measurements that can be consistently utilized for outcome research and quality improvement in PAD treatment. The indicators of outcome quality consented in this Delphi study will in the next stage be field-tested on the 10,000 patients enrolled in the IDOMENEO study.
The Delphi approach is widely accepted and used to gain consensus among a panel of experts  and has previously been used in various specialties including vascular surgery [10–14]. An extensive literature review was conducted to identify indicators of quality upon studies on PAD. Medline was searched for meta-analyses, systematic reviews, and guidelines, using keywords referring to PAD and quality measurement. In addition, a grey literature search was performed. Websites concerning PAD or indicators of outcome quality were searched by hand for guidelines, statements and quality indicators. The search was restricted to German and English. No restriction to publication date was made.
Furthermore, data variable definitions of peripheral arterial revascularisation registries participating in the International Consortium of Vascular Registries (ICVR) were reviewed and possible indicators of outcome quality were extracted .
Leading vascular specialists from different medical specialties including vascular surgery and interventional internal medicine, participating the IDOMENEO study (ClinicalTrials.gov: NCT03098290; German Clinical Trials Registry: DRKS00014649) were invited to take part in this Delphi study. The data privacy compliant GermanVasc registry is used to collect and to validate study data. Additionally, vascular specialists in the outpatient setting, general practitioners, vascular assistants, wound nurses, and the patients’ representatives of the largest patient-society in Germany were invited to participate. The final number of quality indicators was not specified a priori. All participants in this study agreed to the scope of items identified through the abovementioned process. The panel experts were then invited to participate in web-based anonymised electronic questionnaires. Open source software (LimeSurvey GmbH, Hamburg, Germany) was used to generate the online 43-item questionnaires. In the first round, each participant was asked to indicate whether the item has a high clinical relevance and a high practicability in daily clinical practice. Each item was scored for both parameters on a five-point Likert scale, comprising “Strongly agree”, “Agree”, “Neutral”, “Disagree”, and “Strongly disagree”. Items received a consensus recommendation for the minimal dataset if at least 80 % of the participants voted with “Strongly agree” or “Agree” either for relevance or practicability. Items with less than 80 % of agreement were eliminated from consideration. The 80 % criteria were applied in correspondence to valid practical guidelines and specialist society consensus recommendations [16, 17].
Additionally, space for a free text comment for each item was given in the questionnaire for individual arguments, hints or questions. The participants could only submit one set of answers in each Delphi round.
Following the first round, a structured report about the voting results using bar charts for clinical relevance and practicability was given for each item. The charts included graphical highlighting of items having reached the 80 % consensus criteria and anonymised group comments separated in pro and contra arguments (in green/red) (Figure 1).
In the second Delphi round, the participants voted only for quality indicators in order for outcome to achieve a higher response rate. Subsequently, the voting results are presented with bar charts again.
Statistical analyses were performed with IBM SPSS Statistics software version 25.0 (IBM, Armonk, NY, U. S.)
Forty experts from different medical specialties (vascular surgery, interventional internal medicine) and professions (physicians, vascular nurses with at least six years of clinical practice, physician assistants) as well as the patients’ representative of Germany were contacted and 30 (17 vascular surgeons, eight angiologists, one radiologist, one patients’ representative, three vascular nurses) accepted and completed the first online survey and 24 of them the second one between November 2017 and March 2018. In total, 43 indicators of outcome quality were identified in the literature or submitted by the experts and were included in the panel discussion (Table I and II). All quality indicators were reviewed and validated by the expert panel. After two Delphi rounds, a total of twelve (27.9 %) quality indicators reached the consensual limit of agreement for clinical relevance and four (9.3 %) quality indicators reached the consensual limit of agreement for practicability (Figure 2). Three composite quality indicators capturing major adverse limb events (MALE), major amputation, and major re-intervention (or re-operation), respectively, were considered to be of both high relevance and high practicability (Table I).
|Relevance after two rounds||Practicability after two rounds|
|Major adverse cardiovascular events (MACE/MACCE)|
Including: myocardial infarction, coronary revascularisation, cardiac hospitalization, stroke or TIA, all-cause death or cardiac death
|92 %||58 %|
|Major adverse limb events (MALE)|
Including: major amputation, major re-intervention or re-operation
|96 %||96 %|
|Myocardial infarction||88 %||58 %|
|Stroke or transient ischaemic attack (TIA)||83 %||50 %|
|All-cause death||96 %||63 %|
|Major amputation (above ankle level)||96 %||92 %|
|Major re-intervention (bypass, bypass revision, thrombectomy, thrombolysis)||88 %||83 %|
|Re-intervention or re-operation||96 %||63 %|
|Wound infection||88 %||75 %|
|Vascular access-related major complication||92 %||75 %|
|Increase of maximum walking distance||88 %||54 %|
|Increase of Rutherford-classification||92 %||71 %|
|Increase of ankle-brachial-index||79 %||92 %|
Two out of the twelve consented items can be differentiated into specific quality indicators. Major adverse cardiovascular or cardiac events (MACE, MACCE) are inconsistently used to describe major health events like myocardial infarction, coronary artery revascularisations, re-hospitalization for cardiac events, stroke or transient ischaemic attack, all-cause or cardiac-related death. Corresponding to that, MALE is commonly used to describe major health events including major amputation above ankle level, major re-interventions or re-operations to retain blood perfusion of the index leg.
In addition to the two abovementioned composite parameters and their components, the rate of wound infections, access-related complications requiring treatment, increase of maximum walking distance, improvement in Rutherford-classification, and improvement in ankle-brachial-index (ABI) found consensual agreement for clinical relevance during this Delphi process.
A total of 31 indicators failed to reach the consensual limit of agreement (six patient-reported outcomes) (Table II).
|Relevance after two rounds||Practicability after two rounds|
|Minor amputations||79 %||75 %|
|Primary patency||79 %||63 %|
|Secondary patency||79 %||67 %|
|Quality of life||79 %||17 %|
|Healing of wound infections or wound healing disorders||79 %||33 %|
|Cardiac death||75 %||25 %|
|Lymphatic fistula||75 %||54 %|
|Periprocedural major complications (cardiac, respiratory, renal, neurologic, venous, pulmonary embolic events, allergic complications, nerve lesions)||75 %||38 %|
|Ambulation||75 %||29 %|
|Re-stenosis||71 %||50 %|
|Cardiac events||67 %||46 %|
|Primary-assisted patency||67 %||58 %|
|Technical success||63 %||42 %|
|Patient satisfaction||63 %||25 %|
|Contrast-volume applied (Endovascular)||58 %||67 %|
|Treatment of wound infection||54 %||54 %|
|Dose-area product||54 %||67 %|
|Unplanned re-transfer to operating room||50 %||42 %|
|Total length of hospital stay||50 %||67 %|
|Postoperative length of hospital stay||50 %||71 %|
|Experience of pain||46 %||21 %|
|Functional status||46 %||25 %|
|Coronary revascularisation/interventions||42 %||29 %|
|Need for pain medication||42 %||8 %|
|Hospital-acquired infections||42 %||42 %|
|Catheter-related infections||42 %||21 %|
|Cardiac-related rehospitalisation||38 %||13 %|
|Toe-Brachial-Index||29 %||13 %|
|Pressure/decubitus ulcer||25 %||29 %|
|Oxygen pressure (tcpO2)||25 %||17 %|
In this Delphi study with experts, we achieved a consensus on a set of indicators of outcome quality to be used in the treatment of patients with PAD. Twelve quality indicators were recommended, including two composite parameters for major cardiovascular and limb events. The Delphi approach is a widely used and accepted method to find a consensual agreement between a panel of experts on variable questions using at least two rounds of questionnaires with at least six to eleven experts .
In the light of rising numbers of procedures performed in patients with PAD, the question remains how to measure outcome quality of vascular care to improve patient safety and to evaluate if aims of treatment have been reached. Quality improvement programs have already been implemented in various fields of multidisciplinary vascular medicine worldwide, using specific indicators as a measurable aspect of care. Against this backdrop, quality indicators can cover aspects of process, structure, and outcome. However, in the field of PAD, the 2017 European Society of Cardiology (ESC) guideline in collaboration with the European Society for Vascular Surgery (ESVS) does not contain recommendations regarding quality indicators . Ploeg et al. systematically reviewed the existing literature to provide an insight into quality improvement initiatives in vascular surgery . Besides several structural and process measures, they identified 31 reports on outcome measures as indicators of quality of care published between 1991 and 2007. These reports mainly focused on identifying risk factors for morbidity, mortality or failure of treatment. Conte et al. suggested a set of objective performance goals (OPG) for evaluating catheter-based treatments in critical limb ischaemia (CLI), including MALE and MACE as important outcome measure .
To date, there is a paucity of evidence on valid indicators for measuring outcome quality of care in PAD revascularisations. Mortality and morbidity are certainly the most widely used indicators. However, a growing application of endovascular techniques with decreasing mortality and consequently lower variation between hospitals may prompt the emergence of further valid indicators. An increasing number of procedures is performed in patients suffering from intermittent claudication (IC) . To address differences between both groups, one could claim to stratify this Delphi study into IC and CLI. While limb salvage and wound healing plays a major role in treatment of CLI, patient-reported outcomes and maximum walking distance are usually named as aims for invasive treatments in claudicants. However, even if markedly rare, amputations or deaths following elective revascularisations for IC remain fatal events and therefore deserve to be suggested as quality indicators for both groups. All indicators of outcome quality consented in this study can be measured for both IC and CLI but certainly have specific prevalence. Interestingly, two out of the twelve consented indicators (MACE, MALE) are composite indicators that can be further differentiated into specific quality indicators. In particular, the practicability of indicators for outcome quality seems to make the difference.
Interestingly, no patient-reported outcomes reached the limit of consensual agreement in this Delphi study. Although indicators such as quality of life and ambulation reached 79 and 75 % clinical relevance, respectively, their practicability was scored notably low by the panel experts (17 % for quality of life, 29 % for ambulation). To be used in daily clinical practice, a brief quality of life instrument such as the VascuQoL-6 by Nordanstig et al.  could be helpful but needs to be translated and validated accordingly.
One important limitation of this study is the paucity of commonly accepted definitions for the indicators included into the Delphi process. Some of the expert comments suggested that there are various definitions available in daily clinical practice, making it challenging to compare experts voting. However, all comments and relevant questions have been reported back to the experts before finishing the second Delphi round and we took the most common definition as a basis for panel discussion. Another controversy concerned the duration of follow-up and the exact time-point of measuring the indicators of outcome quality. The fact that especially treatments for IC usually do not exceed two days of hospital stay implies that the in-hospital duration is insufficient. A follow-up duration of 30 days is commonly accepted as minimum but one year of follow-up might bring valid results even for patient-reported outcomes. Furthermore, it was difficult to find a consensual agreement for PAD revascularisations in general, although the scope of quality indicators obviously differs between treatment of IC vs. CLI.
The IDOMENEO-study is partially funded by the German Joint Federal Committee (01VSF16008).
The consented indicators of the present study may help to harmonise future studies and quality benchmarks increasing their comparability, validity, and efficiency. The indicators will in the next stage be field-tested on the 10,000 patients enrolled in the IDOMENEO study.
Erwin Blessing (Karlsbad), Wolfgang Paul Tigges (Hamburg), Thomas Nolte (Bad Bevensen), Claus Nolte-Ernsting (Mülheim an der Ruhr), Johannes Hoffmann (Essen), Kai Michael Balzer (Bonn), Simon Classen (Bad Nauheim), Kristian Nitschmann (Soest), Peter Heider (München), Friedhelm Beyersdorf (Freiburg im Breisgau), Thomas Zeller (Bad Krozingen), Markus Kleemann (Lübeck), Hendrik Bergert (Erfurt), Sebastian M. Schellong (Dresden), Thomas Bürger (Kassel), Alexander Selch (Neumünster), Patrick Berg (Kevelaer), Manfred Pfeiffer (Sörgenloch/Mainz), Hartmut Görtz (Lingen), Myriam Seifert (Hamburg), Jörg Teßarek (Lingen), Sven Seifert (Chemnitz), Martin Münchow (Hamburg), Bianca Neitzel (Hamburg), Daniela Karbe (Hamburg), Boris Leithäuser (Hamburg), Friederike Baron (Hamburg), Arndt Hribaschek (Magdeburg), Claus Röttger (Rheine).
There are no conflicts of interest existing.
Comparison of global estimates of prevalence and risk factors for peripheral artery disease in 2000 and 2010: a systematic review and analysis. Lancet. 2013;382:1329–40.
Recent trends in morbidity and in-hospital outcomes of in-patients with peripheral arterial disease: a nationwide population-based analysis. Eur Heart J. 2013;34:2706–14.
AHA/ACC Guideline on the Management of Patients With Lower Extremity Peripheral Artery Disease: A Report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. JACC. 2017;69:e71–e126.
Society for Vascular Surgery practice guidelines for atherosclerotic occlusive disease of the lower extremities: management of asymptomatic disease and claudication. J Vasc Surg. 2015;61:2S–41S.
ESC Guidelines on the Diagnosis and Treatment of Peripheral Arterial Diseases, in collaboration with the European Society for Vascular Surgery (ESVS): Document covering atherosclerotic disease of extracranial carotid and vertebral, mesenteric, renal, upper and lower extremity arteries Endorsed by: the European Stroke Organization (ESO)The Task Force for the Diagnosis and Treatment of Peripheral Arterial Diseases of the European Society of Cardiology (ESC) and of the European Society for Vascular Surgery (ESVS). Eur Heart J. 2018;39:763–816.
Assessing the quality of surgical care in vascular surgery; moving from outcome towards structural and process measures. Eur J Vasc Endovasc Surg. 2010;40:696–707..
IDOMENEO – Ist die Versorgungsrealität in der Gefäßmedizin Leitlinien- und Versorgungsgerecht? Gefässchirurgie. 2017;22:41–7.
Guideline recommendations and quality indicators for invasive treatment of peripheral arterial disease in Germany: The IDOMENEO study for quality improvement and research in vascular medicine]. Bundesgesundheitsblatt. 2018;61:218–23.
Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research. Acad Med. 2016;91:663–8..
Vitiligo Global Issues Consensus G. Developing core outcome set for vitiligo clinical trials: international e-Delphi consensus. Pigment Cell Res. 2015;28:363–9.
Core Outcomes in Aphasia Treatment Research: An e-Delphi Consensus Study of International Aphasia Researchers. American Journal of Speech Language Pathology. 2016;25:S729–S42..
Antiepileptic Drug Management in Pregnancy Collaborative N. Development of a core outcome set for epilepsy in pregnancy (E-CORE): a national multi-stakeholder modified Delphi consensus study. BJOG. 2017;124:661–7.
Diagnosis and Management of Iliac Artery Endofibrosis: Results of a Delphi Consensus Study. Eur J Vasc Endovasc Surg. 2016;52:90–8.
Development of a nationally-representative coordinated registry network for prostate ablation technologies. J Urol. 2018;in press. https://doi.org/10.1016/j.juro.2017.12.058.
International Consortium of Vascular Registries Consensus Recommendations for Peripheral Revascularisation Registry Data Collection. Eur J Vasc Endovasc Surg. 2018;Epub ahead of print. https://doi.org/10.1016/j.ejvs.2018.04.006.
Editor’s Choice - ESC Guidelines on the Diagnosis and Treatment of Peripheral Arterial Diseases, in collaboration with the European Society for Vascular Surgery (ESVS). Eur J Vasc Endovasc Surg. 2018;55:305–68.
S3-Leitlinie zur Diagnostik, Therapie und Nachsorge der peripheren arteriellen Verschlusskrankheit. VASA. 2015;45:1–100.
Suggested objective performance goals and clinical trial design for evaluating catheter-based treatment of critical limb ischemia. J Vasc Surg. 2009;50:1462–73 e1–3.
Destatis SB(2014) Krankenhausdiagnosestatistik. Statistisches Bundesamt DeStatis, Bonn. https://www.gbe-bund.de/.
Vascular Quality of Life Questionnaire-6 faciliates health-related quality of life assessment in peripheral arterial disease. J Vasc Surg. 2014;59:700–7.e1..