A core outcome set for pre-eclampsia research: an international consensus development study

Objective To develop a core outcome set for pre-eclampsia. Design Consensus development study. Setting International. Population Two hundred and eight-one healthcare professionals, 41 researchers and 110 patients, representing 56 countries, participated. Methods Modiﬁed Delphi method and Modiﬁed Nominal Group Technique. Results A long-list of 116 potential core outcomes was developed by combining the outcomes reported in 79 pre-eclampsia trials with those derived from thematic analysis of 30 in-depth interviews of women with lived experience of pre-eclampsia. Forty-seven consensus outcomes were identiﬁed from the Delphi process following which 14 maternal and eight offspring core outcomes were agreed at the consensus development meeting. Maternal core outcomes: death, eclampsia, stroke, cortical blindness, retinal detachment, pulmonary oedema, acute kidney injury, liver haematoma or rupture, abruption, postpartum haemorrhage, raised liver enzymes, low platelets, admission to intensive care required, and intubation and ventilation. Offspring core outcomes: stillbirth, gestational age at delivery, birthweight, small-for-gestational-age, neonatal mortality, seizures, admission to neonatal unit required and respiratory support. Conclusions The core outcome set for pre-eclampsia should underpin future randomised trials and systematic reviews. Such implementation should ensure that future research holds the necessary reach and relevance to inform clinical practice, enhance women’s care and improve the outcomes of pregnant women and their babies.


Introduction
When untreated, pre-eclampsia is life-threatening, and in low-and middle-income countries it is one of the leading causes of maternal mortality, severe maternal morbidity and stillbirth. 1 The development of effective and safe treatments for pre-eclampsia is urgently needed. Potential treatments should be evaluated in randomised trials, and to ensure the greatest gains in reducing the current burden of mortality and severe morbidity associated with pre-eclampsia, research should be undertaken in all settings, including low-and middle-income countries. Several national and international organisations, including the World Health Organization, have prioritised over 50 unanswered research questions relating to the evaluation of potential treatments for pre-eclampsia. [2][3][4] However, complex issues including a failure to consider the perspectives of women with lived experience of pre-eclampsia when designing randomised trials, variations in outcome measures and outcome reporting bias, could undermine the translation of future pre-eclampsia research into clinical practice. 5 Such research waste represents a substantial barrier to improving the care that women and their babies receive.
A recent systematic review characterised outcome reporting across published pre-eclampsia trials. 6 This systematic evaluation illustrated the widespread variation in the reporting of maternal and offspring outcomes. Most preeclampsia trials did not report information on clinically important outcomes, including stroke, liver failure and renal failure, and did not evaluate efficacy and safety in the participants' infants, particularly over the longer term.
The challenges of poor outcome selection, measurement and reporting can be addressed by developing, disseminating and implementing a core outcome set for future preeclampsia research. 7 A core outcome set represents a minimum dataset, developed using robust consensus science methods, engaging diverse stakeholders including healthcare professionals, researchers and patients. 7,8 Core outcomes should be routinely used by researchers, collected in a standardised manner and reported consistently in the final publication allowing comparability between individual randomised trials and efficient meta-analysis. 7,9 The objective of this study was to develop a clinically relevant core data set to standardise outcome selection, collection and reporting across future randomised trials and systematic reviews evaluating potential treatments for preeclampsia.

Methods
The study was prospectively registered with the Core Outcome Measures in Effectiveness Trials (COMET) Initiative, registration number 588. A protocol with explicitly defined objectives, formal consensus development methods, criteria for participant identification and selection, and statistical methods has been published. 10 An international steering group, including healthcare professionals, researchers and women with lived experience of pre-eclampsia, was established to provide a perspective to inform key methodological decisions. The core outcome set was developed in a three-stage process using consensus science methods advocated by the COMET initiative. 7 There is no international consensus regarding the diagnostic criteria for pre-eclampsia. The study did not seek to reach consensus regarding the definition of pre-eclampsia and adopted the International Society for the Study of Hypertension in Pregnancy's pre-eclampsia definition, which is defined as gestational hypertension presenting with new-onset proteinuria, other maternal organ dysfunction and/ or uteroplacental dysfunction. 11 This study is complementary to the work of the Global Pregnancy Collaboration and the International Society for the Study of Hypertension in Pregnancy who are engaged with the standardisation of other aspects of study design, the development of a standardised database for perinatal research studies and the development of clinical practice guidelines.
Potential core outcomes were identified by extracting outcomes reported in published pre-eclampsia trials and undertaking a thematic analysis of in-depth interviews with women with lived experience of pre-eclampsia. Both studies have been published. 6,12 A comprehensive inventory of outcomes and plain-language descriptions was developed in consultation with the study's steering group. This inventory was entered into a modified Delphi method, which was delivered through sequential online surveys using Delphi survey software (DELPHIMANAGER, University of Liverpool, Liverpool, UK).
Healthcare professionals, researchers and women with lived experience of pre-eclampsia were invited to participate. Healthcare professionals were recruited through the Core Outcomes in Women's and Newborn Health (CROWN) initiative, Global Obstetrics Network and the International Society for the Study of Hypertension in Pregnancy. Researchers were recruited through participation in ongoing pre-eclampsia research studies including: (1) community blood pressure monitoring in rural Africa and Asia; 13 (2) detection of underlying pre-eclampsia study, development and validation of a prediction model for risk of complications in early-onset pre-eclampsia study; 14 (3) international prediction of pre-eclampsia individual patient data collaborative network and (4) pre-eclampsia: eclampsia monitoring, prevention and treatment initiative. 15 Women with lived experience of pre-eclampsia were recruited through patient organisations including Action on Pre-eclampsia; Count the Kicks; Group B Strep Support and Tommy's. The Delphi method does not depend on statistical power and between 10 and 15 participants has been demonstrated to yield sufficient results. [16][17][18] The study aimed to recruit at least 18 participants for each stakeholder group but planned to maximise the number to increase generalisability, anticipating an overall attrition rate of 20%.
The round 1 survey was piloted by the study's steering group before use. Feedback was specifically sought regarding the survey instructions, ease of completion, the appropriateness of terminology and time taken to complete the survey. The survey was adjusted in response to feedback.
Before entering the round 1 survey, participants received an explanatory video abstract, a plain-language summary and survey instructions, provided demographic details and made an explicit commitment to complete all three rounds. Following registration, a unique identifier was generated and allocated to each participant, to ensure that future responses were both linked and anonymised. In round 1, participants scored individual outcomes on a nine-point Likert scale. 19 Participants were able to select an 'unable to score' category if they did not feel they had sufficient expertise or experience to score an individual outcome. Before completing the survey, participants were able to suggest additional outcomes. After the round 1 survey had closed, the scores for each outcome were aggregated across individual stakeholder groups. The percentage of participants scoring each outcome at every possible response from one to nine was calculated by the Delphi survey software and tabulated for individual stakeholder groups. Suggested additional outcomes were reviewed by the steering group and unique outcomes were entered into round 2.
In round 2, participants received their own scores and individual stakeholder group feedback for each round 1 outcome. Participants were asked to reflect on their own scores and on the scores of other participants, before re-scoring each outcome. Before completing the survey, participants were able to score additional outcomes suggested by participants in the round 1 survey. After the round 2 survey had closed, the percentage of participants scoring each outcome at every possible response from one to nine was calculated and tabulated for individual stakeholder groups.
In round 3, participants received their own scores and individual stakeholder group feedback for each round 2 outcome. Participants were asked to reflect on their own scores and on the scores of other participants before re-scoring each outcome. After the round 3 survey had closed, it was agreed before review of the results that a consensus definition would be identified when >70% of participants in each stakeholder group scored the outcome 'critical for decision-making' (score seven to nine) and <15% of participants in each stakeholder group scored the outcome 'of limited importance for decision-making' (score one to three). 7 Participants who withdrew from the Delphi survey were requested to complete an anonymous online questionnaire providing free-text comments outlining their reason(s) for withdrawing. These responses were coded and summarised.
With regard to the other consensus method used in the study, the modified nominal group technique was delivered through a half-day consensus development meeting. 18 Healthcare professionals, researchers and women with lived experience of pre-eclampsia, resident in the UK and who had completed all three rounds of the Delphi survey were invited to participate in a consensus development meeting. Anyone who responded favourably was extended an invitation to attend the consensus development meeting. The modified nominal group technique does not depend on statistical power and there is no robust method for calculating the required number of participants. The study aimed to recruit between 10 and 15 participants as this number has assured validity in other settings. 18,20 Before the meeting, participants provided demographic details and made an explicit commitment to participate actively. All consensus outcomes were entered into the process. Participants were able to enter other outcomes which had not reached the consensus threshold, upon request. Following an initial discussion, outcomes were divided into three provisional categories: (1) outcomes to be considered for inclusion in the final core outcome set, (2) outcomes where no consensus existed and (3) outcomes that should not be considered for inclusion in the final core outcome set. Participants were invited to discuss the ordering of the outcomes within each category, considering contextual information, including the relative importance of individual outcomes, feasibility to collect the outcome data and the availability of suitable definitions and measurement instruments. They were encouraged to reformulate outcomes to improve clarity or comprehension. The discussion focused upon ranking the outcomes being considered for inclusion in the final core outcome set and the outcomes where no consensus existed. During the discussion, the outcomes could be moved between the categories. Finally, the core outcome set was agreed.
Descriptive statistics were used to describe participant demographics. Medians (x), interquartile ranges (IQR) and scoring distributions were calculated across individual stakeholder groups (healthcare professionals, researchers and patients) and pooled across individual outcomes. The skewness of each scoring distribution was calculated using Pearson's coefficient of skewness (Sk 2 ). All analyses were performed using GRAPHPAD PRISM (GraphPad, San Diego, CA, USA).
The study was funded by the National Institute for Health Research, Barts Charity and Elisabeth Garrett Anderson Hospital Charity Travelling Fellowship in Memory of Anne Boutwood, Royal College of Obstetricians and Gynaecologists.

Results
Seventy-nine pre-eclampsia trials reported 106 different outcomes and thematic analysis of 30 in-depth interviews with women with lived experience of pre-eclampsia identified 71 outcomes ( Figure 1). 6,12 Combining these resulted in 116 unique outcomes, which were entered into the Delphi survey. 21 The Delphi survey was started by 281 healthcare professionals, 41 researchers and 110 women with lived experience of pre-eclampsia, representing 31 high-income countries and 25 low-and middle-income countries (Table 1). Over the three Delphi survey rounds, 159 participants (37%) withdrew, including 100 healthcare professionals (35%), 11 researchers (27%) and 48 patients (44%). The majority of participants who withdrew from the survey provided an explanation (see Supplementary material, Table S1). In response to the outcomes suggested by participants, the steering group recommended the reformulation of 11 outcomes to improve clarity and added 20 new outcomes to round 2 (see Supplementary material, Figure S1). Therefore, 136 outcomes were subsequently entered into rounds 2 and Systematic review 107 potential core outcomes Qualitative patient interviews 70 potential core outcomes 116 potential core outcomes 136 potential core outcomes scored 136 potential core outcomes scored  The consensus development meeting included 11 healthcare professionals, two researchers and four women with lived experience of pre-eclampsia. Nine participants (56%) had lived, worked or conducted research in low-or middle-income countries. Forty-seven consensus outcomes were considered using the modified nominal group technique. Participants recommended the reformulation of nine consensus outcomes and entered an additional four no consensus outcomes into the process.
Participants prioritised 22 outcomes, comprising 14 maternal and eight offspring outcomes, for inclusion in the core outcome set for pre-eclampsia (Table 2). Outcomes represented maternal and infant mortality and severe morbidity. These included maternal mortality, stroke, pulmonary oedema, acute kidney injury, placental abruption and postpartum haemorrhage. Outcomes demonstrating the impact of pre-eclampsia on the fetus and neonate included stillbirth and neonatal mortality, gestational age at delivery and birthweight, and neonatal seizures. Finally, outcomes representing the resource utilisation resulting from the management of severe maternal and neonatal morbidity included the requirement for maternal admission to intensive care, the requirement for neonatal unit admission and respiratory support.

Main findings
Using robust consensus science methods, healthcare professionals, researchers and women with experience of preeclampsia have developed a core outcome set to standardise outcome selection, collection and reporting across future pre-eclampsia trials and systematic reviews.

Strengths and limitations
This study has met the recently published standards for core outcome set development, developed by an international  group of experienced core outcome set developers, methodologists and potential core outcome set end users. 22 By meeting these recommendations, this study has objectively demonstrated its methodological quality. When considering core outcome set development, a high number of diverse participants is desirable to secure the generalisability of the results and increase its credibility with other researchers. With over 400 participants from 56 countries, the global participation achieved in this study should secure the relevance of the results across an international context. The core outcome set for pre-eclampsia is perceived as an exemplar of core outcome set development and the study's approach has been adopted by many other core outcome set development studies. [23][24][25][26] This consensus study is not without limitations. There is considerable uncertainty regarding core outcome set development methods. 27 The optimal approaches to selecting participants, structuring interaction embedded in different consensus methods, including modified Delphi method, modified Nominal Group Technique and consensus development conference, and methods of synthesising individual judgements. Further methodological research is required to inform future core outcome set development.
When considering the Delphi survey, there was a higher response from participants who identified as white (83%), living in Europe (55%) and living in high-income countries (82%). To participate in the Delphi survey, English proficiency, a computer and internet access were required. Limitations in the representativeness of the sample could have impacted upon the outcomes prioritised; however, given the wide range of outcomes from the previous worldwide literature that fed into the process, this should not have been a major issue.
The study's attrition rate was 37%, which is comparable to other core outcome set development studies in health of women and neonates. 8 This did vary between stakeholder groups with more patients (44%) dropping out than other groups, such as healthcare professionals (35%) and researchers (27%). It may have been possible to reduce attrition, particularly within the patient stakeholder group, by reducing the length of the survey. However, attrition needed to be balanced with the requirement to enter a comprehensive longlist of potential core outcomes into the Delphi survey and for participants to be able to reflect on and re-score individual outcomes in relation to each other.
Although the notion of achieving consensus is fundamental to core outcome set development, the definition of what constitutes consensus is less clear. The pre-specified consensus definition applied within the Delphi survey could be considered as being too accommodating, as it resulted in the identification of 47 initial consensus outcomes. Further methodological research is required to develop an appropriate consensus definition that could accommodate the skewed scoring distribution of the respondents. 28 Interpretation Most trials evaluating potential treatments for pre-eclampsia have neglected to report many of the outcomes included in the core outcome set. 6,29 For example, only one-third of trials have reported eclampsia, less than a tenth of trials have reported stroke and only three trials have reported pulmonary oedema. 6 Selective reporting of outcomes, based on statistical significance could be contributing to these omissions. 30 Systematic implementation of the core outcome set for pre-eclampsia should ensure future pre-eclampsia research reports outcomes that matter to healthcare professionals, researchers and women with pre-eclampsia, and help to limit selective reporting of results based upon statistical significance. Such an approach could be replicated in other areas of the health of women and newborn infants, including endometriosis, twin-twin transfusion syndrome and neonatal care, to tackle the variation in outcome reporting and suspected outcome-reporting bias. [31][32][33] It is considered good practice for researchers planning randomised trials to implement the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement which outlines the scientific, ethical and administrative elements that should be addressed in a clinical trial protocol. 34 This statement specifically recommends the use of core outcome sets where they exist. In addition, the importance of implementing core outcome sets is recognised by the funders of health research.
The CROWN initiative, supported by over 80 specialty journals, have resolved to implement this core outcome set. 8 Participating journals will require researchers to report core outcomes within manuscripts and offer conclusions based on these outcomes. Where core outcome sets have not been collected, the researchers will be asked to report this deficiency and its implications for their findings. 9 The Cochrane Pregnancy and Childbirth Group have committed to implementing the core outcome set for preeclampsia when new and updated reviews are being prepared. Uptake of the core outcome set should facilitate the possibility of more sophisticated methods of evidence synthesis, including individual patient data meta-analysis and network meta-analysis.
The core outcome set has been developed specifically for comparative effectiveness research. The use of this core outcome set in other types of research is highly desirable. There is currently a research priority setting partnership developing research priorities for hypertension in pregnancy. 35 This work should be considered complementary to a wider agenda of reducing research waste across hypertension in pregnancy research. Such agendas have been proposed in areas relevant to women's health, including twin and multiple pregnancy research with research priority setting and core outcome set development as important components. 36,37 Blood pressure and severe hypertension were not identified as core outcomes. In adult non-pregnant populations, blood pressure is a valid surrogate outcome for heart disease, stroke and mortality. [38][39][40] In the context of preeclampsia research, maternal blood pressure has been commonly selected as a surrogate outcome, which represents a single pathway that operates within a complex multifactorial disease, characterised by vasoconstriction, coagulation and intravascular fluid redistribution, resulting in widespread formation of microthrombi and necrosis within maternal end organs. 6,41,42 Reliable conclusions around the impact of pre-eclampsia interventions cannot necessarily be informed by reductions in maternal blood pressure because the consequences of blood pressure changes upon a diverse range of clinically meaningful outcomes, including maternal mortality, pulmonary oedema and renal failure, may be unclear. Developing a core outcome set for pre-eclampsia represents an opportunity to deliver a paradigm shift by measuring treatment effectiveness as a clinical rather than a biological response. Researchers should continue to report blood pressure as a descriptive outcome.
Core outcomes require standardised outcome measures. Without a standardised approach, researchers would be able to choose from a variety of different outcome measures for individual core outcomes. Such variation can make it difficult to synthesise the results of individual trials within secondary research. 36 The collaboration has standardised definitions using formal consensus development methods to secure additional harmony across future preeclampsia trials and ensure that secondary research can be undertaken prospectively, efficiently and harmoniously. 43 It is intended that these consensus outcome measures will be used for core outcomes included in other core outcome sets relevant to the health of women and newborn infants. [44][45][46][47][48][49][50] Conclusion This core outcome set for pre-eclampsia should now underpin all future randomised trials and systematic reviews evaluating potential treatments for pre-eclampsia. Such rationalisation should ensure that future research addresses outcomes agreed as important in a consistent manner, facilitate meta-analysis, enhance patient care and ultimately improve the outcomes of pregnant women and their babies.

Disclosure of interests
CG received expenses to attend an educational conference from Chiesi Pharmaceuticals and his institution has received research funding from Chiesi Pharmaceuticals. SAK reports receiving research funding from Siemens, serving as a consultant to Roche and Thermo Fisher Scientific, having a financial interest in Aggamin Pharmaceuticals and holding multiple patents. AK reports being the inventor of the Hampton system. BWM reports consultancy fees from Guerbet, iGenomix, Merck KGaA and ObsEva. RJM reports research support from Omron. The remaining authors declare no competing interests. Completed disclosure of interests forms are available to view online as supporting information.

Details of ethics approval
Ethical approval was received from the National Research Ethics Service (reference number: 12/SC/0495; 1 July 2015).

Funding
This study was funded by the Barts Charity, Elisabeth Garrett Anderson Hospital Charity and National Institute for Health Research. The funders have no role in the design and conduct of the study, the collection, management, analysis or interpretation of data or the manuscript preparation.
We would like to thank the Delphi survey and consensus development meeting participants. We would like to thank the Radcliffe Women's Health Patient Participation group, Action on Pre-eclampsia and our patient and public representatives who assisted with study design, data interpretation and planned dissemination. We would like to thank colleagues at the Nuffield Department of Primary Care Health Sciences, University of Oxford including Jacqui Belcher, Carla Betts, Lucy Curtin, Dawn Evans, Caroline Jordan, Sarah King, Sam Monaghan, Nicola Small and Clare Wickings for administrative, technical or material support. We would like to thank Prof. Marian Knight, Nuffield Department of Population Health, University of Oxford, for providing subject-specific expertise. We would like to thank colleagues at the Women's Health Research Unit, Queen Mary, University of London, including Khalid Khan, Rehan Khan and Tracy Holtham for administrative and technical support or subject-specific expertise. We would like to thank David J. Mills for administrative and logistical support.

Supporting Information
Additional supporting information may be found online in the Supporting Information section at the end of the article. Figure S1. Participants and flow of outcomes. Table S1. Stated reasons for withdrawal from the Delphi survey.
Appendix S1. Round 3 Delphi survey results reported as the percentage of participants scoring the outcome as critical. &