A core outcome set for trials in miscarriage management and prevention: An international consensus development study

To develop core outcome sets (COS) for miscarriage management and prevention.


| I N TRODUC T ION
Miscarriage is defined as the spontaneous loss of a pregnancy before viability; specifically a pregnancy loss is the spontaneous demise of a pregnancy, which has been confirmed by at least two positive βhuman chorionic gonadotropins (hCGs) in the serum or urine. 1 Clinical miscarriage affects around 15% of pregnancies, which translates to approximately 23 million miscarriages a year worldwide. 2 There is a great deal of research focused on trying to reduce miscarriage rates and improve the care for couples who suffer pregnancy loss. However, studies on miscarriage often do not address the same outcomes, making it difficult to draw conclusions and make recommendations when the evidence is synthesised. A systematic review of published trials on miscarriage management and prevention found 112 different outcomes used in miscarriage management trials and 61 outcomes for miscarriage prevention trials. 3 Core outcome sets (COS) are an agreed, standardised set of outcomes based on what key stakeholders (e.g. healthcare providers, patients, researchers, guideline developers and funding organisations) consider the essential and guideline decision making outcomes in the management or prevention of a condition. 4 The use of core outcome sets reduces inconsistencies in trial reporting, allowing results from different studies to be accurately compared and combined, and thus can reduce research waste. Core outcome sets are now being widely used across all specialties for several health conditions. The Cochrane collaboration and the National Institute for Health Research Health Technology Assessment (UK) advocate their use. Examples of COS specific to women's health include those published on endometriosis, 5 postpartum haemorrhage, 6 pre-eclampsia 7 and subfertility. 8 Developing a COS involves three stages. First, investigators systematically review published trials and conduct qualitative interviews with patients to collate all potential core outcomes. The systematic review relevant to our miscarriage COS was published in 2019. 3 Secondly, COS developers refine the list of core outcomes using formal consensus methods. The third stage is to determine the final list of core outcomes and outline how each outcome should be defined and measured. The work of this paper describes the process of stages 2 and 3 and the resulting final COS for both miscarriage management and prevention trials. This study has focused on the production of the core outcomes themselves; the standardisation of definitions and measures will be reported in a separate paper. The aim of this COS is to form a foundation on which to support clinical guideline decisionmaking and improve the quality of future research for couples suffering a miscarriage. In addition, this work will help to improve the care and understanding of the efficacy interventions for miscarriage treatment and prevention.

| M ET HODS
This study was prospectively registered with the Core Outcome Measures in Effectiveness Trials (COMET) initiative under registration numbers 679, 815 and 816 (http://www.comet -initi ative.org). The original plan was for separate medical and surgical management of miscarriage core outcome sets; however, it became apparent during stage 1 (the systematic review) that separate miscarriage management core outcome sets were not necessary and thus the two have been combined. We established an international steering group, including healthcare providers, researchers and patients, and published a protocol describing the study's methods. 9 Results: The final COS for miscarriage management comprises six outcomes: efficacy of treatment, heavy vaginal bleeding, pelvic infection, maternal death, treatment or procedure-related complications, and patient satisfaction. The final COS for miscarriage prevention comprises 12 outcomes: pregnancy loss <24 weeks' gestation, live birth, gestation at birth, pre-term birth, congenital abnormalities, fetal growth restriction, maternal (antenatal) complications, compliance with intervention, patient satisfaction, maternal hospitalisation, neonatal or infant hospitalisation, and neonatal or infant death. Other outcomes identified as important were mental health-related outcomes, future fertility and health economic outcomes. Conclusions: This study has developed two core outcome sets, through robust methodology, that should be implemented across future randomised trials and systematic reviews in miscarriage management and prevention. This work will help to standardise outcome selection, collection and reporting, and improve the quality and safety of future studies in miscarriage.

K E Y W O R D S
consensus development study, core outcome set, Delphi method, miscarriage management, miscarriage prevention

| Identification of outcomes
The core outcome set was developed in a three-stage process using methods advocated by the COMET initiative. 4 Potential core outcomes were identified through a systematic review of previously published trials in miscarriage management and prevention. 3 In addition, we performed qualitative work by conducting patient focus groups and semistructured interviews with 15 couples, seen in the Tommy's miscarriage centre clinics in the UK, with lived experiences of miscarriage. Interviews were completed when thematic saturation was achieved. To avoid replication of outcomes, the study steering group reviewed the list of all outcomes identified from the literature, group sessions and interviews before developing the list for the consensus survey. A final comprehensive inventory of outcomes and plain-language descriptions in consultation with the study's steering group was developed. This inventory was entered into a modified Delphi method, which was delivered through online surveys using Delphi survey software (DELPHIMANAGER, University of Liverpool).

| Delphi surveys
We invited a wide range of stakeholders, through national and international platforms, to participate in the Delphi surveys: healthcare providers specialising in early pregnancy (including physicians, nurses and allied healthcare providers), international experts, researchers, charities or patient support groups (e.g. Tommy's Charity and the Miscarriage Association, UK) and couples with lived experience of miscarriage. The Delphi method does not depend on statistical power 10 and therefore we did not set a target number for each stakeholder group. Instead, we endeavoured to recruit as many as possible within each group.
The study's steering group piloted the round 1 survey before implementation. We sought feedback regarding the survey instructions, ease of completion, the appropriateness of terminology and time taken to complete the survey. We made relevant adjustments in response to feedback.
In round 1, participants scored individual outcomes on a nine-point Likert scale, scoring between one ('not important') and nine ('critical'). This scale was devised by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group and is widely used for COS development. 11 Participants could select an 'unable to score' category if they did not have enough expertise or experience to score a particular outcome. At the end of the survey, participants were able to suggest additional outcomes. After the round-1 survey closed, the scores for each outcome were aggregated across individual stakeholder groups. The percentage of participants scoring each outcome (from one to nine) was calculated and tabulated for individual stakeholder groups. The steering group considered additional outcomes as suggested by survey users, and relevant outcomes that had not been present in round 1 were entered into round 2.
In round 2, participants received their own scores and individual stakeholder group feedback for each of the round-1 outcomes. Participants were asked to reflect on their own scores and on the scores of other participants before rescoring each outcome. Participants could also score the additional outcomes suggested by round-1 survey responders.
The 70/15% consensus definition advocated by the COMET initiative was applied to the round-2 Delphi survey results. 4 This meant that a consensus outcome was identified when >70% of participants in each stakeholder group scored the outcome as 'critical for decision making' (with a score of 7-9) and <15% of participants in each stakeholder group scored the same outcome as being 'of limited importance' for 'decision making' (with a score of 1-3).

| Consensus meeting
Following the Delphi survey, a virtual consensus development meeting was arranged. The panel consisted of experts in early pregnancy invited from across the world and also included patients with lived experience of miscarriage. The consensus development meeting used a modified nominal group technique to prioritise consensus outcomes further. We invited healthcare providers, researchers and women with experience of miscarriage to participate. The modified nominal group technique does not depend on statistical power. We aimed to recruit at least 10 participants, as this number has yielded sufficient results and assured validity in other settings. 10,12 The final consensus meeting was held virtually via Zoom (Zoom Video Communications). Before the meeting, a list of the outcomes for review was disseminated to all members of the consensus group, enabling input from those unable to attend in person. For pragmatic purposes, we asked all participants to categorise each outcome as either 'essential' or 'desirable', with justification where appropriate. Responses from reviewers who could not attend the virtual meeting were included in the discussions. In the consensus meeting, the results of the Delphi surveys for both miscarriage prevention and miscarriage management were systematically reviewed and all outcomes were discussed. Each stakeholder was encouraged to join in discussions and was able to propose additional outcomes for review if deemed relevant. At the end of the meeting, a final set of agreed outcomes was produced. Figure 1 shows the number of participants and outcomes at each stage of the COS development process. The full list of outcomes evaluated and scored at each round of the Delphi survey and at the consensus meeting is given in Tables S1 and S2. A map demonstrating the geographical spread of participants is displayed in Figure 2.

| Miscarriage management
We included 34 outcomes from the systematic review in round 1 of the Delphi survey, with 129 participants from 15 different countries, of which seven were lower-or middleincome countries (LMIC) contributing 10 participants: India, Sri Lanka, Pakistan, Egypt, Brazil, Malawi and Nigeria. Table 1 shows a breakdown of participant demographics.
The stakeholder group with the largest representation was women and partners with experience of miscarriage. Most participants were female, from the UK, and aged 30-39 years. Figure 2 shows the geographical spread of the Delphi survey participants.
As suggested by stakeholders, additional outcomes were added into the round-2 Delphi survey, giving a total of 40 outcomes for review. The second round was completed by 110  participants, with a very similar demographic breakdown to the round 1 users (Table 1). Following round 2, no additional outcomes were suggested and 33 outcomes reached consensus; it was therefore decided that a third round of Delphi was unnecessary. The attrition rate between rounds 1 and 2 was 15%.
The expert consensus team was comprised of 22 people with representation from international experts in early pregnancy, WHO and patients with lived experience of miscarriage. Table 1 shows demographic details of the expert consensus team. The stakeholder groups of gynaecologists and international experts/researchers in early pregnancy were not mutually exclusive. Of the 22 individuals, 11 were able to attend the virtual consensus meeting; the remaining stakeholders sent their contributions and comments via email ahead of the meeting and were asked to review, and dispute if necessary, the final agreed outcome set following the meeting. The attendees to the virtual meeting were comprised of a patient representative, a WHO representative and nine international experts in early pregnancy, eight of whom were also gynaecologists. There was LMIC representation from Malawi, Sri Lanka and Nigeria. Of the 33 outcomes entered into the consensus discussions, the stakeholders agreed on a final list of six core outcomes for miscarriage management. The final outcomes for management are as follows: efficacy of treatment, heavy vaginal bleeding, pelvic infection, maternal death, procedure-related complications and patient satisfaction. Table 3 shows the final core outcome set for miscarriage management.

| Miscarriage prevention
A total of 36 outcomes from the systematic review were entered in round 1 of the Delphi survey. Round 1 was completed by 437 participants from 15 different countries, of which seven were LMIC contributing 11 participants: India, Sri Lanka, Pakistan, Egypt, Brazil, Malawi and Nigeria. Table 2 shows a breakdown of participant demographics. As was seen for the miscarriage management survey, the stakeholder group with the most representation was women and partners with experience of miscarriage. The majority of participants were also female, from the UK and aged 30-39 years ( Table 2).
Additional outcomes suggested by participants were added into round 2 of the Delphi survey, giving a total of 40 outcomes for review. This was completed by 246 participants, with a very similar demographic breakdown to round-1 users ( Table 2). Following round 2, participants did not suggest any additional outcomes, and 31 outcomes reached consensus. Therefore a third Delphi round was not necessary. The attrition rate between rounds 1 and 2 was higher for miscarriage prevention than for miscarriage management (44%). The 31 outcomes from the Delphi survey were distributed to all experts for review and were discussed in the virtual consensus meeting. Stakeholders agreed on a final list of 12 core outcomes: pregnancy loss, live birth, congenital abnormalities, fetal growth restriction, gestation at birth, pre-term birth, neonatal or infant death, maternal complications, compliance with intervention, patient satisfaction, maternal hospitalisation, and neonatal or infant hospitalisation. The following measures were suggested for selected outcomes: pregnancy loss, live birth, fetal growth restriction, gestation at birth and pre-term birth. Table 3 shows the final core outcome set for miscarriage prevention.

| Additional important outcomes
Discussions within the consensus meeting revealed outcomes that were considered highly important but which may not be easily reported or captured in all studies, particularly in LMICs, namely, mental health-related outcomes, future fertility and health economic outcomes. These outcomes did not reach a consensus for inclusion in the final COS for miscarriage management or prevention. Still, due to their relative importance, particularly as considered by the patient representatives, they have been included as outcomes that researchers should still strive to report. Table 4 details these additional outcomes.

| Main findings
This study has applied validated consensus science methods, involving healthcare professionals, researchers, international experts and couples with lived experience of miscarriage to develop a COS for both miscarriage management and miscarriage prevention trials. The final COS for miscarriage management comprises six outcomes: efficacy of treatment, heavy vaginal bleeding, pelvic infection, maternal death, treatment or procedure-related complications, and patient satisfaction. The final COS for miscarriage prevention T A B L E 2 Participant characteristics for miscarriage prevention core outcome sets. comprises 12 outcomes: pregnancy loss <24 weeks' gestation, live birth, gestation at birth, pre-term birth, congenital abnormalities, fetal growth restriction, maternal (antenatal) complications, compliance with intervention, patient satisfaction, maternal hospitalisation, neonatal or infant hospitalisation, and neonatal or infant death. Additional outcomes highlighted as important, but not included in the final core outcome sets, were mental health-related outcomes, future fertility and health economic outcomes. This work will standardise outcome selection, collection and reporting across future miscarriage trials, systematic reviews and meta-analysis of individual patient data.

| Strengths and limitations
Our study followed robust methodological standards for COS development as set out by the COMET initiative. 13 Our study met all 11 minimum standards established by the COS-STAD project, across three broad domains, to be followed by COS developers when planning their projects, and by users when deciding whether a COS development applied reasonable methods. 13 There was broad representation across all stakeholder groups for both Delphi surveys and the final consensus panel, which included gynaecologists, international experts, researchers in early pregnancy, and women and partners who have suffered miscarriage. In addition, there was international representation, with stakeholders from 15 countries and WHO. The stakeholder group with greatest representation was patients and partners, we believe this to be a strength of our COS, as this is the population most affected by our recommendations.
Overall, there is uncertainty and lack of consensus regarding COS development methodology and what the optimal approaches should be, for example, to select participants in the best way. 14 Within our study there were some areas of potential bias regarding the representativeness of the study stakeholders. For example, when considering the Delphi survey, there was a higher response from stakeholders who lived in Europe than in other countries, and a higher response from couples with lived experience. We recognise that this skewed population of participants in the Delphi survey could have affected the outcomes prioritised. However, the balance of individuals within the consensus panel is likely to have addressed this issue. Another limitation was that to participate in the Delphi survey, English proficiency and access to a computer and the internet were required. Limitations in the representativeness of the sample could have impacted the outcomes prioritised, particularly as there was minimal representation from LMIC countries (8% in the miscarriage management COS and 3% in the prevention COS). However, by having a wide range of outcomes from the previous international literature that fed into the COS process and having three LMIC representatives in our expert group, this limitation has been addressed to some degree.
The attrition rate between rounds 1 and 2 of the Delphi survey for miscarriage management was 15% and for miscarriage prevention it was 44%. It is unclear why there was a wide discrepancy in the attrition rates for the two parallel studies, as they were released simultaneously and subject to the same advertising through research groups, charities, T A B L E 3 Core outcome set for miscarriage management and miscarriage prevention.

Domain Outcome
Miscarriage management social media and relevant academic conferences. There was also disparity in the attrition rates between the stakeholder groups. For miscarriage management, between rounds 1 and 2, an 8% attrition rate was seen for gynaecologists, compared with 18% for patients and partners. Similarly, for miscarriage prevention, only 3% of gynaecologists failed to complete round 2, compared with 48% of patients and partners. These figures are comparable to other core outcome development studies. 15 It may have been possible to reduce attrition, particularly within the patient and partner stakeholder group, by reducing the survey length. However, attrition rates needed to be balanced with the requirement to enter a comprehensive list of potential core outcomes into the Delphi survey and for participants to be able to consider and re-score individual outcomes in relation to each other.
There was a large difference in the number of stakeholders who completed the miscarriage management Delphi survey compared with the prevention survey (n = 129 versus n = 437, respectively). Ideally, we would have aimed at a similar number of stakeholders for each COS. The disparity is largely based on the numbers of patients and partners who contributed; it is perhaps unsurprising that couples with experience of miscarriage were more engaged with participating in research on miscarriage prevention than miscarriage management. Although this is a potential bias, as the overall contribution by percentage of stakeholders in each survey is balanced we feel this disparity in total numbers does not affect the outcomes produced.
The a priori consensus definition applied in this study could be perceived as another potential limitation. The modified Delphi method used the 70/15% consensus definition, which is subjective and not based upon research. Although the notion of achieving consensus is critical to COS development, the definition of what constitutes 'consensus' remains unclear. This is an area where further methodological research is required. 16

| Interpretation
The systematic review preceding this study showed wide variation in the outcomes reported across miscarriage management and miscarriage prevention trials. The most commonly reported outcome for miscarriage management trials was efficacy of miscarriage treatment, which was reported in 64/114 (56%) trials. 3 Symptoms of bleeding and infection were only reported in 7/114 (6%) trials. 3 All three of the above-mentioned outcomes were agreed to be critical for inclusion in the miscarriage management core outcome set.
For miscarriage prevention, the systematic review found only one trial, of 94 trials, reporting on maternal death. 3 The outcome of maternal death was discussed in detail at the consensus group meeting. The agreement was that although maternal death is a very rare outcome, it should be included in the COS because of its significant impact and ability to be recorded easily. In addition, maternal death was an outcome which scored highly in the Delphi surveys, particularly among the patient and partner stakeholder group.
One of the most debated subjects within this study was whether to include mental health-related outcomes within the final COS. The patient and partner stakeholder group were the primary proponents for inclusion of mental health and future fertility outcomes. In the final consensus meeting, it was agreed by all, including the patient stakeholders, that although mental health-related outcomes are of high importance, collecting the data for these outcomes would be difficult in low-or middle-income settings. To make the COS as pragmatic and widely applicable as possible, it was decided that alongside the COS there would be a list of additional important outcomes which researchers should strive to report, or at least the reasons given for a lack of reporting.
Patient satisfaction is an outcome included in both the miscarriage management and prevention COS. Our consensus group agreed that assessing acceptability and involving patients in the trial design work-up would be crucial to defining patient satisfaction and how to measure it. The definitions and measurements of all the outcomes will be addressed in a separate paper.
It is considered good practice to develop clinical trial protocols using the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement. 17 This statement specifically recommends the use of COS where they exist. In addition, the Core Outcomes in Women's and Newborn Health (CROWN) initiative aims to tackle poor outcome selection, collection and reporting within our specialty. 18 It will be important to demonstrate the uptake of the COS for miscarriage management and prevention in order to quantify their contribution to tackling research waste. In addition to standardising outcome reporting, it is crucial also to standardise the definitions and measures applied to the outcomes. A follow-on piece of work expanding into specific definitions, measures and also statistical considerations for the outcomes in miscarriage management and prevention will be conducted. Future work following on from both studies will be to assess COS uptake and explore why researchers do, or do not, implement the core outcome sets.

| Conclusion
This study has developed two core outcome sets, on miscarriage management and prevention, that should be implemented across future randomised trials and systematic reviews. This work will help to standardise outcome selection, collection and reporting, and improve the quality of future studies and meta-analyses of individual patient data on miscarriage research. Further research will be conducted to standardise the definitions and measures for the miscarriage core outcomes identified in this study.

AC K NO W L E D GE M E N T S
None.

F U N DI NG I N FOR M AT ION
This study was funded as part of a postdoctoral fellowship award from the National Institute for Health Research (NIHR), reference: PDF-2015-08-099.

C ON F L IC T OF I N T E R E S T S TAT E M E N T
None declared. Completed disclosure of interest forms are available to view online as supporting information.

DATA AVA I L A BI L I T Y S TAT E M E N T
Data available on request from the authors.