Predicting risk of postpartum haemorrhage: a systematic review

Postpartum haemorrhage (PPH) causes substantial morbidity and mortality worldwide. A reliable prognostic tool for PPH has potential to aid prevention efforts.


Introduction
Postpartum haemorrhage (PPH) remains a leading cause of morbidity and mortality globally, and was the second highest cause of direct maternal death in the UK in 2013-2015. 1 The incidence of PPH is problematic in developing countries but is also noted to be increasing in developed countries. 2,3 Early diagnosis is essential in the management of PPH, but diagnosis of PPH itself also presents a challenge due to the reliance upon quantification of the volume of blood loss. For vaginal delivery, cut-offs for haemorrhage are typically over 500 ml of blood loss and for caesarean section (CS) over 1000 ml. 4,5 Prevention of PPH could be achieved through identification of women at highest risk, allowing for measures to be taken for active management of third stage of labour, the presence of experienced clinicians and immediate access to resources such as oxytocin infusion and tranexamic acid. There are numerous studies identifying individual risk factors for PPH, 6,7 but these don't reliably identify women at greatest risk by combining multiple risk factors. A combination of risk factors is common in practice but quantifying the associated risk without the aid of a clinical prediction model is challenging. Once a reliable and high performing prediction model is developed, this could be converted into a user-friendly tool such as an online risk calculator or embedded within electronic health records. 8 A review by Kleinroueler et al. (2016) found over 200 prognostic models available in obstetrics, three of which related to PPH. 9 The review found very few models in any area of obstetrics that were being applied in routine clinical practice and the majority of studies did not present model formulas to allow researchers to conduct independent external validation of the models. 10 To advance efforts to identify women at risk of PPH as early and accurately as possible, a systematic review of existing prognostic models was considered essential. This would enable assessment of existing models for their suitability for immediate use, or identify those which perform well internally but require external validation on an independent cohort before consideration for clinical use. This approach has potential to be more efficient than the addition of a new model to aid prevention of PPH. 9 Since publication of the aforementioned review, several attempts at developing prognostic models for PPH have been published. This review aims systematically to identify and appraise studies which develop prognostic models that can predict the chance of PPH in pregnant women.

Methods
This review adhered to principles outlined in guidance published by PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analysis), 11 CHARMS (CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies) and PROBAST (Prediction model risk of bias assessment tool for studies developing, validating or updating prediction models). 12 The protocol for this review has been published by PROS-PERO and is available online. 13 A literature search was conducted from 1946 to 25 May 2020 in the following databases: MEDLINE, Embase, CINAHL and the Cochrane Library, following liaison with a librarian. To inform the full search strategy, a limited search of MEDLINE was first conducted followed by an extensive search of the literature of the aforementioned databases. Hand-searching of reference lists of included articles was also performed. A copy of the search strategy for MEDLINE and Embase is available in Appendix S1. The main search terms were 'predict$', 'risk score' and 'postpartum haemorrhage' with the appropriate synonyms adopted.
Inclusion criteria and exclusion criteria for this review are outlined in Table 1. Titles and abstracts were independently screened by two reviewers (CN and SN) and any disagreements resolved by a third reviewer (MB).
Data extraction and risk of bias/applicability assessment (at study level) were conducted independently in accordance with the CHARMS checklist and PROBAST tools, respectively. The risk of bias review allowed identification of potential bias in primary studies and identified limitations to applicability of the results. Items extracted from each study included: source of data; participants; outcome to be predicted; candidate predictors (or index tests); sample size; missing data; model development; model performance; model evaluation; results and interpretation (including whether authors deemed their model fit for purpose or nature of further research required before using). The PROBAST tool incorporates assessment of risk of bias and applicability relating to the data source, the predictors and outcome assessed, and the analysis. This includes whether the data source has appropriately included or excluded women to allow a correct probability to be calculated and whether the participants match the review question. Assessment of bias in model performance due to predictors includes the definition and measurement of predictors, e.g. bias could arise due to knowledge of the outcome data when assessing predictors or lack of availability of the predictors when the model is intended to be used. PROBAST supports assessment of the outcome studied by considering how it was determined, how objective it is, whether it incorporates any predictor data, how consistently it was determined across individuals, timing of determination, whether this was independent of knowledge of

Results
The search strategy identified 2146 citations; following removal of duplicates and screening, 56 full-text articles were assessed for eligibility (PRISMA Flow Diagram, Figure 1). This review included 14 studies 14-27 with a total of 14 final prediction models identified.
The populations of the included studies are shown in Tables S1 and S2. Five studies 16,18,21,22,27 included only women with placenta praevia, four studies 17,23-25 included only vaginal deliveries, three studies 14,15,20 had a population consisting of CS (planned and unplanned), one study defined population as consisting of women with placenta accreta spectrum 26 and one study 19 had a population encompassing the general obstetric population.
The key findings of the studies are detailed in Table S1, including whether the study, as judged by the primary study authors, is to be interpreted as exploratory (requiring more research) or confirmatory (of use in clinical practice). All candidate predictors and the predictors included in the final published models is listed in Table S2. The setting of the included studies were hospitals in the following countries; Italy, China, France, USA, UK, South Korea, Netherlands, Spain, Zimbabwe, Denmark and Egypt. The study designs included were 11 cohort studies, [14][15][16]18,20,[22][23][24]26,27 of which one used whole population registry data, 21 and three case control studies, 19,25 of which one was nested within a population cohort. 17 The number of participants included in each study ranged from 110 in a prospective cohort study to 56 967 in a retrospective cohort. 14,27 Despite the attempt to predict PPH across all studies, the chosen outcomes differed. Seven studies [17][18][19][20]22,23,26 listed PPH or massive haemorrhage as an outcome, four studies [14][15][16]21 listed blood transfusion or massive blood transfusion as an outcome, two studies 24,25 reported postpartum blood loss and one study 27 had a combined outcome of peripartum complications encompassing perioperative blood transfusion or uterine artery embolisation or caesarean hysterectomy. There is also variation in the definition and method of measurement of each outcome, as shown in Table S1.
The risk of bias and applicability, assessed using the PROBAST tool, is summarised in Table 2. Overall, there was a high risk of bias across the studies. The participants were deemed as a source of low risk of bias in ten studies, [14][15][16][17]20,[22][23][24]26,27 with three studies at high risk due to large proportion of women excluded due to incomplete data without exploration of how these women compared with those included, 18,19,21 and one at high risk due to retrospective selection of women in a case-control study with a control group at high risk of PPH. 25 Eight studies were at high risk of bias due to predictors being available only after the birth (e.g. neonatal birthweight) or due to lack of detail on how and when these were assessed. [15][16][17]19,21,22,25,26 Three studies were at high risk of bias due to a lack of definition or method of measurement of the outcome to be predicted. 19,25,26 In relation to the analysis conducted, all studies except one were deemed to be at high risk of bias. Seven studies had a small sample size with a low number of events per variable (EPV). 16,18,[21][22][23][24]27 Risk of bias for missing data was uncertain for all papers because none reported any missing data beyond those where women with incomplete data were excluded at the outset.
The predictive ability of the statistical models evaluated using measures of calibration (concerned with agreement between the predicted probabilities of the outcome and the observed proportions of the outcome) and discrimination (how well the model can differentiate between patients with high and low risk) 28 was evident in six 14,15,17,21,24,26 and six [14][15][16]19,20,22   calibration curve demonstrating the non-parametric correlation between observed outcome and predicted risk. 30 Albright et al. 15 assessed calibration in this manner. Discrimination was reported as the area under the receiver operator curve (AUC) where 1 is perfect discrimination and 0.5 is no better than a coin toss. The AUC ranged from 0.70 to 0.9 across all studies, as shown in Table S1. Eight studies attempted to address risk of overfitting or optimism through internal validation. The approach used varied from a random split of data (developing the model in one split and testing it in another) 14,15,18,20,24 to threefold cross-validation 26 and bootstrapping techniques. 17,21 Three studies described external validation which ranged from temporal sampling (testing on a more recent sample of data) 21,24 to testing in a different geographical location. 26 Of 14 studies, eight presented validated models deemed by their primary study authors as ready for use in clinical practice. 14 Given that all studies were at high risk of bias due to aspects of predictors, outcome or analysis, none was considered ready for clinical use by the review authors. Those which performed reasonably well (AUC ≥0.7) in the development phase and which withstood testing for overfitting or optimism, are deemed suitable for robust external validation. These include Ahmadzia et al., 14 Albright et al., 15 Dunkerton et al. 20 and Kim et al. 21 (all for women undergoing CS), Biguzzi et al. 17 and Rubio-Alvarez et al. 24 (for women having vaginal birth), Chen et al. 18 (for women with placenta praevia) and Wu et al. 26 (for women with placenta accreta spectrum disorders).

Main findings
This review is, to our knowledge, the first systematically to identify published studies attempting to provide risk scoring or prognostic models for prediction of PPH. Of 14 included, eight have been internally validated; three of these include externally validated risk tools. All three are at high risk of bias due to analytical issues. Both Kim et al.

Strengths and limitations
A strength of this review is the prospective publication of the protocol in PROSPERO and strict adherence to this. The aim was to find a robust and clinically meaningful formula or tool which could be of use to a clinician in daily practice. Although all studies produced a formula, scoring system or tool for predicting PPH, these may not be appropriate as they may encourage use of a poorly validated model. Numerous related studies [32][33][34] have not published a useable tool or logistic regression model with a formula for use by clinicians in clinical practice, as they describe poor (or poorer than anticipated) performance of the model. This review benefits from use of broad and general search criteria to maximise identification of relevant studies. Additionally, the results yielded by the search strategy were doublescreened by two reviewers (CN and SN). The use of the CHARMS checklist allowed for systematic data extraction and use of PROBAST supported systematic assessment of risk of bias and applicability. 12 A limitation of this review is that it was not possible to obtain three studies which might have been appropriate for inclusion. One of these was part of an unpublished PhD thesis 35 and the other two were behind a paywall. 36,37 In addition, the value of findings may theoretically be limited due to inclusion only of studies in English language; however, in reality, no (otherwise eligible) non-English studies were identified in the search.
This review highlights shortcomings regarding the risk of bias and reporting of the included studies.

Interpretation
This review suggests that there are no published prediction tools for PPH ready for clinical use. Future research to generate prognostic models for use specifically in elective CS or in women aiming for vaginal birth would facilitate advanced planning of personnel to optimise care provided.
The clinical usefulness of models generated by some of the identified studies is limited by the target population. Four studies 17,23-25 focus on vaginal births, which is of limited use as vaginal delivery cannot be guaranteed in advance. The circumstances during labour are subject to change, with a risk of CS present until the fetal head is delivered, thus the tool would no longer be applicable once a decision for CS is made. Only one study 19 produced a scoring tool aimed at use in the general obstetric population, but the study design was unclear and attempts to contact the author were unsuccessful. That study included 923 women in Beijing, China, of whom almost half had a PPH, and it did not assess predictive performance via internal or external validation. Therefore, despite the presentation of an equation to predict PPH with AUC of 0.86, its lack of performance assessment means it cannot be recommended for use in clinical practice.
Most studies were retrospective, meaning that some relevant predictors may not have been measured, but the vast majority of known risk factors for PPH can be assessed retrospectively so this is not considered a major concern. 38,39 Some studies' prediction models or tools are clinically unhelpful in regard to the final predictors included, as some were not known at the time of birth. Both Biguzzi et al. and Rubio-Alvarez et al. included neonatal birthweight as a predictor, which suggests that the intended time for the nomogram and risk tool use is after weighing of the baby, most likely once the highest risk of PPH has passed. These models are therefore of limited value for preparation of resources prior to birth. Estimated birthweight may be a more appropriate measure but has not been included as a predictor in any model.
Use of intrapartum factors can aid risk assessment in a dynamic scenario. Two studies 23,24 have included these: duration of the first and second stage of delivery and nonuse of uterotonics and cord traction. Intrapartum risk scoring may be facilitated by use of electronic health records, where the tool could be embedded within the system, but otherwise it may present logistical difficulties if it requires ongoing computer access as per Rubio-Alvarez et al.'s proposed risk tool.
Robust external validation was absent from all prediction models identified, suggesting that this is poorly understood and undervalued. Of the models externally validated 21,24,26 all utilised Hosmer-Lemesow testing, which is not recommended, 28 and only two provided validation results. Internal validation is a reasonable alternative, as this assesses how well the model performs in the underlying population from which the model was developed, but only eight studies 14,15,17,18,20,21,24,26 did this and only two considered the model for prospective use in a population of placenta praevia or placenta accreta spectrum disorders. 21,26 The prediction models identified were at high risk of bias overall, with small sample size and suboptimal statistical analysis being common, and missing data not reported in any study. Without missing data information, it is not possible to assess fully the related risk of bias. 39 The need for adequately powered studies is clear. Half the included studies showed a low EPV (<20) with only one 24 conducting any shrinkage methods to overcome problems arising from overfitting of the model (and risk of optimistic predictions) when there is a low number of events. Despite this, several authors recommended use of affected models without external validation. 14,15,18,20 As a result of heterogeneity and low EPV, it was not possible to conduct a meta-analysis of the results. There is potential for individual participant data meta-analysis of findings for predicting PPH in a population of women with placenta praevia.

Conclusion
Three PPH risk prediction tools reviewed have potential for clinical use pending robust external validation: one in cases of CS (Leicester PPH predict score), 20 one for the prediction of massive transfusion in CS with known placenta praevia 21 and one for prediction of PPH in women with placenta accreta spectrum disorders undergoing MRI placental evaluation ahead of CS. 26 Development and robust validation of PPH prediction tools applicable to the general obstetric population is needed.

Disclosure of interests
None declared. Completed disclosure of interest forms are available to view online as supporting information.

Contribution to authorship
CN drafted the manuscript and performed the first literature search, data extraction and analysis. SN performed the second screening of the titles, abstracts and full papers and approved the final draft of the manuscript. DJM commented on all versions of the manuscript and provided methodological advice throughout the study. MB designed the study, supervised all steps and commented on all drafts of the manuscript. MB is corresponding author.

Funding
This study received no external or specific source of funding.

Supporting Information
Additional supporting information may be found online in the Supporting Information section at the end of the article.
Appendix S1. MEDLINE search strategy. Table S1. Key components of each study. Table S2. Study population of included studies alongside predictors. &