SEARCH

SEARCH BY CITATION

Keywords:

  • Diagnostic tests;
  • IOTA;
  • ovarian neoplasms;
  • risk of Malignancy Index;
  • triage;
  • ultrasonography

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

Please cite this paper as: Van Calster B, Timmerman D, Valentin L, McIndoe A, Ghaem-Maghami S, Testa A, Vergote I, Bourne T. Triaging women with ovarian masses for surgery: observational diagnostic study to compare RCOG guidelines with an International Ovarian Tumour Analysis (IOTA) group protocol. BJOG 2012;119:662–671.

Objective  To compare guidelines from the Royal College of Obstetricians and Gynaecologists (RCOG) based on the Risk of Malignancy Index (RMI) with a protocol based on logistic regression model LR2 developed by the International Ovarian Tumour Analysis (IOTA) group for triaging women with an ovarian mass as low, moderate, or high risk of malignancy.

Design and setting  Observational diagnostic study conducted between 2005 and 2007 at 21 oncology referral centres, referral centres for ultrasonography and general hospitals.

Sample  In all, 1938 women undergoing surgery for an ovarian mass.

Methods  RCOG guidelines use the RMI to triage women as low (RMI < 25), moderate (25–250), or high (above >250) risk. The IOTA protocol uses LR2s estimated probability of malignancy (<0.05 indicates low risk, ≥0.05 but <0.25 moderate risk, and ≥0.25 high risk).

Main outcome measure  Percentages of benign, borderline and invasive tumours classified as low, moderate or high risk.

Results  The IOTA and RCOG protocols classified 71.1% and 62.1% of benign tumours as low risk, respectively (difference 9.0; 95% CI 6.2–11.9, < 0.0001). Of invasive tumours, 88.6% and 73.6% were labelled high risk (difference 15.0; 10.6–19.4, < 0.0001), and 3.0% and 5.2% were labelled low risk (difference −2.2; −4.6 to 0.2, = 0.07) respectively by each protocol. Similar results were found after stratification for menopausal status.

Conclusions  The IOTA protocol was more accurate for triage than the RCOG protocol. The IOTA protocol would avoid major surgery for more women with benign tumours while still appropriately referring more women with an invasive tumour to a gynaecological oncologist.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

Ovarian cancer remains a significant problem with 6500 cases and 4400 deaths in 2008 among UK women.1 In the whole of Europe the figures are 66 700 and 41 900, respectively.2 A significant contributor to prognosis is the quality of the primary surgery, as survival from the disease is related to residual tumour mass after ‘debulking’.3 Less radical treatment options require adequate surgical staging to ensure that women are not under-treated. Consequently, treatment in specialised centres dealing with large numbers of women improves survival in women with ovarian cancer and is more cost-effective.4–8 On the other hand, benign ovarian lesions are more common than carcinoma, and in this group misdiagnosis may lead to unnecessary levels of intervention. Therefore, to optimise care, referring the right women to specialist centres is crucial. This hinges on an accurate preoperative assessment of the likely pathology.

It has been suggested that decisions on how to manage women with an adnexal mass are taken on the basis of the Risk of Malignancy Index (RMI).9–13 For example, guidelines from the Royal College of Obstetricians and Gynaecologists (RCOG) in the UK suggest using the RMI to categorise women with an adnexal mass into three groups: low, moderate and high risk of ovarian cancer.14 For tumours classified as low risk, the proposed management is expectant management or laparoscopic surgery by a generalist in a gynaecology unit. If at moderate risk, laparoscopic surgery in a cancer unit by a surgeon with a special interest is suggested. If at high risk, referral of the woman to a cancer centre for a full staging procedure by a subspecialist gynaecological oncologist is advised.

Recently, the International Ovarian Tumour Analysis (IOTA) group collected a large database of women with an adnexal mass and developed logistic regression models to calculate the risk of malignancy in adnexal masses using clinical information and features derived from ultrasonography.15 Two logistic regression models (LR1 containing twelve variables and LR2 containing six variables) were developed and validated internally (IOTA phase 1, n = 1066). These models then underwent temporal validation (IOTA phase 1b, n = 507) and external validation (IOTA phase 2, n = 1938) with excellent performance,16,17 also in comparison with RMI and other models from the literature as evaluated using the area under the receiver operating characteristics curve.18 On external validation in 12 centres LR2 and RMI had areas under the curve of 0.95 and 0.91, respectively, and the advantage of LR2 over RMI in terms of area under the curve was larger for premenopausal women than for postmenopausal women.18

The aim of this study is to compare the RCOG protocol with a protocol based on the IOTA logistic regression model LR2 with respect to the classification of adnexal masses as being at low, moderate or high risk of malignancy. The key objective is to investigate whether the IOTA protocol improves the selection of women referred for potentially radical surgery by labelling more benign tumours as low risk, more invasive tumours as high risk, and classifying fewer tumours as moderate risk. A secondary objective is the comparison of the two protocols for specific subgroups of women. The most important subgroups are premenopausal and postmenopausal women.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

Design and setting

This is a multicentre observational diagnostic study to evaluate two triaging protocols on women presenting with an adnexal (ovarian, para-ovarian, or tubal) mass who later underwent surgery. We used the data from phase 2 of the IOTA study, which have also been used in previous publications.17–20 Patients were recruited between November 2005 and October 2007 in 11 oncology referral centres, three referral centres for ultrasonography, and five general hospitals in eight countries (Belgium, Sweden, Italy, UK, Czech Republic, Poland, China and Canada). The centres are listed in Table 1. The principal investigators were gynaecologists or radiologists specialised in gynaecological ultrasonography and with a special interest in adnexal masses. The research protocols were ratified by the local Ethics Committee at each recruitment centre.

Table 1.   Pathological tumour diagnosis stratified by recruitment centre
Recruitment centreBenign n (%)Borderline n (%)Invasive n (%)Total n
Oncology referral centres
University Hospitals K.U.Leuven, Belgium155 (62) 24 73 252
Ospedale San Gerardo, Universita di Milano-Bicocca, Monza, Italy199 (79) 17 35 251
Medical University Lublin, Poland101 (66)  3 50 154
Universita Cattolica del Sacro Cuore, Rome, Italy54 (44) 11 57 122
Istituto Europeo di Oncologia, Milan, Italy41 (44) 10 43  94
General Faculty Hospital, Charles University, Prague, Czech Rep.39 (43) 15 36  90
Chinese PLA General Hospital, Beijing, China57 (78)  1 15  73
King’s College Hospital, London, UK40 (62)  9 16  65
Skåne University Hospital Lund, Lund University, Sweden31 (82)  1  6  38
Universita degli Studi di Udine, Italy10 (59)  0  7  17
Istituto Nazionale dei Tumori, Fondazione Pascale, Naples, Italy4 (44)  0  5   9
Referral centres for ultrasonography
University of Bologna, Italy124 (92)  3  8 135
DCS Sacco University of Milan, Italy45 (90)  0  5  50
Universita degli Studi di Napoli, Naples, Italy51 (80)  2 11  64
General hospitals
Ziekenhuis Oost-Limburg, Genk, Belgium173 (87)  5 22 200
Ospedale San Giovanni di Dio, Cagliari, Italy134 (87)  3 17 154
Skåne University Hospital Malmö, Lund University, Sweden110 (80)  6 21 137
Macedonia Melloni Hospital, University of Milan, Italy17 (81)  1  3  21
St. Joseph’s Hospital, McMaster University, Hamilton, Canada11 (92)  0  1  12
Summary
Oncology referral centres731 (63) 91 (8)343 (29)1165
Referral centres for ultrasonography220 (88)  5 (2) 24 (10) 249
General hospitals445 (85) 15 (3) 64 (12) 524
Total1396 (72)111 (6)431 (22)1938

Participants

Women presenting to one of the recruitment centres with at least one persistent adnexal mass that was selected for surgical intervention by the managing clinicians were eligible for inclusion conditional on oral informed consent before the ultrasound scan and surgery. In the event of multiple masses, the mass with the most complex ultrasound morphology was used to collect information on tumour characteristics for statistical analysis. When masses with similar morphology were observed we included the larger of the two masses or the one most easily visible by ultrasonography. Exclusion criteria were pregnancy, refusal of transvaginal ultrasonography, and surgical removal of the mass more than 120 days after the ultrasound examination.

Data collection

A dedicated, secure data collection system was developed for the study (IOTA 2 study screen; astraia GmbH, Munich, Germany). An automatically generated unique identifier was made for each woman’s record. Clinicians could only view or update the records from their own centre. Data security was ensured by encrypting all data communication. Client-side checks in the astraia system and manual checks by one biostatistician and two experienced ultrasound examiners were used to ensure data integrity and completeness.

Immediately before the ultrasound examination a standardised history was taken, including the patient’s age and menopausal status, information on personal history of ovarian and breast cancer, number of first-degree relatives with ovarian or breast cancer, current hormonal therapy, and previous gynaecological surgery. Women aged 50 years or more who had undergone hysterectomy were defined as postmenopausal. A standardised approach was used to carry out transvaginal ultrasonography in all the women. The examination technique and the ultrasound terms and definitions used to describe the ultrasound findings have been described elsewhere.21 In the event that a large mass could not be seen in its entirety using a transvaginal probe, transabdominal ultrasonography was used. All centres used high-quality ultrasound equipment with sensitive colour Doppler functions. Information on more than 40 grey scale and Doppler ultrasound variables was collected to characterise each adnexal mass.

The participating centres were encouraged to measure the level of serum CA125 in peripheral blood from all women in the study, but the availability of CA125 results was not a requirement for inclusion in the IOTA study. The decision to measure CA125 or not was a reflection of routine clinical practice in the different centres: some centres measured serum CA125 levels in every woman whereas others did not. Second-generation immunoradiometric assay kits for CA-125 II22 from the following companies were used: Roche Diagnostics, Basel, Switzerland; Centocor, Malvern, PA, USA; Cis-Bio, Gif-sur-Yvette, France; Abbott Laboratories Diagnostic Division, Abbott Park, IL, USA; Bayer Diagnostics, Tarrytown, NY, USA; bioMérieux, Marcy l’Etoile, France. All kits used the OC 125 antibody. Serum CA125 levels are expressed in units per millilitre (U/ml).

Reference standard: pathological tumour outcome

In this study we focus on the pathological categorisation of removed tissues as benign, borderline or invasive. Surgery was performed by laparoscopy or laparotomy according to the surgeon’s judgment, and the subsequent tissue examination was performed at the local centre by a dedicated gynaecological pathologist. In case of a borderline or invasive tumour, surgical stage was recorded according to the criteria recommended by the International Federation of Gynaecology and Obstetrics.23 We evaluate borderline and invasive tumours separately. In many studies, including those in which RMI and LR2 were developed,9,15 borderline and invasive tumours are combined into a single group of malignant tumours for statistical analysis because borderline tumours require similar preoperative, perioperative and postoperative precautions as invasive tumours.

The triaging protocols

The RCOG guideline number 3414 suggests management of adnexal masses using the RMI.9,24 The RMI contains one clinical variable, five ultrasound variables and one biochemistry variable. It is obtained as the product of the menopausal status score M, the ultrasound score U, and the serum CA125 level in U/ml: M*U*CA-125. M has value 3 if the woman is postmenopausal and 1 if the woman is premenopausal. U has score 0, 1 or 3 depending on whether none, one or more than one of the following five ultrasound characteristics are present: multilocular cyst, evidence of solid components, presence of ascites, bilateral masses and presence of metastases. If the RMI is below 25 the guideline recommends considering the woman as at low risk of malignancy, if the RMI is between 25 and 250 the guideline assumes moderate risk, and if the RMI is above 250 high risk is assumed. According to Davies et al.,24 the likelihood of having a malignant tumour in the low, moderate and high risk groups is 2%, 21%, and 75%, respectively.

The IOTA protocol is based on the logistic regression model LR2.15 The data that were used to develop LR2 were collected between 1999 and 2002 in nine centres (IOTA phase 1), seven of which also participated in IOTA phase 2. LR2 contains one clinical variable (a), four grey scale ultrasound variables (b, c, d, e) and one Doppler ultrasound variable (f): (a) patient age (years), (b) maximal diameter of the solid components (mm), (c) presence of ascites (yes = 1, no = 0), (d) irregular internal cyst walls (yes = 1, no = 0), (e) presence of acoustic shadows (yes = 1, no = 0) and (f) the presence of papillary structures with detectable flow (yes = 1, no = 0). The risk of malignancy is derived as 1/(1 + ez), with z  =  −5.3718  +  0.0354 * (a)  +  0.0697 * (b)  +  1.6159 * (c)  +  0.9586 * (d)  − 2.9486 * (e) + 1.1768 * (f). A priori, we determined the risk thresholds for triaging women as low, moderate or high risk to be 0.05 and 0.25, i.e. 5% (1 in 20) and 25% (1 in 4) estimated risk of malignancy according to LR2: a risk below 0.05 corresponds to low risk; a risk of at least 0.05 but less than 0.25 corresponds to moderate risk; a risk of 0.25 or higher corresponds to high risk.

Statistical analysis

All statistical analyses were performed using sas version 9.2 (SAS Institute, Cary, NC, USA).

Patients were cross-tabulated based on the true outcome (benign, borderline, invasive) and the assigned risk (low, moderate, high), and the percentages of women classified in each risk group were calculated for each tumour type to assess the impact of the two protocols. Another index of a protocol’s impact is the percentage of all women classified as moderate risk and therefore not receiving a clear diagnosis. Differences in paired proportions were analysed using 95% confidence intervals obtained by the standard asymptotic method and the McNemar test. We compared the protocols by assessing whether the IOTA protocol reclassified women to a more appropriate risk group than the RCOG protocol using a measure called the Net Reclassification Improvement.25 For benign and invasive tumours the measure computes the proportion of tumours that are reclassified to a more appropriate risk group minus the proportion of tumours reclassified to less appropriate risk groups to give the net percentage of tumours with improved reclassification.

The serum CA125 level was missing in 22% of the women, and was more often missing in women with a benign tumour (28%) than in women with a borderline or invasive tumour (10%). The missing values are highly likely to have occurred for two reasons. First, different management practices caused some centres to be more committed than others to measure CA125. Second, investigators sometimes did not measure CA125 given the overall clinical picture and the ultrasound appearance of the tumour. Rather than discarding cases with a missing value, which would probably result in a biased patient sample, we handled the missing values using multiple imputation.26,27 In this approach, the missing CA125 values are estimated (i.e. imputed) multiple times to acknowledge the uncertainty in the imputed values. To estimate the missing values, we used predictive mean matching regression28 using variables that were related to either the level of CA125 or the unavailability of CA125 (this is a binary indicator with value 1 if CA125 is missing and 0 otherwise). We generated 100 imputations of the missing values such that 100 completed datasets were obtained. The RCOG protocol, which relies on the CA125 level, was analysed on each of the 100 datasets generated by imputation, and the results were averaged. For cross-tabulations of women with respect to true outcome and risk group assignment, results were rounded to integer values while respecting the (fixed) number of women in each true outcome category. P-values for the McNemar test for each completed dataset were combined using the method from Li et al.29

One of the variables of the RMI is the presence of metastases. This was not a mandatory variable in the IOTA study, and therefore information on metastases was sometimes missing (23%). To address this, we analysed the ability of the RMI to discriminate between benign and malignant tumours when 1) the variable metastases was not used, 2) the variable was imputed by 0 in case of missing information, and 3) multiple imputation was used. The resulting receiver operating characteristics curves were virtually identical for all three approaches. We decided to impute the variable by 0 in case of missing information, because it is likely that in most cases where information on metastases was missing there were no ultrasound signs of metastases.

In addition to reporting the results for the whole study population we report results for specific subgroups. First, the classification performance of the protocols is examined for invasive tumours of varying histology. Next, results for all tumours are examined for the following subgroups: premenopausal women, postmenopausal women, women with available CA125 information, women with information on both the CA125 level and presence of metastases, women from oncology referral centres, women from referral centres for ultrasonography, and women from general hospitals.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

There were 1970 women enrolled in the study (see Figure 1 for a flow diagram). Thirty-two women were excluded for the following reasons: 15 did not undergo surgical removal of the mass within 120 days after the ultrasound examination, 12 were pregnant at the time of the examination, four were excluded because of errors in data entry, and one was excluded because of a protocol violation. The analysis dataset contained the remaining 1938 women, of whom 1396 had benign (72%), 111 borderline (6%), and 431 invasive (22%) tumours. Tables 1–3 describe the pathological and demographic details of the women. Table 1 presents the distribution of the outcome (benign, borderline, invasive) for the total sample and for different centres. Oncology referral centres had more borderline (8%) and invasive (29%) tumours than general hospitals or referral centres for ultrasonography (2–3% borderline tumours and 10–12% invasive tumours). Table 2 summarises the specific histology of the masses in the dataset. The most common benign tumours were endometriomas, serous cystadenomas and teratomas. Borderline tumours were almost all stage I tumours, whereas primary invasive tumours were most often stage III. Table 3 presents the demographic characteristics, CA125 level, RMI, and risk of malignancy estimated by logistic regression model LR2.

image

Figure 1.  Flow diagram of the RCOG and IOTA protocols as triage systems for women with adnexal masses.

Download figure to PowerPoint

Table 2.   Specific pathological tumour diagnoses
Pathological diagnosisn (%)
Benign1396 (72.0)
Endometrioma400 (20.6)
Serous cystadenoma236 (12.2)
Teratoma226 (11.7)
Mucinous cystadenoma138 (7.1)
Simple cyst or parasalpingeal cyst131 (6.8)
Fibroma86 (4.4)
Functional cyst77 (4.0)
Hydrosalpinx or salpingitis49 (2.5)
Abscess24 (1.2)
Rare benign18 (0.9)
Peritoneal pseudocyst11 (0.6)
Borderline111 (5.7)
Stage I99 (5.1)
Stage II3 (0.2)
Stage III8 (0.4)
Stage IV1 (0.1)
Invasive431 (22.2)
Primary invase Stage I70 (3.6)
Primary invase Stage II30 (1.6)
Primary invase Stage III202 (10.4)
Primary invase Stage IV30 (1.6)
Rare primary invasive41 (2.1)
Metastatic invasive58 (3.0)
Table 3.   Demographic characteristics, CA125 level, RMI, and the risk of malignancy estimated by logistic regression model LR2 for the women in the study stratified by tumour type
VariableStatisticsBenign (n = 1396)Borderline (n = 111)Invasive (n = 431)
  1. IQR, interquartile range.

Age in yearsMedian (IQR)41 (31–52)48 (35–64)57 (50–66)
Postmenopausal, %n (%)382 (27)49 (44)311 (72)
Nulliparous, %n (%)651 (47)44 (40)98 (23)
CA125 in U/mLMedian (IQR)18 (11–44)33 (18–91)218 (58–772)
Risk of Malignancy Index (RMI)Median (IQR)12 (0–47)95 (41–312)1232 (214–5423)
LR2 estimated risk of malignancyMedian (IQR)2.3% (1.3–5.8)30.7% (8.4–74.4)69.8% (48.8–87.0)

Table 4 presents the cross-tabulation of the outcome (benign, borderline, invasive) and the risk group assignments for the RCOG and IOTA triage protocols. The IOTA protocol classified fewer women as moderate risk: 17.8% versus 31.4% for the RCOG protocol (difference − 13.6%, 95% CI− 16.1 to − 11.1; P < 0.0001). The Net Reclassification Improvement measure demonstrated improved reclassification of benign and invasive tumours by the IOTA protocol relative to the RCOG protocol: the net percentages of tumours with improved reclassification were 5.8% (95% CI 2.6–9.0; P = 0.0004) and 15.5% (95% CI 10.6–20.4; < 0.0001), respectively. Benign tumours were more often classified as low risk by the IOTA protocol (71.1% versus 62.1%; difference 9.0%, 95% CI 6.2–11.9; P  <  0.0001). Invasive tumours were more often classified as high risk by the IOTA protocol (88.6% versus 73.6%; difference 15.0%, 95% CI 10.6–19.4, < 0.0001), and less often as low risk (3.0% versus 5.2%; difference − 2.2%, 95% CI − 4.6 to 0.2; = 0.07). On the other hand, the IOTA protocol classified more benign tumours as high risk (8.8% versus 5.3%; difference 3.5%, 95% CI 1.75.2; = 0.0001). With respect to borderline tumours, the difference between the protocols was that the IOTA protocol classified more tumours as high risk whereas the RCOG protocol classified more tumours as moderate risk. Of the tumours classified as low, moderate and high risk by the RCOG protocol, 2.5%, 15.0% and 75.0% were invasive tumours, respectively. The corresponding figures for the IOTA protocol were 1.3%, 10.4% and 67.1%. We can also summarise Table 4 in classical terms of sensitivity and specificity for malignancy. Using an RMI cut-off of 25 and an LR2 cut-off of 0.05 to predict malignancy (i.e. low risk versus moderate/high risk), the RCOG protocol achieved a sensitivity of 92.8% and a specificity of 62.1% whereas the IOTA protocol achieved 94.3% sensitivity and 71.1% specificity.

Table 4.   Triage results for women when using the RCOG protocol or the IOTA protocol depending on tumour type
Triage resultType of tumour
All women n (%)Benign n (%)Borderline n (%)Invasive n (%)
RCOG protocol
Low risk906 (46.7)867 (62.1)17 (14.9)22 (5.2)
Moderate risk609 (31.4)454 (32.6)63 (57.0)92 (21.2)
High risk423 (21.8)75 (5.3)31 (28.1)317 (73.6)
IOTA protocol
Low risk1024 (52.8)993 (71.1)18 (16.2)13 (3.0)
Moderate risk345 (17.8)280 (20.1)29 (26.1)36 (8.4)
High risk569 (29.4)123 (8.8)64 (57.7)382 (88.6)

The RCOG and IOTA triage protocols disagreed in risk assignment for 662 women (34.2% of the total sample). In this group, the IOTA protocol classified 53% of benign tumours as low risk and 74% of invasive tumours as high risk, compared with 27% and 16% for the RCOG protocol.

Table 5 presents the results for women with an invasive tumour after stratification for tumour histology. For all histological types of primary invasive ovarian cancer except ‘epithelial not otherwise specified tumours’ the IOTA protocol classified more invasive tumours as high risk than the RCOG protocol. Irrespective of the specific histology, the IOTA protocol classified between 70% and 100% of the invasive tumours as high risk, whereas the RCOG protocol classified around 50% of the mucinous, non-epithelial primary invasive, and metastatic invasive tumours as high risk. The higher number of invasive tumours classified as low risk by the RCOG protocol (Table 4) is explained mainly by the misclassification of non-epithelial primary invasive and metastatic invasive tumours.

Table 5.   Triage results for women with an invasive tumour when using the RCOG protocol or IOTA protocol depending on tumour histology
Triage resultPrimary invasive histologiesMetastatic invasive
Serous n (%)Mucinous n (%)Endometrioid n (%)Clear cell n (%)Epithelial NOS n (%)Non-epithelial n (%) 
  1. NOS, not otherwise specified.

RCOG protocol
Low risk4 (1.9)1 (3.6)0 (0)0 (0)2 (14.3)9 (20.0)6 (10.5)
Moderate risk22 (10.3)12 (43.4)12 (27.2)7 (27.2)2 (14.3)17 (36.7)20 (33.4)
High risk190 (87.8)15 (53.1)33 (72.8)17 (72.8)10 (71.4)20 (43.3)32 (56.2)
IOTA protocol
Low risk3 (1.4)1 (3.6)0 (0)0 (0)2 (14.3)4 (8.7)3 (5.2)
Moderate risk11 (5.1)4 (14.3)3 (6.7)0 (0)2 (14.3)9 (19.6)7 (12.1)
High risk202 (93.5)23 (82.1)42 (93.3)24 (100)10 (71.4)33 (71.7)48 (82.8)
Total216284524144658

A summary of results for all women and for the subgroups is presented in Table 6. The difference in performance between the two protocols was similar for all subgroups, including the subgroup of women with available CA125 level (= 1504, 77.6% of all tumours) and the subgroup of women with available information on the CA125 level and on the presence of metastases (n = 1147, 59.2% of all tumours). The only exception was that in general hospitals the number of invasive tumours classified as low risk was lower for the RCOG protocol (two out of 64) than for the IOTA protocol (three out of 64), but this difference was based on only one woman.

Table 6.   Classification of tumours by the RCOG and IOTA protocols in all women and in various subgroups
 Percentage of all tumours classified as moderate riskPercentage of benign tumours classified as low riskPercentage of benign tumours classified as high riskPercentage of invasive tumours classified as low riskPercentage of invasive tumours classified as high risk
All women (n = 1938)
RCOG31.462.15.35.273.6
IOTA17.877.18.83.088.6
Premenopausal women (n = 1196; 61.7%)
RCOG27.571.63.910.156.7
IOTA14.182.15.16.778.3
Postmenopausal women (n = 742; 38.3%)
RCOG37.836.89.23.380.1
IOTA23.742.218.61.692.6
Available CA125 level (n = 1504, 77.6%)
RCOG32.459.66.05.673.3
IOTA18.468.310.43.388.0
Available information on CA125 and metastasis (n = 1147, 59.2%)
RCOG33.656.26.94.774.0
IOTA18.665.911.73.288.2
Oncology referral centres (n = 1165; 60.1%)
RCOG31.559.36.84.875.5
IOTA17.867.611.52.988.0
Referral centres for ultrasonography (n = 249; 12.8%)
RCOG35.462.13.316.747.3
IOTA16.578.24.10100
General hospitals (n = 524; 27.0%)
RCOG29.466.63.93.173.2
IOTA18.573.56.74.787.5

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

This study has shown that the IOTA protocol is more accurate than the RMI-based protocol recommended by the RCOG for classifying adnexal masses as being at low, moderate or high risk of malignancy. The IOTA protocol resulted in a substantial increase in the number of benign ovarian masses classified as low risk and in the number of invasive tumours classified as high risk. It was also associated with a reduction in the number of invasive tumours classified as low risk. The difference between the protocols was similar in all subgroups considered. Further, the IOTA protocol performed fairly consistently in all histological types of invasive tumours, whereas the RCOG protocol showed poorer performance in mucinous, non-epithelial primary invasive, and metastatic invasive tumours. This is likely to be explained by the lower expression of CA125 in these tumour types (data not shown).

The major strengths of this study are that the protocols were applied on a large, multicentre, international database. This means that the results are likely to be robust. Also, the performance of the RMI and the LR2 logistic regression model was evaluated in a way that corresponds to their use in clinical practice instead of—as in many other evaluation studies—relying on the area under the receiver operating charactieristics curve.

Our study also has limitations, one being the missing values for CA125. This issue is appropriately tackled with the technique of multiple imputation,27 and with a comparison of the results based on multiple imputation (Table 4) with those based only on cases with available CA125 levels. It is reassuring that the results based on women with available information were very similar to those obtained after multiple imputation. Another weakness is that information on the presence of metastases was sometimes missing. As described in the Methods section, different strategies to manage the missing values were compared, and this did not affect the conclusion of this study. Results based on women with available CA125 levels as well as information on the presence of metastases (Table 6) were similar to the main results presented in Table 4. A third criticism could be that both the RMI and the LR2 were used in the hands of experienced ultrasound operators with a specific interest in adnexal pathology, and that therefore, the results of this study might not be generalisable to less experienced ultrasound examiners. However, we believe that it should be possible for any qualified ultrasound practitioner to obtain reliable information on all the ultrasound variables required for both models.

According to our results, the IOTA protocol would result in more women appropriately undergoing minimally invasive surgery, whereas most invasive cancers would still be correctly referred to gynaecological oncologists. As a consequence, the use of the IOTA triage system would have a beneficial impact on the management of women with adnexal tumours. Additionally, because the IOTA protocol labelled fewer women as moderate risk, fewer women would receive an uncertain diagnosis. This might be a psychological advantage. The IOTA protocol may, however, lead to an increase in the number of women with a benign tumour being unnecessarily referred to a cancer centre. We believe that this disadvantage is of limited importance relative to the benefits of the IOTA protocol.

The relative performance of the two models with respect to borderline tumours merits a comment. The IOTA protocol classified more borderline tumours as high risk than the RCOG protocol. This implies that with the IOTA protocol more women with borderline tumours would be managed in gynaecological oncology centres. We argue that this is an advantage because, even though there may be more conservative fertility-sparing treatment options in these women, decisions about such treatments are likely to be better handled by subspecialist gynaecological oncologists.

Some may argue that the RMI is a simpler method to use in everyday clinical practice than the LR2, because the RMI can be calculated without the use of a computer. However, nowadays computers are routinely used in most—if not all—hospitals and outpatient clinics, and the LR2 risk can be easily calculated using an excel file. It should also be easy to incorporate the LR2 risk calculation model in a database handling patient data.

LR2 was developed on a large dataset including women from nine clinical centres in five countries. This should make the model generally applicable. LR2 has been shown to have only slightly lower performance than LR1 (LR1 is a model with 12 predictors) both on internal, temporal, and external validation.15–18 Moreover, LR2 and LR1 have been shown to perform as well as advanced algorithms based on neural networks and support vector machines.16,30 An interesting issue is whether adding information on CA125 to the LR2 model would improve patient triage. We therefore developed a model where CA125 was added to the six predictor variables used in LR2, using the same data as those on which LR2 was developed.15 The triaging protocol based on this model (labelled the IOTA + CA125 protocol) is based on the same risk thresholds as the IOTA protocol. Triage results of the IOTA + CA125 protocol are shown in the Table S1. When comparing the IOTA protocol with the IOTA + CA125 protocol using the Net Reclassification Improvement measure, the addition of CA125 slightly improved triage of invasive tumours (P = 0.12) but also slightly deteriorated triage of benign tumours (P = 0.44).

Conclusion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

In summary, we have shown that a management protocol based on triaging women using the IOTA logistic regression model LR2 performs significantly better than the RMI-based protocol that is currently proposed by the RCOG. LR2 has been developed on a large database of adnexal masses and undergone external validation in several different centres. We believe that the LR2 model should be considered as an alternative to the RMI for inclusion in triaging protocols for adnexal pathology.

Contribution to authorship

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

BVC, DT, LV, AM and TB designed the study. DT, LV, ACT and IV were involved in data acquisition. BVC performed the statistical analysis. BVC, DT, LV, AM, SGM and TB interpreted the results and BVC and TB drafted the manuscript. BVC, DT, LV, AM, SGM, ACT, IV, TB revised the manuscript and approved the final version of the manuscript.

Details of ethics approval

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

The University Hospitals K.U.Leuven is the coordinating centre. The protocol of IOTA phase 2 was approved by Ethics Committee of the University Hospitals K.U.Leuven (‘Commissie Medische Ethiek’) on 23 November 2005 with reference number ML1248. The research protocol was further ratified by the local Ethics Committee at each recruitment centre.

Funding

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

This work was supported by the Research Foundation—Flanders (FWO) (grants 1251609N, 1251612N, G049312N), Flanders’ Agency for Innovation by Science and Technology (IWT) (grant IWT-TBM070706-IOTA3); Swedish Medical Research Council: grants numbers K2001-72X 11605-06A, K2002-72X-11605-07B, K2004-73X-11605-09A and K2006-73X-11605-11-3; funds administered by Malmö University Hospital; and two Swedish governmental grants: ALF-medel and Landstingsfinansierad Regional Forskning. Tom Bourne is supported by Imperial Healthcare NHS Trust NIHR Biomedical Research Centre.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

B. Van Calster is a postdoctoral fellow of the Research Foundation–Flanders (FWO). We acknowledge the free use of a dedicated study screen developed by astraia GMBH. The Steering Committee of the International Ovarian Tumor Analysis (IOTA) study group consists of Dirk Timmerman (University Hospitals K.U.Leuven, Leuven, Belgium), Lil Valentin (Skåne University Hospital Malmö, Lund University, Sweden), Tom Bourne (Imperial College Hammersmith Campus, London, UK), Antonia C Testa (Università Cattolica di Sacre Cuore, Rome, Italy), Sabine Van Huffel (Katholieke Universiteit Leuven, Leuven, Belgium) Ignace Vergote (University Hospitals K.U.Leuven, Leuven, Belgium). The IOTA principal investigators are (in alphabetical order) Jean-Pierre Bernard (Maurepas, France), Artur Czekierdowski (Lublin, Poland), Elisabeth Epstein (Lund and Stockholm, Sweden), Enrico Ferrazzi (Milano, Italy), Daniela Fischerová (Prague, Czech Republic), Dorella Franchi (Milano, Italy), Robert Fruscio (Monza, Italy), Stefano Greggi (Napoli, Italy), Stefano Guerriero (Cagliari, Italy), Jingzhang (Beijing, China), Davor Jurkovic (London, UK), Fabrice Lécuru (Paris, France), Francesco PG Leone (Milano, Italy), Andrea A Lissoni (Monza, Italy), Ulrike Metzger (Paris, France), Henry Muggah (Hamilton, ON, Canada), Dario Paladini (Napoli, Italy), Alberto Rossi (Udine, Italy), Luca Savelli (Bologna, Italy), Antonia C Testa (Roma, Italy), Dirk Timmerman (Leuven, Belgium), Diego Trio (Milano, Italy), Lil Valentin (Malmö, Sweden), Caroline Van Holsbeke (Genk, Belgium), Gerardo Zanetta [deceased] (Monza, Italy).

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information
  • 1
    Cancer Research UK. Ovarian cancer statistics – UK. 2011. [Available at: http://info.cancerresearchuk.org/cancerstats/types/ovary/]. Last accessed 13 February 2012.
  • 2
    Ferlay J, Parkin DM, Steliarova-Foucher E. Estimates of cancer incidence and mortality in Europe in 2008. Eur J Cancer 2010;46:76581.
  • 3
    Bristow RE, Tomacruz RS, Armstrong DK, Trimble EL, Montz FJ. Survival effect of maximal cytoreductive surgery for advanced ovarian carcinoma during the platinum era: a meta-analysis. J Clin Oncol 2002;20:124859.
  • 4
    Greving JP, Vernooij F, Heintz APM, van der Graaf Y, Buskens E. Is centralization of ovarian cancer care warranted? A cost-effectiveness analysis Gynecol Oncol 2009;113:6874.
  • 5
    Vergote I, De Brabanter J, Fyles A, Bertelsen K, Einhorn N, Sevelda P, et al. Prognostic importance of degree of differentiation and cyst rupture in stage I invasive epithelial ovarian carcinoma. Lancet 2001;357:17682.
  • 6
    Bristow RE, Berek JS. Surgery for ovarian cancer: how to improve survival. Lancet 2006;367:155860.
  • 7
    Earle CC, Schrag D, Neville BA, Yabroff KR, Topor M, Fahey A, et al. Effect of surgeon specialty on process of care and outcomes for ovarian cancer patients. J Natl Cancer Inst 2006;98:17280.
  • 8
    Engelen MJA, Kos HE, Willemse PHB, Aalders JG, de Vries EGE, Schaapveld M, et al. Surgery by consultant gynecologic oncologists improves survival in patients with ovarian carcinoma. Cancer 2006;106:58998.
  • 9
    Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas JG. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol 1990;97:9229.
  • 10
    Geomini P, Kruitwagen R, Bremer GL, Cnossen J, Mol BWJ. The accuracy of risk scores in predicting ovarian malignancy. A systematic review. Obstet Gynecol 2009;113:38494.
  • 11
    Clarke SE, Grimshaw R, Rittenberg P, Kieser K, Bentley J. Risk of malignancy index in the evaluation of patients with adnexal masses. J Obstet Gynaecol Can 2009;31:4405.
  • 12
    Bailey J, Tailor A, Naik R, Lopes A, Godfrey K, Hatem HM, et al. Risk of malignancy index for referral of ovarian cancer cases to a tertiary center: does it identify the correct cases? Int J Gynecol Cancer 2006;16(Suppl. 1):304.
  • 13
    National Collaborating Centre for Cancer. Recognition and initial management of ovarian cancer: NICE clinical guideline 122. London, UK: National Institute for Health and Clinical Excellence, 2011, Available from: http://www.nice.org.uk/CG122. Last accessed 13 February 2012.
  • 14
    Rufford BD, Jacobs IJ. Green-top Guideline No. 34. Ovarian cysts in postmenopausal women. London, UK: Royal College of Obstetricians and Gynaecologists, 2003, Available from: http://www.rcog.org.uk/files/rcog-corp/GTG3411022011.pdf. Last accessed 13 February 2012.
  • 15
    Timmerman D, Testa AC, Bourne T, Ferrazzi E, Ameye L, Konstantinovic ML, et al. A logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis (IOTA) group. J Clin Oncol 2005;23:8794801.
  • 16
    Van Holsbeke C, Van Calster B, Testa AC, Domali E, Lu C, Van Huffel S, et al. Predicting malignancy of ovarian tumors: prospective evaluation of models from the IOTA study. Clin Cancer Res 2009;15:68491.
  • 17
    Timmerman D, Van Calster B, Testa AC, Guerriero S, Fischerova D, Lissoni AA, et al. Ovarian cancer prediction by logistic regression models: a prospective evaluation of diagnostic accuracy. Ultrasound Obstet Gynecol 2010;36:22634.
  • 18
    Van Holsbeke C, Van Calster B, Bourne T, Ajossa S, Testa AC, Guerriero S, et al. External validation of diagnostic models to estimate the risk of malignancy in adnexal masses. Clin Cancer Res 2012;18:815825.
  • 19
    Van Calster B, Valentin L, Van Holsbeke C, Testa AC, Bourne T, Van Huffel S, et al. Polytomous diagnosis of ovarian tumors as benign, borderline, primary invasive or metastatic: development and validation of standard and kernel-based risk prediction models. BMC Med Res Methodol 2010;10:96.
  • 20
    Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, et al. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ 2011;341:c6839.
  • 21
    Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the International Ovarian Tumor Analysis (IOTA) group. Ultrasound Obstet Gynecol 2000;16:5005.
  • 22
    Kenemans P, van Kamp GJ, Oehr P, Verstraeten RA. Heterologous double-determinant immunoradiometric assay CA 125 II: reliable second-generation immunoassay for determining CA 125 in serum. Clin Chem 1993;39:250913.
  • 23
    Heintz APM, Odicino F, Maisonneuve P, Quinn MA, Benedet JL, Creasman WT, et al. Carcinoma of the ovary. Int J Gynaecol Obstet 2006;95(Suppl. 1):S16192.
  • 24
    Davies AP, Jacobs I, Woolas R, Fish A, Oram D. The adnexal mass: benign or malignant? Evaluation of a risk of malignancy index Br J Obstet Gynaecol 1993;100:92731.
  • 25
    Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under ROC curve to reclassification and beyond. Stat Med 2008;27:15772.
  • 26
    Little JR, Rubin D. Statistical analysis with missing data, 2nd edn. New York, NY: Wiley; 2002.
  • 27
    Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ 2009;338:b2393.
  • 28
    Schenker N, Taylor JMG. Partially parametric techniques for multiple imputation. Comput Stat Data Anal 1996;22:42546.
  • 29
    Li K-H, Meng X-L, Raghunathan TE, Rubin DB. Significance levels from repeated p-values with multiply-imputed data. Stat Sin 1991;1:6592.
  • 30
    Van Calster B, Timmerman D, Lu C, Suykens JAK, Valentin L, Van Holsbeke C, et al. Preoperative diagnosis of ovarian tumors using Bayesian kernel-based methods. Ultrasound Obstet Gynecol 2007;29:496504.

Supporting Information

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Disclosure of interest
  9. Contribution to authorship
  10. Details of ethics approval
  11. Funding
  12. Acknowledgements
  13. References
  14. Supporting Information

Table S1. Triage results for patients when using the RCOG protocol, the IOTA protocol, and the IOTA+CA125 protocol.

FilenameFormatSizeDescription
BJO_3297_sm_TableS1.pdf50KSupporting info item

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.