SEARCH

SEARCH BY CITATION

Keywords:

  • adnexal masses;
  • training;
  • ultrasound

ABSTRACT

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. REFERENCES
  8. Supporting Information

Objective

To assess the feasibility of a specific training program for ultrasound diagnosis of adnexal masses.

Methods

A 2-month intensive training program was developed. The program protocol consisted of a 1-day intensive theoretical course focused on clinical and sonographic issues related to adnexal masses and ovarian cancer, followed by a 4-week real-time ultrasound training program in a tertiary center (25–30 adnexal masses evaluated per month) and a final 4-week period for offline assessment of three-dimensional (3D) volumes from adnexal masses. In this final period, each trainee evaluated five sets of 100 3D volumes. 3D volumes contained gray-scale and power Doppler information, and the trainee was provided with clinical data for each case (patient age, menopausal status and reported symptoms). 3D volumes were obtained from surgically removed masses that had undergone histological diagnosis or from masses that had been followed up until resolution. After assessment of each set, the trainee's diagnostic performance was calculated (sensitivity and specificity) and each incorrectly classified mass was evaluated with the trainer. The objective was to achieve a sensitivity of > 95% and a specificity of > 90%. Learning curve cumulative summation (LC-CUSUM) graphs were plotted to assess the learning curve for the trainees.

Results

One trainer and two trainees with little experience in gynecological ultrasound (one gynecologist and one radiologist) participated in this study. LC-CUSUM graphs showed that competence was achieved after 170 or 185 examinations. The objectives for diagnostic performance were achieved after assessment of the second set of 3D volumes (200 cases) for each trainee.

Conclusions

The proposed training program appears to be feasible. High diagnostic performance can be achieved after analysis of 200 cases and maintained thereafter. Copyright © 2013 ISUOG. Published by John Wiley & Sons Ltd.


INTRODUCTION

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. REFERENCES
  8. Supporting Information

Accurate diagnosis of adnexal masses is crucial for their adequate management[1]. Ultrasound is currently considered as the first-line imaging technique for discriminating between benign and malignant adnexal masses[2]. However, this technique is highly dependent on the expertise of the examiner[3], with previous studies showing that reproducibility, diagnostic performance and examiner's confidence in providing a diagnosis are poorer in non-expert examiners[4-9].

Several scoring systems and logistic regression models have been developed in an attempt to overcome this limitation of ultrasound[10-12]. However, evidence shows that these approaches are not superior to an expert examiner[13] and that they are not useful when the examiner is uncertain about the nature of the mass[14].

These facts highlight the relevance of adequate training for assessing adnexal masses by ultrasound. Several programs, including theoretical and practical aspects for ultrasound training in obstetrics and gynecology, have been proposed[15-18]. However, to the best of our knowledge, no specific training program for assessing adnexal masses by ultrasound has been previously proposed.

The use of simulators for basic gynecological ultrasound training has been proposed[19, 20]. However, training in ultrasound assessment of adnexal masses is not feasible using these simulators and real-time ultrasound training is needed. It is important to note, however, that real-time ultrasound training may be time-consuming, depending on the workflow of masses evaluated at a given institution or ultrasound laboratory.

Three-dimensional (3D) ultrasound has become increasingly available in clinical practice. This technique allows the acquisition and storage of 3D volumes of structures of interest that can subsequently be assessed offline[21], and this assessment is reproducible between observers[22]. Indeed, we have shown that diagnostic accuracy for discriminating benign from malignant adnexal masses using offline assessment of 3D volumes is similar to that of real-time ultrasound[23].

We hypothesized that the combined use of real-time ultrasound and evaluation of offline 3D volumes could constitute adequate training for ultrasound assessment of adnexal masses over a relatively short time period. For this reason we have developed a specific training program incorporating these two methods. The aim of this study was to assess the feasibility of the program and its preliminary results.

METHODS

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. REFERENCES
  8. Supporting Information

The training program was developed at the Department of Obstetrics and Gynecology, Clinica Universidad de Navarra, University of Navarra, Pamplona, Spain. The program design was presented to the Department's Educational Committee who evaluated and approved the start of the program. Institutional Review Board approval was also obtained.

The objective of this program was to train examiners with no or very little experience in ultrasound assessment of adnexal masses. Given that expert examiners have shown a consistently high diagnostic performance in terms of sensitivity (90–98%) and specificity (80–94%)[23-28], the objective was that trainees should achieve a high diagnostic performance with sensitivity > 95% and specificity > 90%.

The program consisted of three phases (Figure 1). Phase 1 consisted of a 1-day (8 h) theoretical course addressing clinical and ultrasound issues related to adnexal masses. It included short lectures on the epidemiological aspects of ovarian cancer and adnexal masses, the surgical management of adnexal masses and ovarian cancer, the principles of gray-scale ultrasound and Doppler and the use of ultrasound for assessing adnexal masses, mainly focused on the use of pattern recognition (Appendices S1 and S2; and Volumes S1–S5).

image

Figure 1. Training program flow chart. 3D, three-dimensional.

Download figure to PowerPoint

Phase 2 consisted of 4 weeks of training in real-time ultrasound in our tertiary care center, in which 25–30 adnexal masses were evaluated per month. During this phase, training was performed under the direct mentorship and supervision of an expert examiner with more than 20 years' experience in gynecological ultrasound (J.L.A.). In this phase, the trainee was taught to perform transabdominal and transvaginal ultrasound, specifically pelvic ultrasound, including normal pelvic anatomy and uterine and ovarian biometry. Specifically, they were trained in the assessment of adnexal masses by gray-scale and Doppler ultrasound and to look for features specifically related to the dynamic aspects of real-time ultrasound, such as mass mobility or tenderness[29]. They were also taught to apply machine settings in both gray-scale and Doppler imaging.

Phase 3 consisted of a half-day course with specific training on the use of dedicated software (4D View; GE Medical Systems, Zipf, Austria) for assessing 3D volumes, and a further 4 weeks for assessing stored 3D volumes. Trainees were provided with five sets of 100 3D volumes from adnexal masses, along with an Excel file (Microsoft Corporation, Redmond, WA, USA) containing clinical data from each patient (patient age, menopausal status and reported symptoms). The 3D volumes contained both gray-scale and power Doppler information.

All 500 adnexal masses were from patients evaluated and treated at our institution between January 2003 and December 2007. In 434 cases, a definitive histological diagnosis had been obtained after surgical removal of the tumor; in the remaining 66 cases, considered benign, the ovarian cyst was followed up until spontaneous resolution.

For each set of 3D volumes the trainee had to analyze the 3D volume and interpret the images along with the clinical data. Volume analysis was performed by virtual navigation in all three orthogonal planes through the mass and involved measuring structures such as tumor diameter, size of solid components, height of papillary projections, thickness of cyst wall and septations (if present); looking at vessel location, distribution and amount (absent, scanty, moderate or abundant); and analyzing echogenicity of cyst content, looking for acoustic shadowing and analyzing tumor contour (regular or irregular).

The trainee had to provide a diagnosis of benign or malignant based on pattern recognition[30] and the level of confidence in their diagnosis (certainly benign, probably benign, uncertain, probably malignant or certainly malignant). In those cases for which the trainee was uncertain about their diagnosis, they had to provide a diagnosis as benign or malignant in spite of uncertainty.

After finishing each set, the trainee was provided with the final diagnosis and they had to compare their diagnosis with the final diagnosis to determine success or failure. Those cases with an incorrect diagnosis were then reviewed with the trainer in an attempt to identify the reasons for error in the diagnosis and to reinterpret the images.

As stated above, each set of 3D volumes contained 100 cases selected by the trainer. This selection was made according to a single criterion: the prevalence of malignant cases had to be similar in all five sets (set 1, 33%; set 2, 28%; set 3, 35%; set 4, 35%; and set 5, 32%) (Table 1). Videoclips were not used in this training program.

Table 1. Definitive diagnosis of adnexal masses included in each set of ultrasound volumes
Final diagnosisSet 1Set 2Set 3Set 4Set 5
  1. Data are given as n. With the exception of functional cysts (cysts that resolved spontaneously, n = 66), all diagnoses were histologically confirmed.

Functional cyst171615153
Endometrioma2128211216
Dermoid cyst98969
Serous/simple cyst71181316
Mucinous cyst64546
Fibroma22233
Cystadenofibroma21012
Hydrosalpinx21454
Peritoneal cyst10122
Brenner tumor00022
Struma ovarii00012
Granulosa cell tumor00002
Leiomyoma01011
Carcinoma3026333328
Borderline tumor32224
Total100100100100100

The training program was offered to two trainees: an obstetrics and gynecology specialist with experience in transabdominal obstetrical ultrasound but very little experience in gynecological ultrasound (L.D., Trainee A); and a 3rd-year resident in radiology with no experience of gynecological ultrasound (P.F., Trainee B). Training took place between November 2011 and December 2011.

Statistical analysis

To analyze changes in each trainee's diagnostic performance, the sensitivity, specificity and likelihood ratios (LR) with 95% CI were calculated for each consecutive set of 3D volumes, before cases of incorrect diagnosis were reviewed with the trainer.

The learning curve cumulative summation (LC-CUSUM) test was used to assess each trainee's learning curve[31]. Acceptable and unacceptable failure rates were set at 15% and 25%, respectively. These limits were chosen assuming that the pooled failure rate for an expert examiner could be around 15–25%, taking into account both false-positive and false-negative results[23-28]. Type I (α) and type II (β) error rates were set at 0.1. CUSUM values are plotted on the y-axis and the numbers of examinations are plotted on the x-axis. Horizontal lines are plotted at regular intervals on the y-axis, defining h0 and h1 for the spacing between acceptable and unacceptable boundary lines, respectively. The competence is declared when the plot falls below two consecutive boundary lines[32].

To check competence in a real-life setting, both trainees were asked to report the number of cases evaluated by each of them in their respective working places during the 10 months after training, as well as their diagnosis and final outcome (histology or spontaneous resolution). Sensitivity, specificity and LRs for this series were calculated.

RESULTS

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. REFERENCES
  8. Supporting Information

Both trainees completed the program. Sensitivity, specificity and LRs after each set of 3D volumes are shown, for both trainees, in Table 2. LC-CUSUM plots showed that competence was declared after 170 cases for Trainee A and after 185 cases for Trainee B (Figure 2). Changes in trainees' confidence are shown in Figure 3, which demonstrates that confidence increased with training.

Table 2. Diagnostic performance for Trainees A and B following analysis of each set of three-dimensional ultrasound volumes of adnexal masses (100 volumes per set)
Trainee

Sensitivity

(%)

Specificity

(%)

LR+LR–
  1. 95% CI values are shown in parentheses. LR+, positive likelihood ratio; LR–, negative likelihood ratio.

Trainee A
  Set 174 (57–86)83 (72–90)4.38 (2.46–7.81)0.31 (0.17–0.57)
  Set 296 (79–99)92 (84–96)12.59 (5.81–27.31)0.05 (0.01–0.32)
  Set 397 (86–99)95 (88–98)21.73 (7.18–65.76)0.03 (0.004–0.20)
  Set 4100 (90–100)89 (79–95)9.50 (4.45–20.25)
  Set 5100 (89–100)92 (83–97)12.60 (5.43–29.22)
Trainee B
  Set 176 (58–88)88 (78–94)6.45 (3.26–12.76)0.27 (0.14–0.52)
  Set 296 (80–99)94 (86–98)17.01 (6.54–44.23)0.04 (0.01–0.30)
  Set 391 (76–97)89 (80–95)8.57 (4.22–17.41)0.10 (0.03–0.30)
  Set 497 (86–99)84 (72–91)6.05 (3.32–11.03)0.03 (0.005–0.23)
  Set 5100 (89–100)90 (79–95)8.86 (4.41–17.79)
image

Figure 2. Cumulative summation test for the learning curve (LC-CUSUM) graphs for discriminating benign from malignant masses for two trainees. Trainee A reached performance after Case 170. Trainee B reached performance after Case 185. Dotted horizontal lines show acceptable/unacceptable boundary lines of the CUSUM score (at intervals of 3.4).

Download figure to PowerPoint

image

Figure 3. Histograms showing how confidence changed with training for Trainee A (a) and Trainee B (b). Diagnoses stated as certainly benign or malignant increased from the first to the final set from 50% to 87% for Trainee A and from 48% to 80% for Trainee B. image, Certainly benign or malignant; image, probably benign or malignant; image, uncertain.

Download figure to PowerPoint

After training, Trainee A evaluated 102 cases from January 2012 to October 2012, with a final outcome available for 89 masses (eight malignant and 81 benign). Sensitivity was 100% (95% CI, 59.8–100), specificity was 91.3% (95% CI, 82.4–96.2%) and the positive LR (LR+) was 11.6 (95% CI, 5.7–23.5) (the negative LR (LR–) could not be calculated). Trainee B evaluated 74 adnexal masses during the same period, with a final outcome available for 53 cases (seven malignant and 46 benign). Sensitivity was 100% (95% CI, 56–100%), specificity was 89.1% (95% CI, 75.6–95.9%) and the LR + was 9.2 (95% CI, 4.0–21.0) (the LR– could not be calculated).

DISCUSSION

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. REFERENCES
  8. Supporting Information

Expertise is recognized as essential for achieving a good diagnostic performance in assessing adnexal masses by ultrasound. Van Holsbeke et al. reported on the effect of a theoretical course on trainees' competence for discriminating between benign and malignant adnexal masses[6]. They concluded that theoretical ultrasound teaching did not improve the performance of pattern recognition in the hands of trainees and that practical training is likely to be of paramount importance if diagnostic performance is to be optimized. Furthermore, real-time ultrasound is better than two-dimensional (2D) static images for assessing adnexal masses[33].

In this study we have presented the development and testing of a training program specifically designed for training in ultrasound assessment of adnexal masses that includes a theoretical course as well as practical and offline training. The use of offline assessment of 3D volumes for teaching trainees to identify normal anatomy and structural anomalies has been previously proposed in obstetrics[34].

In the case of adnexal masses, according to LC-CUSUM plots, trainee competence can be achieved after 170 examinations. This is in agreement with the figures for diagnostic performance after the second set (i.e. 200 cases) of examinations (sensitivity and specificity of 96% and 92%, respectively, for Trainee A, and 96% and 94%, respectively, for Trainee B). We observed that good diagnostic performance was maintained in the three subsequent sets of 3D volumes. More importantly, when trainees returned to their practices to diagnose cases in a clinical setting, their diagnostic performance was high, indicating that this training program is effective.

Another advantage we believe that this program may offer is the short time required to achieve competence. This program was designed for an 8-week period, whereas training based solely on real-time ultrasound would need much longer to achieve the same level of competence. At our institution, in which 25–30 adnexal masses per month are evaluated, a trainee would need to spend at least 6–7 months assessing masses in order to assess the ˜170 required for competence. With the program we propose that this time would be reduced by two-thirds.

Another interesting finding was that the trainees' level of confidence increased with training. This was to be expected because an increase in diagnostic confidence is inherent to the process of learning[35],[36]. However, we were surprised by the very high level of confidence achieved by both trainees after training. It is likely that this can be explained by factors related to the process of learning, such as the trainees' personal attitudes and motivation, training environment and perceived training quality[37].

Some limitations may be considered for this training program. First, training is mainly based on the offline assessment of static 3D volumes and clinical data with only a short period of real-time ultrasound training. This means that training in the dynamic aspects of real-time ultrasound, such as tenderness, mass mobility and upper abdomen assessment, is limited. Additionally, no specific selection was made for 3D volumes except in order to achieve a similar prevalence of malignancy in each set. Difficult cases were not specifically addressed. It would be of interest to know whether diagnostic performance and confidence is maintained at a high level in more difficult cases assessed after training. One additional limitation is that the study was performed by only one trainer and two trainees and that there was no control group. Therefore, the generalizability needs to be confirmed.

In conclusion, we consider the proposed training program for ultrasound assessment of adnexal masses to be feasible and render good results. It is our belief that a similar program could be implemented for training in ultrasound assessment for other gynecological conditions, such as endometrial or uterine pathology, or in reproductive medicine.

REFERENCES

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. REFERENCES
  8. Supporting Information
  • 1
    Liu JH, Zanotti KM. Management of the adnexal mass. Obstet Gynecol 2011; 117: 14131428.
  • 2
    American College of Obstetricians and Gynecologists. ACOG Practice Bulletin. Management of adnexal masses. Obstet Gynecol 2007; 110: 201214.
  • 3
    Timmerman D, Schwärzler P, Collins WP, Claerhout F, Coenen M, Amant F, Vergote I, Bourne TH. Subjective assessment of adnexal masses with the use of ultrasonography: an analysis of interobserver variability and experience. Ultrasound Obstet Gynecol 1999; 13: 1116.
  • 4
    Guerriero S, Alcazar JL, Pascual MA, Ajossa S, Gerada M, Bargellini R, Virgilio B, Melis GB. Diagnosis of the most frequent benign ovarian cysts: is ultrasonography accurate and reproducible? J Womens Health 2009; 18: 519527.
  • 5
    Guerriero S, Alcazar JL, Pascual MA, Ajossa S, Gerada M, Bargellini R, Virgilio B, Melis GB. Intraobserver and interobserver agreement of grayscale typical ultrasonographic patterns for the diagnosis of ovarian cancer. Ultrasound Med Biol 2008; 34: 17111716.
  • 6
    Van Holsbeke C, Daemen A, Yazbek J, Holland TK, Bourne T, Mesens T, Lannoo L, De Moor B, De Jonge E, Testa AC, Valentin L, Jurkovic D, Timmerman D. Ultrasound methods to distinguish between malignant and benign adnexal masses in the hands of examiners with different levels of experience. Ultrasound Obstet Gynecol 2009; 34: 454461.
  • 7
    Van Holsbeke C, Daemen A, Yazbek J, Holland TK, Bourne T, Mesens T, Lannoo L, Boes AS, Joos A, Van De Vijver A, Roggen N, de Moor B, de Jonge E, Testa AC, Valentin L, Jurkovic D, Timmerman D. Ultrasound experience substantially impacts on diagnostic performance and confidence when adnexal masses are classified using pattern recognition. Gynecol Obstet Invest 2010; 69: 160168.
  • 8
    Guerriero S, Alcazar JL, Pascual MA, Ajossa S, Graupera B, Hereter L, Melis GB. The diagnosis of ovarian cancer: is color Doppler imaging reproducible and accurate in examiners with different degrees of experience? J Womens Health 2011; 20: 273277.
  • 9
    Faschingbauer F, Benz M, Häberle L, Goecke TW, Beckmann MW, Renner S, Müller A, Wittenberg T, Münzenmayer C. Subjective assessment of ovarian masses using pattern recognition: the impact of experience on diagnostic performance and interobserver variability. Arch Gynecol Obstet 2012; 285: 16631669.
  • 10
    Alcázar JL, Mercé LT, Laparte C, Jurado M, López-García G. A new scoring system to differentiate benign from malignant adnexal masses. Am J Obstet Gynecol 2003; 188: 685692.
  • 11
    Ameye L, Valentin L, Testa AC, Van Holsbeke C, Domali E, Van Huffel S, Vergote I, Bourne T, Timmerman D. A scoring system to differentiate malignant from benign masses in specific ultrasound-based subgroups of adnexal tumors. Ultrasound Obstet Gynecol 2009; 33: 92101.
  • 12
    Van Holsbeke C, Van Calster B, Testa AC, Domali E, Lu C, Van Huffel S, Valentin L, Timmerman D. Prospective internal validation of mathematical models to predict malignancy in adnexal masses: results from the international ovarian tumor analysis study. Clin Cancer Res 2009; 15: 684691.
  • 13
    Valentin L, Hagen B, Tingulstad S, Eik-Nes S. Comparison of ‘pattern recognition’ and logistic regression models for discrimination between benign and malignant pelvic masses: a prospective cross validation. Ultrasound Obstet Gynecol 2001; 18: 357365.
  • 14
    Valentin L, Ameye L, Savelli L, Fruscio R, Leone FP, Czekierdowski A, Lissoni AA, Fischerova D, Guerriero S, Van Holsbeke C, Van Huffel S, Timmerman D. Adnexal masses difficult to classify as benign or malignant using subjective assessment of gray-scale and Doppler ultrasound findings: logistic regression models do not help. Ultrasound Obstet Gynecol 2011; 38: 456465.
  • 15
    ISUOG Education Committee. Update on proposed minimum standards for ultrasound training for residents in OB/GYN. Ultrasound Obstet Gynecol 1996; 8: 363365.
  • 16
    Di Renzo GC, Clerici G. Teaching sonography in obstetrics and gynecology. Ann N Y Acad Sci 1998; 847: 228232.
  • 17
    Calhoun BC, Hume RF. Integrated Obstetric Curriculum for Obstetrics and Gynecology Residency, Radiology Residency and Maternal-Fetal Medicine Fellowship program at an accredited American Institute of Ultrasound in Medicine Diagnostic Ultrasound Center. Ultrasound Obstet Gynecol 2000; 16: 6871.
  • 18
    Salvesen KA, Lees C, Tutschek B. Basic European ultrasound training in obstetrics and gynecology: where are we and where do we go from here? Ultrasound Obstet Gynecol 2010; 36: 525529.
  • 19
    Sidhu HS, Olubaniyi BO, Bhatnagar G, Shuen V, Dubbins P. Role of simulation-based education in ultrasound practice training. J Ultrasound Med 2012; 31: 785791.
  • 20
    Tutschek B, Tercanli S, Chantraine F. Teaching and learning normal gynecological ultrasonography using simple virtual reality objects: a proposal for a standardized approach. Ultrasound Obstet Gynecol 2012; 39: 595596.
  • 21
    Alcázar JL, Galán MJ, García-Manero M, Guerriero S. Three-dimensional sonographic morphologic assessment in complex adnexal masses: preliminary experience. J Ultrasound Med 2003; 22: 249254.
  • 22
    Alcázar JL, García-Manero M, Galván R. Three-dimensional sonographic morphologic assessment of adnexal masses: a reproducibility study. J Ultrasound Med 2007; 26: 10071011.
  • 23
    Alcázar JL, Iturra A, Sedda F, Aubá M, Ajossa S, Guerriero S, Jurado M. Three-dimensional volume off-line analysis as compared to real-time ultrasound for assessing adnexal masses. Eur J Obstet Gynecol Reprod Biol 2012; 161: 9295.
  • 24
    Guerriero S, Alcazar JL, Coccia ME, Ajossa S, Scarselli G, Boi M, Gerada M, Melis GB. Complex pelvic mass as a target of evaluation of vessel distribution by color Doppler sonography for the diagnosis of adnexal malignancies: results of a multicenter European study. J Ultrasound Med 2002; 21: 11051111.
  • 25
    Van Calster B, Timmerman D, Bourne T, Testa AC, Van Holsbeke C, Domali E, Jurkovic D, Neven P, Van Huffel S, Valentin L. Discrimination between benign and malignant adnexal masses by specialist ultrasound examination versus serum CA-125. J Natl Cancer Inst 2007; 99: 17061714.
  • 26
    Sokalska A, Timmerman D, Testa AC, Van Holsbeke C, Lissoni AA, Leone FP, Jurkovic D, Valentin L. Diagnostic accuracy of transvaginal ultrasound examination for assigning a specific diagnosis to adnexal masses. Ultrasound Obstet Gynecol 2009; 34: 462470.
  • 27
    Guerriero S, Alcazar JL, Ajossa S, Galvan R, Laparte C, García-Manero M, Lopez-Garcia G, Melis GB. Transvaginal color Doppler imaging in the detection of ovarian cancer in a large study population. Int J Gynecol Cancer 2010; 20: 781786.
  • 28
    Alcázar JL, Guerriero S. Gray-scale ultrasound versus CA-125 levels for predicting malignancy in adnexal masses. Int J Gynaecol Obstet 2011; 114: 290291.
  • 29
    Testa AC, Van Holsbeke C, Mascilini F, Timmerman D. Dynamic and interactive gynecological ultrasound examination. Ultrasound Obstet Gynecol 2009; 34: 225229.
  • 30
    Valentin L. Use of morphology to characterize and manage common adnexal masses. Best Pract Res Clin Obstet Gynaecol 2004; 18: 7189.
  • 31
    Biau DJ, Porcher R, Salomon LJ. CUSUM: a tool for ongoing assessment of performance. Ultrasound Obstet Gynecol 2008; 31: 252255.
  • 32
    Cruz-Martinez R, Figueras F, Moreno-Alvarez O, Martinez JM, Gomez O, Hernandez-Andrade E, Gratacos E. Learning curve for lung area to head circumference ratio measurement in fetuses with congenital diaphragmatic hernia. Ultrasound Obstet Gynecol 2010; 36: 3236.
  • 33
    Van Holsbeke C, Yazbek J, Holland TK, Daemen A, De Moor B, Testa AC, Valentin L, Jurkovic D, Timmerman D. Real-time ultrasound vs. evaluation of static images in the preoperative assessment of adnexal masses. Ultrasound Obstet Gynecol 2008; 32: 828831.
  • 34
    Tutschek B, Pilu G. Virtual reality ultrasound imaging of the normal and abnormal fetal central nervous system. Ultrasound Obstet Gynecol 2009; 34: 259267.
  • 35
    Bucholz EM, Sue GR, Yeo H, Roman SA, Bell RH, Sosa JA. Our trainees' confidence: results from a national survey of 4136 US general surgery residents. Arch Surg 2011; 146: 907914.
  • 36
    Singhal R, Dheerendra SK, Charalambous CP, Waseem M. Teaching basic shoulder ultrasonography to orthopaedic postgraduate trainees—effectiveness of a training workshop. Med Ultrason 2012; 14: 120124.
  • 37
    Orpen C. The influence of the training environment on trainee motivation and perceived training quality. Int J Training Develop 1999; 3: 3443.
    Direct Link:
SUPPORTING INFORMATION ON THE INTERNET

The following supporting information may be found in the online version of this article:

Appendix S1 Information on theoretical course for training in ultrasound assessment of adnexal masses.

Appendix S2 Brief description of the five exemplar three-dimensional volumes used in the training program and made available online (Volumes S1–S5).

Volumes S1–S5 Five of the three-dimensional ultrasound volumes used in the training program.

Supporting Information

  1. Top of page
  2. ABSTRACT
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. REFERENCES
  8. Supporting Information
FilenameFormatSizeDescription
uog12440-sup-0001-AppendixS1.docWord document24KAppendix S1 Information on theoretical course for training in ultrasound assessment of adnexal masses.
uog12440-sup-0002-AppendixS2.docWord document605KAppendix S2 Brief description of the five exemplar three-dimensional volumes used in the training program and made available online (Volumes S1–S5).
uog12440-sup-0003-VolumesS1-S5.zipapplication/x-zip-compressed139678KVolumes S1–S5. Five of the three-dimensional ultrasound volumes used in the training program.

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.