SEARCH

SEARCH BY CITATION

Keywords:

  • Doppler;
  • fetal anemia;
  • interobserver variability;
  • middle cerebral artery

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Objectives

To investigate the interobserver variability of fetal middle cerebral artery (MCA) peak systolic velocity (PSV) Doppler measurements in a tertiary fetal medicine unit.

Methods

This was a prospective cohort study of pregnant women between 22 and 34 weeks of gestation who underwent fetal MCA-PSV Doppler velocimetry by a paired combination of operators. A set protocol was followed; ease of scanning was recorded and three quality assurance parameters were analyzed. The interobserver variability was determined. In addition, individual operator characteristics were determined by analyzing the variation of each operator relative to all his/her paired colleagues, and their quality assurance parameters.

Results

Two hundred and eighty-five women had completed paired fetal MCA-PSV Doppler measurements. Eighty-three (29%) of the ultrasound examinations requested were for suspected fetal anemia and 202 (71%) were for other obstetric indications. The interobserver variation was less than 10% in 78% of the paired MCA-PSV Doppler measurements whereas 99% had less than 15% variation. The intraclass correlation coefficient of each operator when compared with all his/her colleagues ranged from 0.82 to 0.95. The overall mean variability of the MCA-PSV recorded by a given operator, relative to all his/her paired colleagues, ranged from + 5.26% to − 6.47% in all but one operator whose value was + 13.5% (standard deviation factor, 1.13–1.22). Logistic regression analysis, using a 10% or greater variation in MCA-PSV as a binary outcome variable, showed a significant difference when inappropriate angle correction was detected (P < 0.001).

Conclusions

Clinically acceptable interobserver variability was obtained in fetal MCA-PSV Doppler measurements. Inappropriate angle correction was found to be a significant predictive factor for increased interobserver variability. Analysis of mean variation identified operators who were outliers enabling quality assurance within the unit. Copyright © 2008 ISUOG. Published by John Wiley & Sons, Ltd.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Doppler estimation of peak systolic velocity (PSV) of the fetal middle cerebral artery (MCA) for the prediction of fetal anemia has shown great promise as a non-invasive approach for monitoring the red cell alloimmunized pregnancy1. The advantages of fetal MCA-PSV estimation are its non-invasiveness, ease of ascertainment and reproducibility. One study comparing MCA-PSV with amniocentesis for assessment of amniotic fluid delta optical density 450 (OD450) showed similar sensitivities, but a better prediction for actual fetal hemoglobin with MCA-PSV Doppler2. Many others have established the value of MCA Doppler measurement as a non-invasive indicator of fetal anemia3–6. The prospective, international multicenter DIAMOND (Diagnostic Amniocentesis or Noninvasive Doppler study group) trial concluded that MCA-PSV measurement could safely replace invasive testing in the management of Rhesus alloimmunized pregnancies7.

However, Segata and Mari have raised serious concerns about operator training in recording the fetal MCA-PSV8. They concluded that sonologist and sonographer training was a prerequisite, a conditio sine qua non. Green and Alfirevic have commented that the introduction of MCA-PSV Doppler estimation as a new screening test would challenge the international fetal medicine community to find the best way of incorporating the test so as to avoid false positives9. Hellmeyer et al. concluded that internal quality standards need to be in place before using MCA-PSV for clinical purposes10. Mari et al. suggested that the implementation of a set protocol for obtaining MCA-PSV Doppler measurements had the potential to improve the utility of the test11. Previous studies on interobserver variability lacked large numbers10–12. We are not aware of any large studies published on clinically acceptable interobserver variability or studies demonstrating internal quality assurance in obstetric imaging units.

MCA-PSV Doppler measurement is the primary mode of surveillance at the Women's and Children's Hospital, North Adelaide, when the fetus is at risk of anemia. Recently, experienced obstetric sonographers have been added to the group performing MCA-PSV Doppler imaging after appropriate training. The aim of this study was to test the interobserver variability between paired operators in a tertiary imaging unit, after operator training with a set protocol. In addition, individual operator characteristics were determined.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Pregnant women at 22–34 weeks' gestation referred to the Perinatal Imaging Unit for ultrasound examination for various obstetric indications, from November 2005 to December 2006, were prospectively enrolled into this study. Women with multifetal pregnancies and fetuses with anomalies were excluded. The study was approved by the Ethics Committee of the Women's and Children's Hospital.

The protocol for obtaining MCA-PSV Doppler measurements is as follows.

  • Sonographers should be ‘certified’ by the maternal–fetal medicine team for MCA-PSV measurements.

  • Zoom the B-mode image just above the sphenoid wings below the biparietal diameter plane.

  • Use color flow Doppler imaging to identify the circle of Willis and visualize both MCAs.

  • Place the spectral gate at the proximal portion of the near-field MCA close to the internal carotid artery (ICA) origin.

  • No portion of the ICA is to be incorporated.

  • Obtain an angle of insonation of 0° when possible. When 0° insonation is not obtainable, angle correction up to 30° is permitted.

  • Obtain waveforms when the fetus is in a state of rest.

  • Measure the highest MCA-PSV on that trace. Obtain a second waveform and measure the highest PSV.

  • If waveforms/velocities are not consistent, a third waveform/measurement should be obtained.

  • If the near-field proximal portion cannot be interrogated, the far-field proximal portion can be interrogated.

Ultrasound examinations were performed using Philips iU22 and HDI 5000 ultrasound systems (Philips Medical Systems, Bothell, WA, USA) by experienced obstetric sonographers and maternal–fetal medicine fellows/subspecialists, all familiar with the set protocol for obtaining MCA-PSV.

After the indicated obstetric scan had been performed, the first operator obtained a MCA-PSV Doppler measurement, followed by the second operator, who was blinded to the first operator's result. Data collected during each paired evaluation were as follows: operator performing MCA-PSV Doppler measurement, actual MCA-PSV measurement by each operator, and the level of ease in obtaining the MCA-PSV. Data regarding the indications for ultrasound evaluation and gestational ages at examination were collected.

The color images were reviewed in pairs post hoc by the primary investigator (J.T.T.) for quality assessment. Quality was determined by evaluation of the angle of insonation, adequacy of view of the circle of Willis and visualization of the distal MCA.

We planned to perform 20 pairs of observations for each of the paired combinations of six operators. All data were entered into an Access database (Microsoft Corporation, Redmond, WA, USA).

Statistical analysis

Analysis was carried out using SPSS for Windows version 11.0.1 (SPSS Inc., Chicago, IL, USA) and S-Plus (Insightful Corporation (NASDAQ: IFUL), Seattle, WA, USA). The interobserver variability was assessed using the logarithm of MCA-PSV (because variance in MCA-PSV increases with the mean). For each operator, a mean difference between the log MCA-PSV of each reading and the observation of his/her paired operator was calculated. The percentage difference was calculated from the exponential of this mean difference. A ‘pseudo’ intraclass correlation coefficient (ICC) was estimated for each operator from an analysis of variance examining operator differences within individual subjects (fetal MCA-PSV). For pragmatic reasons and out of consideration for the women who agreed to participate in the study, no more than two operators made ultrasound measurements on the same subject. Hence we were unable to use the more usual ‘panel’ design, in which all observers record observations on the same subject. As only two observers made measurements on each fetus, the second operator was a composite of all the index operator's pairings.

In addition, individual operator characteristics were determined by analyzing the variation of each operator relative to all of his/her paired colleagues. Logistic regression analysis, using a 10% or greater variation in the MCA-PSV as the binary outcome, was performed to review the effect on interobserver variability of the ease of obtaining the measurement, angle of insonation, adequacy of view of the circle of Willis and visualization of the distal MCA.

The Women's and Children's Hospital uses a modification of the graph by Moise5 for the threshold values of PSV in the MCA for the management of fetal anemia (Figure 1). The cut-off of interobserver variability of 10% in our logistic regression analysis was chosen because approximately 20% variation exists at any gestational age (17% at 18 weeks' gestation and 21% at 34 weeks) in Zone B (1.29–1.5 MoM) , which corresponds to mild anemia.

thumbnail image

Figure 1. Gestational age-specific middle cerebral artery peak systolic velocity (MCA-PSV) graph used at the Women's and Children's Hospital, modified from Moise5. Zone A represents moderate to severe anemia, Zone B represents mild anemia and Zone C represents no anemia. Median (equation image), 1.5 multiples of the median (MoM) (equation image), 1.29 MoM (equation image) and 50th percentile of Zone B (equation image) are shown.

Download figure to PowerPoint

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

Three hundred paired MCA-PSV measurements were attempted. Fifteen (5% of the total) MCA-PSV measurements could not be completed owing to technical difficulties. A total of 215 women were recruited for the study, including 23 women with suspected fetal anemia who participated several times during the pregnancy. The maternal age ranged from 17 to 45 years. There were 78 (27%) fetuses between 22 and 28 completed gestational weeks at the time of scanning and 207 (73%) fetuses between 29 and 34 completed weeks of gestation. Eighty-three (29%) ultrasound examinations were performed for assessment of fetal anemia, 128 (45%) for assessment of fetal growth, 74 (26%) for placental localization or follow-up of fetal renal pelvic dilatation.

The interobserver variability was less than 10% in 78% (222/285) of the paired MCA-PSV Doppler measurements, and 99% (282/285) had less than 15% variation. The ICC of each operator when compared with all his/her pairings ranged from 0.82 to 0.95.

Fifteen fetal transfusions were performed during the study period but only seven fetuses were eligible for inclusion in the study; the other transfusions were performed on twin pregnancies. All seven fetuses had MCA-PSV Doppler values in Zone A, suggestive of severe anemia (modified gestational-age specific MCA-PSV Doppler chart). Fetal anemia was confirmed by fetal blood sampling in six of the seven fetuses that required fetal transfusion. The interobserver variability of all the MCA-PSV Doppler measurements in this small subgroup was < 10%.

Logistic regression analysis of the MCA-PSV, using 10% or greater variation as a binary outcome variable, resulted in a model that showed no significant difference associated with the level of ease of obtaining the measurement, visualization of the circle of Willis or the contralateral MCA. However, there was a significant difference when inappropriate angle correction was used (P < 0.001) (Table 1).

Table 1. Interobserver variability and appropriate angle correction
Variability (%)Appropriate angle correctionInappropriate angle correctionTotalP*
  • Values are n (%).

  • *

    Logistic regression using a ≥ 10% variation in the middle cerebral artery (MCA) peak systolic velocity measurement as the binary outcome and angle of insonation as a predictor. Inappropriate angle correction was defined as no correction done (except when angle of insonation = 0°), or undercorrection or overcorrection (defined as the line of angle correction running obliquely to the MCA alignment as seen on the color Doppler image printout).

< 10190 (67)32 (11)222 (78)< 0.001
≥ 1044 (15)19 (7)63 (22) 

The number of individual operators increased to nine during the study period for logistic reasons. The overall mean variation of the MCA-PSV Doppler measurement of each operator with his/her pairs ranged from + 13.5% to − 6.47% (SD factor, 1.13–1.22) (Table 2). However, excluding one operator (an outlier at + 13.5%), the mean interobserver variation of the other eight operators ranged from + 5.26% to − 6.47%.

Table 2. Percentage mean variation in the middle cerebral artery peak systolic velocity Doppler measurements of each operator relative to all his/her pairs
Ultrasound operatorNumber of pairingsOverall mean percentage variation relative to all other associatesSD factor*
  • *

    Geometric means (corrected for the increased variance with the mean) were calculated for all operator pairs and hence the SD is expressed in terms of a multiplier (factor), i.e. an operator with a SD factor of 1.18 has a SD of (100 × 1.18 − 100) = + 18% and (100/1.18 − 100) = − 15% of his/her associate's estimate.

Operator 134+ 13.51.20
Operator 258− 5.721.19
Operator 374+ 5.261.18
Operator 458+ 3.051.18
Operator 5107+ 0.681.17
Operator 654− 0.131.18
Operator 790− 6.471.22
Operator 842− 2.771.13
Operator 921+ 1.11.13

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

The aim of our study was to evaluate the interobserver variability in MCA-PSV Doppler measurements after the addition of new operators to our sonographic team. Establishing the interobserver variability of MCA-PSV Doppler measurements for members of a fetal medicine unit is necessary to enable clinicians to have confidence in all the operators performing the test11. We were able to demonstrate that the interobserver variability of the MCA-PSV Doppler measurement was within clinically acceptable limits, regardless of the operator when he/she was familiar with the protocol. This has shown MCA-PSV Doppler measurement to be an acceptable and efficient clinical tool in the assessment of fetal anemia in a busy obstetric imaging unit.

Abel et al. published a study in which four sonographers measured the proximal and distal MCA-PSV of 11 patients, and reported ICCs of 0.94 and 0.83, respectively12, whereas in a study on 30 healthy fetuses conducted in two institutions, Mari et al. reported ICCs of 0.98–0.99 between two pairs of sonographers and sonologists11. The ICCs in our study with nine operators and 285 pairs of readings were between 0.82 and 0.95, similar to those reported by Abel et al.

Interobserver variability was assessed in 11 fetuses which were part of a larger study by Mari et al. and was found to be 2.5%13, whereas in a study of 81 patients Hellmeyer et al. reported a 20% divergence from the presumed line of best fit in more than 42% of measurements10. This contrasts with our results, in which 78% of the paired measurements had less than 10% variability and 99% of the paired measurements had less than 15% variability. This is the first study we are aware of with a large number (285) of paired Doppler MCA-PSV measurements performed in a busy tertiary ultrasound unit that established the level of interobserver variability. Our results may differ from those of Hellmeyer et al. owing to our development of early operator training and keeping to a set sonographic protocol. This is crucial when introducing new sonographers to this technology. Our findings have important clinical implications because multiple operators may perform MCA-PSV measurements during the surveillance of an anemic fetus in a busy fetal medicine unit. The clinician must be confident that an increased slope represents a genuine increase rather than just interobserver variability. This is important because the decision to perform fetal blood sampling on a fetus suspected to be anemic may be based on the slope of increase in MCA-PSV rather than the actual value4.

The interobserver variability in measurement of the MCA-PSV published so far has been based on small numbers in research settings. We wished to test the interobserver variability in a clinical setting. Zone B (between 1.29 and 1.5 multiples of the median (MoM)), corresponding to mild anemia in the gestational age-specific MCA-PSV graph, contains around 20% variation in MCA-PSV at any given gestational age (Figure 1). With an interobserver variability of < 10%, we can be confident that any MCA-PSV reading below the 50th percentile line of this zone would not be in Zone A if measured by a different observer. Therefore, we can be assured that a measurement of MCA-PSV below the median of Zone B would be likely to distinguish a mildly anemic/non-anemic fetus from a moderately/severely anemic fetus, and so would not result in any difference in clinical management. A measurement of MCA-PSV above the median of Zone B, on the other hand, requires the implementation of diligent surveillance and early follow-up measurements. Close attention to the slope of consecutive measurements may be more important at these higher MCA-PSV readings than any single measurement.

Moreover, the ability to plot individual operator characteristics meant that we were able to give feedback to the operators with regard to variations in technique. As with other important ultrasound measurements that require close scrutiny, early and constant operator evaluations may prevent measurement divergence. For example, the variation of + 13.5% seen for Operator 1 (Table 2), allows us the opportunity to review the quality assurance parameters and technique of this operator. This may allow us to ascertain why these measurements were consistently higher than those of the other operators.

We found that when the interobserver variability was equal to or greater than 10%, of the three quality assurance parameters studied (angle of insonation, adequacy of view of the circle of Willis and visualization of the distal MCA), inappropriate angle correction was found to be statistically significant. Zimmerman et al. have suggested that angle correction would be needed when the angle of insonation exceeds 10–15°, but has not reported on the upper limit of the angle correction that is acceptable14. The article by Mari et al. on the use of MCA-PSV Doppler measurement attempted insonation as close to 0° possible, but when a correction was made the angle used was not reported13. Subsequently the reliability of angle correction was evaluated and they reported that angle-corrected measurements were more difficult to reproduce11. Abel et al. did not include any angle-corrected measurements in their analysis of interobserver variability12. The set protocol we used regarding angle correction was similar to the protocol used by the DIAMOND study group where up to 30° angle correction was used if 0° was not possible7. Our findings verify the importance of highlighting angle correction in the training of sonographers with this clinical tool.

Our study had a number of limitations. We were unable adequately to study the possible interobserver variability in severely anemic fetuses as only six fetuses had confirmed fetal anemia and only seven fetuses in the study had MCA-PSV Doppler measurements greater than 1.5 MoM. However, in reviewing this small subset, we found that the interobserver variability of the paired MCA-PSV Doppler measurements was < 10%, suggesting reasonable interobserver variability. A larger cohort of anemic fetuses will have to be studied to establish the interobserver variability in this group.

This study was not designed to correlate the interobserver variability of fetal MCA-PSV measurement with fetal hemoglobin. Therefore, the clinical implication of our findings may not be applicable to the severely anemic fetus. The pool of operators varied during the study, and so a systematic random pairing as planned was not possible. This led to unequal distribution of operator pairings. However, the strength of the study relies on the fact that multiple operators from a busy fetal medicine unit participated in a large number of paired measurements, making our findings applicable to other busy fetal imaging units.

In conclusion, by using a set imaging protocol and departmental operator training, we were able to demonstrate clinically acceptable interobserver variability for the measurement of MCA-PSV in the non-anemic and the mildly anemic fetus. Therefore, in busy fetal medicine imaging units we can have confidence in making important clinical decisions based on this non-invasive clinical tool. Future research may include the study of interobserver variability in fetuses with moderate to severe anemia, the post-transfusion fetus, as well as the effect of antepartum steroids on MCA-PSV measurements.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References

We would like to thank all of the pregnant women who took part in the study, without whose help this study would not have been possible. We would also like to acknowledge the sonographers and the departmental staff of Perinatal Imaging, Women's and Children's Hospital, North Adelaide, who helped carry out this study, especially Ben Khor, Cara Kirsten, Catrina Pannucio, Gail Zabroski, Kate Gratten, Lara Rule, Penny Scott, Piotr Niznik and Eunice Carey.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  • 1
    Mari G, Deter RL, Carpenter RL, Rahman F, Zimmerman R, Moise KJ Jr, Dorman KF, Ludomirsky A, Gonzalez R, Gomez R, Oz U, Detti L, Copel JA, Bahado-Singh R, Berry S, Martinez-Poyer J, Blackwell SC. Noninvasive diagnosis by Doppler ultrasonography of fetal anemia due to maternal red-cell alloimmunization. Collaborative Group for Doppler Assessment of the Blood Velocity in Anemic fetuses. N Engl J Med 2000; 342: 914.
  • 2
    Nishie EN, Brizot ML, Liao AW, Carvalho MH, Toma O, Zugaib M. A comparison between middle cerebral artery peak systolic velocity and amniotic fluid optical density at 450 nm in the prediction of fetal anemia. Am J Obstet Gynecol 2003; 188: 214219.
  • 3
    Bahado-Singh RO, Oz AU, Hsu C, Kovanci E, Deren O, Onderoglu L, Mari G. Middle cerebral artery Doppler velocimetric deceleration angle as a predictor of fetal anemia in Rh-alloimmunized fetuses without hydrops. Am J Obstet Gynecol 2000; 183: 746751.
  • 4
    Detti L, Mari G, Akiyama M, Cosmi E, Moise KJ Jr, Stefor T, Conaway M, Deter R. Longitudinal assessment of the middle cerebral artery peak systolic velocity in healthy fetuses and in fetuses at risk for anemia. Am J Obstet Gynecol 2002; 187: 937939.
  • 5
    Moise KJ Jr. Management of rhesus alloimmunization in pregnancy. Obstet Gynecol 2002; 100: 600611.
  • 6
    Teixeira JM, Duncan K, Letsky E, Fisk NM. Middle cerebral artery peak systolic velocity in the prediction of fetal anemia. Ultrasound Obstet Gynecol 2000; 15: 205208.
  • 7
    Oepkes D, Seaward PG, Vandenbussche FP, Windrim R, Kingdom J, Beyene J, Kanhai HH, Ohlsson A, Ryan G; DIAMOND Study Group. Doppler ultrasonography versus amniocentesis to predict fetal anemia. N Engl J Med 2006; 355: 156164.
  • 8
    Segata M, Mari G. Fetal anemia: new technologies. Curr Opin Obstet Gynecol 2004; 16: 153158.
  • 9
    Green P, Alfirevic Z. The evidence base for fetal medicine. Best Pract Res Clin Obstet Gynaecol 2005; 19: 7583.
  • 10
    Hellmeyer L, Sierra F, Heller G, Tekesin I, Kuhnert M, Schmidt S. [Evaluation of the reproducibility of Doppler ultrasonographic measurements in obstetrics.] Z Geburtshilfe Neonatol 2003; 207: 127131.
  • 11
    Mari G, Abuhamad AZ, Cosmi E, Segata M, Altaye M, Akiyama M. Middle cerebral artery peak systolic velocity: technique and variability. J Ultrasound Med 2005; 24: 425430.
  • 12
    Abel DE, Grambow SC, Brancazio LR, Hertzberg BS. Ultrasound assessment of the fetal middle cerebral artery peak systolic velocity: a comparison of the near-field versus far-field vessel. Am J Obstet Gynecol 2003; 189: 986989.
  • 13
    Mari G, Adrignolo A, Abuhamad AZ, Pirhonen J, Jones DC, Ludomirsky A, Copel JA. Diagnosis of fetal anemia with Doppler ultrasound in the pregnancy complicated by maternal blood group immunization. Ultrasound Obstet Gynecol 1995; 5: 400405.
  • 14
    Zimmerman R, Carpenter RJ Jr, Durig P, Mari G. Longitudinal measurement of peak systolic velocity in the fetal middle cerebral artery for monitoring pregnancies complicated by red cell alloimmunisation: a prospective multicentre trial with intention-to-treat. BJOG 2002; 109: 746752.