To investigate the interobserver variability of fetal middle cerebral artery (MCA) peak systolic velocity (PSV) Doppler measurements in a tertiary fetal medicine unit.
To investigate the interobserver variability of fetal middle cerebral artery (MCA) peak systolic velocity (PSV) Doppler measurements in a tertiary fetal medicine unit.
This was a prospective cohort study of pregnant women between 22 and 34 weeks of gestation who underwent fetal MCA-PSV Doppler velocimetry by a paired combination of operators. A set protocol was followed; ease of scanning was recorded and three quality assurance parameters were analyzed. The interobserver variability was determined. In addition, individual operator characteristics were determined by analyzing the variation of each operator relative to all his/her paired colleagues, and their quality assurance parameters.
Two hundred and eighty-five women had completed paired fetal MCA-PSV Doppler measurements. Eighty-three (29%) of the ultrasound examinations requested were for suspected fetal anemia and 202 (71%) were for other obstetric indications. The interobserver variation was less than 10% in 78% of the paired MCA-PSV Doppler measurements whereas 99% had less than 15% variation. The intraclass correlation coefficient of each operator when compared with all his/her colleagues ranged from 0.82 to 0.95. The overall mean variability of the MCA-PSV recorded by a given operator, relative to all his/her paired colleagues, ranged from + 5.26% to − 6.47% in all but one operator whose value was + 13.5% (standard deviation factor, 1.13–1.22). Logistic regression analysis, using a 10% or greater variation in MCA-PSV as a binary outcome variable, showed a significant difference when inappropriate angle correction was detected (P < 0.001).
Clinically acceptable interobserver variability was obtained in fetal MCA-PSV Doppler measurements. Inappropriate angle correction was found to be a significant predictive factor for increased interobserver variability. Analysis of mean variation identified operators who were outliers enabling quality assurance within the unit. Copyright © 2008 ISUOG. Published by John Wiley & Sons, Ltd.
Doppler estimation of peak systolic velocity (PSV) of the fetal middle cerebral artery (MCA) for the prediction of fetal anemia has shown great promise as a non-invasive approach for monitoring the red cell alloimmunized pregnancy1. The advantages of fetal MCA-PSV estimation are its non-invasiveness, ease of ascertainment and reproducibility. One study comparing MCA-PSV with amniocentesis for assessment of amniotic fluid delta optical density 450 (OD450) showed similar sensitivities, but a better prediction for actual fetal hemoglobin with MCA-PSV Doppler2. Many others have established the value of MCA Doppler measurement as a non-invasive indicator of fetal anemia3–6. The prospective, international multicenter DIAMOND (Diagnostic Amniocentesis or Noninvasive Doppler study group) trial concluded that MCA-PSV measurement could safely replace invasive testing in the management of Rhesus alloimmunized pregnancies7.
However, Segata and Mari have raised serious concerns about operator training in recording the fetal MCA-PSV8. They concluded that sonologist and sonographer training was a prerequisite, a conditio sine qua non. Green and Alfirevic have commented that the introduction of MCA-PSV Doppler estimation as a new screening test would challenge the international fetal medicine community to find the best way of incorporating the test so as to avoid false positives9. Hellmeyer et al. concluded that internal quality standards need to be in place before using MCA-PSV for clinical purposes10. Mari et al. suggested that the implementation of a set protocol for obtaining MCA-PSV Doppler measurements had the potential to improve the utility of the test11. Previous studies on interobserver variability lacked large numbers10–12. We are not aware of any large studies published on clinically acceptable interobserver variability or studies demonstrating internal quality assurance in obstetric imaging units.
MCA-PSV Doppler measurement is the primary mode of surveillance at the Women's and Children's Hospital, North Adelaide, when the fetus is at risk of anemia. Recently, experienced obstetric sonographers have been added to the group performing MCA-PSV Doppler imaging after appropriate training. The aim of this study was to test the interobserver variability between paired operators in a tertiary imaging unit, after operator training with a set protocol. In addition, individual operator characteristics were determined.
Pregnant women at 22–34 weeks' gestation referred to the Perinatal Imaging Unit for ultrasound examination for various obstetric indications, from November 2005 to December 2006, were prospectively enrolled into this study. Women with multifetal pregnancies and fetuses with anomalies were excluded. The study was approved by the Ethics Committee of the Women's and Children's Hospital.
The protocol for obtaining MCA-PSV Doppler measurements is as follows.
Sonographers should be ‘certified’ by the maternal–fetal medicine team for MCA-PSV measurements.
Zoom the B-mode image just above the sphenoid wings below the biparietal diameter plane.
Use color flow Doppler imaging to identify the circle of Willis and visualize both MCAs.
Place the spectral gate at the proximal portion of the near-field MCA close to the internal carotid artery (ICA) origin.
No portion of the ICA is to be incorporated.
Obtain an angle of insonation of 0° when possible. When 0° insonation is not obtainable, angle correction up to 30° is permitted.
Obtain waveforms when the fetus is in a state of rest.
Measure the highest MCA-PSV on that trace. Obtain a second waveform and measure the highest PSV.
If waveforms/velocities are not consistent, a third waveform/measurement should be obtained.
If the near-field proximal portion cannot be interrogated, the far-field proximal portion can be interrogated.
Ultrasound examinations were performed using Philips iU22 and HDI 5000 ultrasound systems (Philips Medical Systems, Bothell, WA, USA) by experienced obstetric sonographers and maternal–fetal medicine fellows/subspecialists, all familiar with the set protocol for obtaining MCA-PSV.
After the indicated obstetric scan had been performed, the first operator obtained a MCA-PSV Doppler measurement, followed by the second operator, who was blinded to the first operator's result. Data collected during each paired evaluation were as follows: operator performing MCA-PSV Doppler measurement, actual MCA-PSV measurement by each operator, and the level of ease in obtaining the MCA-PSV. Data regarding the indications for ultrasound evaluation and gestational ages at examination were collected.
The color images were reviewed in pairs post hoc by the primary investigator (J.T.T.) for quality assessment. Quality was determined by evaluation of the angle of insonation, adequacy of view of the circle of Willis and visualization of the distal MCA.
We planned to perform 20 pairs of observations for each of the paired combinations of six operators. All data were entered into an Access database (Microsoft Corporation, Redmond, WA, USA).
Analysis was carried out using SPSS for Windows version 11.0.1 (SPSS Inc., Chicago, IL, USA) and S-Plus (Insightful Corporation (NASDAQ: IFUL), Seattle, WA, USA). The interobserver variability was assessed using the logarithm of MCA-PSV (because variance in MCA-PSV increases with the mean). For each operator, a mean difference between the log MCA-PSV of each reading and the observation of his/her paired operator was calculated. The percentage difference was calculated from the exponential of this mean difference. A ‘pseudo’ intraclass correlation coefficient (ICC) was estimated for each operator from an analysis of variance examining operator differences within individual subjects (fetal MCA-PSV). For pragmatic reasons and out of consideration for the women who agreed to participate in the study, no more than two operators made ultrasound measurements on the same subject. Hence we were unable to use the more usual ‘panel’ design, in which all observers record observations on the same subject. As only two observers made measurements on each fetus, the second operator was a composite of all the index operator's pairings.
In addition, individual operator characteristics were determined by analyzing the variation of each operator relative to all of his/her paired colleagues. Logistic regression analysis, using a 10% or greater variation in the MCA-PSV as the binary outcome, was performed to review the effect on interobserver variability of the ease of obtaining the measurement, angle of insonation, adequacy of view of the circle of Willis and visualization of the distal MCA.
The Women's and Children's Hospital uses a modification of the graph by Moise5 for the threshold values of PSV in the MCA for the management of fetal anemia (Figure 1). The cut-off of interobserver variability of 10% in our logistic regression analysis was chosen because approximately 20% variation exists at any gestational age (17% at 18 weeks' gestation and 21% at 34 weeks) in Zone B (1.29–1.5 MoM) , which corresponds to mild anemia.
Three hundred paired MCA-PSV measurements were attempted. Fifteen (5% of the total) MCA-PSV measurements could not be completed owing to technical difficulties. A total of 215 women were recruited for the study, including 23 women with suspected fetal anemia who participated several times during the pregnancy. The maternal age ranged from 17 to 45 years. There were 78 (27%) fetuses between 22 and 28 completed gestational weeks at the time of scanning and 207 (73%) fetuses between 29 and 34 completed weeks of gestation. Eighty-three (29%) ultrasound examinations were performed for assessment of fetal anemia, 128 (45%) for assessment of fetal growth, 74 (26%) for placental localization or follow-up of fetal renal pelvic dilatation.
The interobserver variability was less than 10% in 78% (222/285) of the paired MCA-PSV Doppler measurements, and 99% (282/285) had less than 15% variation. The ICC of each operator when compared with all his/her pairings ranged from 0.82 to 0.95.
Fifteen fetal transfusions were performed during the study period but only seven fetuses were eligible for inclusion in the study; the other transfusions were performed on twin pregnancies. All seven fetuses had MCA-PSV Doppler values in Zone A, suggestive of severe anemia (modified gestational-age specific MCA-PSV Doppler chart). Fetal anemia was confirmed by fetal blood sampling in six of the seven fetuses that required fetal transfusion. The interobserver variability of all the MCA-PSV Doppler measurements in this small subgroup was < 10%.
Logistic regression analysis of the MCA-PSV, using 10% or greater variation as a binary outcome variable, resulted in a model that showed no significant difference associated with the level of ease of obtaining the measurement, visualization of the circle of Willis or the contralateral MCA. However, there was a significant difference when inappropriate angle correction was used (P < 0.001) (Table 1).
|Variability (%)||Appropriate angle correction||Inappropriate angle correction||Total||P*|
|< 10||190 (67)||32 (11)||222 (78)||< 0.001|
|≥ 10||44 (15)||19 (7)||63 (22)|
The number of individual operators increased to nine during the study period for logistic reasons. The overall mean variation of the MCA-PSV Doppler measurement of each operator with his/her pairs ranged from + 13.5% to − 6.47% (SD factor, 1.13–1.22) (Table 2). However, excluding one operator (an outlier at + 13.5%), the mean interobserver variation of the other eight operators ranged from + 5.26% to − 6.47%.
|Ultrasound operator||Number of pairings||Overall mean percentage variation relative to all other associates||SD factor*|
|Operator 1||34||+ 13.5||1.20|
|Operator 2||58||− 5.72||1.19|
|Operator 3||74||+ 5.26||1.18|
|Operator 4||58||+ 3.05||1.18|
|Operator 5||107||+ 0.68||1.17|
|Operator 6||54||− 0.13||1.18|
|Operator 7||90||− 6.47||1.22|
|Operator 8||42||− 2.77||1.13|
|Operator 9||21||+ 1.1||1.13|
The aim of our study was to evaluate the interobserver variability in MCA-PSV Doppler measurements after the addition of new operators to our sonographic team. Establishing the interobserver variability of MCA-PSV Doppler measurements for members of a fetal medicine unit is necessary to enable clinicians to have confidence in all the operators performing the test11. We were able to demonstrate that the interobserver variability of the MCA-PSV Doppler measurement was within clinically acceptable limits, regardless of the operator when he/she was familiar with the protocol. This has shown MCA-PSV Doppler measurement to be an acceptable and efficient clinical tool in the assessment of fetal anemia in a busy obstetric imaging unit.
Abel et al. published a study in which four sonographers measured the proximal and distal MCA-PSV of 11 patients, and reported ICCs of 0.94 and 0.83, respectively12, whereas in a study on 30 healthy fetuses conducted in two institutions, Mari et al. reported ICCs of 0.98–0.99 between two pairs of sonographers and sonologists11. The ICCs in our study with nine operators and 285 pairs of readings were between 0.82 and 0.95, similar to those reported by Abel et al.
Interobserver variability was assessed in 11 fetuses which were part of a larger study by Mari et al. and was found to be 2.5%13, whereas in a study of 81 patients Hellmeyer et al. reported a 20% divergence from the presumed line of best fit in more than 42% of measurements10. This contrasts with our results, in which 78% of the paired measurements had less than 10% variability and 99% of the paired measurements had less than 15% variability. This is the first study we are aware of with a large number (285) of paired Doppler MCA-PSV measurements performed in a busy tertiary ultrasound unit that established the level of interobserver variability. Our results may differ from those of Hellmeyer et al. owing to our development of early operator training and keeping to a set sonographic protocol. This is crucial when introducing new sonographers to this technology. Our findings have important clinical implications because multiple operators may perform MCA-PSV measurements during the surveillance of an anemic fetus in a busy fetal medicine unit. The clinician must be confident that an increased slope represents a genuine increase rather than just interobserver variability. This is important because the decision to perform fetal blood sampling on a fetus suspected to be anemic may be based on the slope of increase in MCA-PSV rather than the actual value4.
The interobserver variability in measurement of the MCA-PSV published so far has been based on small numbers in research settings. We wished to test the interobserver variability in a clinical setting. Zone B (between 1.29 and 1.5 multiples of the median (MoM)), corresponding to mild anemia in the gestational age-specific MCA-PSV graph, contains around 20% variation in MCA-PSV at any given gestational age (Figure 1). With an interobserver variability of < 10%, we can be confident that any MCA-PSV reading below the 50th percentile line of this zone would not be in Zone A if measured by a different observer. Therefore, we can be assured that a measurement of MCA-PSV below the median of Zone B would be likely to distinguish a mildly anemic/non-anemic fetus from a moderately/severely anemic fetus, and so would not result in any difference in clinical management. A measurement of MCA-PSV above the median of Zone B, on the other hand, requires the implementation of diligent surveillance and early follow-up measurements. Close attention to the slope of consecutive measurements may be more important at these higher MCA-PSV readings than any single measurement.
Moreover, the ability to plot individual operator characteristics meant that we were able to give feedback to the operators with regard to variations in technique. As with other important ultrasound measurements that require close scrutiny, early and constant operator evaluations may prevent measurement divergence. For example, the variation of + 13.5% seen for Operator 1 (Table 2), allows us the opportunity to review the quality assurance parameters and technique of this operator. This may allow us to ascertain why these measurements were consistently higher than those of the other operators.
We found that when the interobserver variability was equal to or greater than 10%, of the three quality assurance parameters studied (angle of insonation, adequacy of view of the circle of Willis and visualization of the distal MCA), inappropriate angle correction was found to be statistically significant. Zimmerman et al. have suggested that angle correction would be needed when the angle of insonation exceeds 10–15°, but has not reported on the upper limit of the angle correction that is acceptable14. The article by Mari et al. on the use of MCA-PSV Doppler measurement attempted insonation as close to 0° possible, but when a correction was made the angle used was not reported13. Subsequently the reliability of angle correction was evaluated and they reported that angle-corrected measurements were more difficult to reproduce11. Abel et al. did not include any angle-corrected measurements in their analysis of interobserver variability12. The set protocol we used regarding angle correction was similar to the protocol used by the DIAMOND study group where up to 30° angle correction was used if 0° was not possible7. Our findings verify the importance of highlighting angle correction in the training of sonographers with this clinical tool.
Our study had a number of limitations. We were unable adequately to study the possible interobserver variability in severely anemic fetuses as only six fetuses had confirmed fetal anemia and only seven fetuses in the study had MCA-PSV Doppler measurements greater than 1.5 MoM. However, in reviewing this small subset, we found that the interobserver variability of the paired MCA-PSV Doppler measurements was < 10%, suggesting reasonable interobserver variability. A larger cohort of anemic fetuses will have to be studied to establish the interobserver variability in this group.
This study was not designed to correlate the interobserver variability of fetal MCA-PSV measurement with fetal hemoglobin. Therefore, the clinical implication of our findings may not be applicable to the severely anemic fetus. The pool of operators varied during the study, and so a systematic random pairing as planned was not possible. This led to unequal distribution of operator pairings. However, the strength of the study relies on the fact that multiple operators from a busy fetal medicine unit participated in a large number of paired measurements, making our findings applicable to other busy fetal imaging units.
In conclusion, by using a set imaging protocol and departmental operator training, we were able to demonstrate clinically acceptable interobserver variability for the measurement of MCA-PSV in the non-anemic and the mildly anemic fetus. Therefore, in busy fetal medicine imaging units we can have confidence in making important clinical decisions based on this non-invasive clinical tool. Future research may include the study of interobserver variability in fetuses with moderate to severe anemia, the post-transfusion fetus, as well as the effect of antepartum steroids on MCA-PSV measurements.
We would like to thank all of the pregnant women who took part in the study, without whose help this study would not have been possible. We would also like to acknowledge the sonographers and the departmental staff of Perinatal Imaging, Women's and Children's Hospital, North Adelaide, who helped carry out this study, especially Ben Khor, Cara Kirsten, Catrina Pannucio, Gail Zabroski, Kate Gratten, Lara Rule, Penny Scott, Piotr Niznik and Eunice Carey.