Agreement and reliability of pelvic floor measurements during contraction using three-dimensional pelvic floor ultrasound and virtual reality

Authors


Abstract

Objectives

Virtual reality is a novel method of visualizing ultrasound data with the perception of depth and offers possibilities for measuring non-planar structures. The levator ani hiatus has both convex and concave aspects. The aim of this study was to compare levator ani hiatus volume measurements obtained with conventional three-dimensional (3D) ultrasound and with a virtual reality measurement technique and to establish their reliability and agreement.

Methods

100 symptomatic patients visiting a tertiary pelvic floor clinic with a normal intact levator ani muscle diagnosed on translabial ultrasound were selected. Datasets were analyzed using a rendered volume with a slice thickness of 1.5 cm at the level of minimal hiatal dimensions during contraction. The levator area (in cm2) was measured and multiplied by 1.5 to get the levator ani hiatus volume in conventional 3D ultrasound (in cm3). Levator ani hiatus volume measurements were then measured semi-automatically in virtual reality (cm3) using a segmentation algorithm. An intra- and interobserver analysis of reliability and agreement was performed in 20 randomly chosen patients.

Results

The mean difference between levator ani hiatus volume measurements performed using conventional 3D ultrasound and virtual reality was 0.10 (95% CI, − 0.15 to 0.35) cm3. The intraclass correlation coefficient (ICC) comparing conventional 3D ultrasound with virtual reality measurements was > 0.96. Intra- and interobserver ICCs for conventional 3D ultrasound measurements were > 0.94 and for virtual reality measurements were > 0.97, indicating good reliability for both.

Conclusion

Levator ani hiatus volume measurements performed using virtual reality were reliable and the results were similar to those obtained with conventional 3D ultrasonography. Copyright © 2012 ISUOG. Published by John Wiley & Sons, Ltd.

Introduction

Pelvic floor disorders affect a substantial number of women worldwide. The levator ani muscle is thought to be of great importance for pelvic organ support1. It has been shown that abnormal distensibility of the levator ani hiatal dimension (i.e. ‘ballooning’) is associated with an increased risk of pelvic organ prolapse2, 3.

In the 1980s, magnetic resonance imaging (MRI) was the only available imaging method capable of assessing the levator ani muscle in vivo4. However, cost, availability and contraindications limited the implementation of MRI in clinical practice. With the introduction of three-dimensional (3D) pelvic floor ultrasound imaging in the 1990s a non-invasive and more practical technique for assessing levator ani morphology has become possible5, 6. Furthermore four-dimensional (4D) dynamic ultrasound can be performed, which allows the investigator to perform measurements during rest, contraction and Valsalva maneuver. Previous studies indicate that 3D ultrasound can be used for obtaining levator ani hiatus volumes instead of MRI7, 8.

Silva-Filho et al.9 generated a 3D computer model derived from MRI imaging visualizing the different aspects of the levator ani and showed that the levator area can be visualized as a non-Euclidean hyperbolic structure. They illustrated the different dimensions of the levator ani for the anterior, middle and posterior compartments at different sections through the hiatal area, as well as for patients with and without prolapse. These findings were supported by Kruger et al.10, who compared MRI with 3D ultrasound imaging and found that investigators previously assumed that the minimal dimensions of the hiatus can be measured in a flat plane. However, the 3D nature of the hiatus means that the true levator hiatus occupies a warped (non-Euclidean) plane and that hiatal measurements may be subject to systematic error if performed in a flat (Euclidean) plane.

Measurements of the levator ani hiatus with 3D ultrasound are performed in two dimensions (2D) using a rendered volume, single plane measurements and/or volume contrast imaging with different slice thickness. The non-Euclidean nature, i.e. the concave and convex shape of the levator hiatus as described by Silva-Filho et al.9 and Kruger et al.10 is not taken into consideration in volume measurements using transperineal ultrasound. Therefore, the levator hiatal area visualized and measured in 2D might overestimate or underestimate the ‘real’ dimensions of the levator hiatal area.

By using the I-Space virtual-reality system (which has already been successfully used for 3D prenatal ultrasonography11, 12) the concave and convex features of the levator can be visualized, allowing the investigator to represent better the true (3D) appearance of the levator hiatus and measure it more effectively.

The aim of this study was to compare levator ani hiatus volume measurements performed using conventional 3D ultrasound with those performed using a virtual reality application and to establish the agreement and reliability of both techniques.

Methods

In 2008, 100 symptomatic patients attending a tertiary pelvic floor clinic with a normal levator ani were selected. A normal levator ani was defined as an intact levator ani attachment in eight slices when utilizing the tomographic ultrasound imaging technique described by Dietz13. All had undergone a standardized interview and pelvic floor ultrasound imaging in the supine position and after voiding, using a Voluson 730 Expert system with a 4–8-MHz RAB transabdominal probe (GE Medical Systems, Zipf, Austria) as previously described by Dietz5. Offline analysis of the levator ani hiatus during maximal contraction was performed blinded, without knowledge of the patient's history.

Conventional 3D ultrasound measurements

Offline conventional 3D ultrasound measurements were performed using specialized 3D imaging software, 4D View version 9.0 (GE Medical Systems). A rendered volume with a slice thickness of 1.5 cm was obtained at the level of minimal hiatal dimensions during contraction (Figure 1)6. In 4D View the levator ani area of this rendered dataset is measured in cm2. The slice thickness of the rendered volume was set at 1.5 cm; therefore the conventional volume measurements in cm2 were multiplied by 1.5 to get the levator ani hiatus volume in cm3. These conventional volume measurements were later compared with the levator ani hiatus volume measured in virtual reality (cm3).

Figure 1.

Rendered volume (slice thickness 1.5 cm) of a normal levator ani as seen and measured using 4D View.

Virtual reality measurements

We performed levator ani hiatus volume measurements in virtual reality by storing the 3D datasets as Cartesian volumes in 4D View. These 3D datasets were then visualized in the I-Space, a so-called four-walled CAVE-like (Cave Automatic Virtual Environment) virtual reality system14. In the I-Space a researcher is surrounded by computer-generated stereo images, which are projected by eight high-quality digital light processing projectors onto three walls and the floor of a small room. With V-Scope15, a volume-rendering application developed in-house, an interactive hologram of the ultrasound image is created and can be manipulated and measured by means of a virtual pointer, controlled by a wireless joystick (Figure 2). The hologram must be viewed through glasses with polarizing lenses to create the perception of depth. Hereby, the I-Space allows medical professionals to view and interact with their volumetric data in all three dimensions, which provides them with views much more like those they will experience during surgery16, 17.

Figure 2.

Operator examining the levator ani muscle using the I-Space virtual reality system.

To perform volume measurements, a flexible and robust segmentation algorithm that is based on a region-growing approach in combination with a neighborhood variation threshold was implemented18. This algorithm has been modified to handle the speckles in ultrasound data by smoothing the gray-level data19.

Prior to the levator ani hiatus volume measurement, the 3D datasets were enlarged, rotated and cropped. The outside of the puborectalis has to be ‘brushed away’ with an eraser to avoid segmentation of parts other than the levator ani. The hypoechoic inside of the levator ani was then chosen by placing a seed point after selecting an upper and lower gray-level threshold and an upper threshold for the standard deviation of the voxel neighborhood. If the volume measurement was incomplete, the user could manually grow (or shrink) the segmented region with a spherical, free hand ‘paint brush’ to add voxels to or delete voxels from the segmented structure when necessary11, 12, 19. Figure 3 shows the complete volume, displayed in blue, as measured in virtual reality.

Figure 3.

Two-dimensional virtual reality images of a volume measurement of the levator hiatal area, displayed in blue, showing the axial (a), coronal (b) and midsagittal (c) planes. The concave–convex shape of the hiatus can be observed.

Agreement and reliability

The conventional 3D ultrasound and virtual reality levator ani hiatus volume measurements of all 100 patients were performed by one operator (L.S.) and repeated three times, which is standard in the clinical application of ultrasound measurements. The mean of the three levator ani hiatus volume measurements obtained using conventional 3D ultrasound and that obtained in virtual reality were used for comparison between the two methods.

For calculating interobserver reliability and agreement of both conventional 3D ultrasound and virtual reality levator ani hiatus volume measurements, 20 datasets of randomly chosen patients were selected. Investigators L.S. and A.B.S. independently performed three volume measurements of each dataset and the mean was used for comparison. We randomly selected these patients with the help of a research randomizer (http://www. randomizer.org/form.htm).

To assess intraobserver reliability and agreement of both conventional 3D ultrasound and virtual reality measurements, L.S. performed another three measurements in 20 randomly chosen datasets, and the mean of these measurements was compared with the mean of the three measurements previously obtained by L.S from the same 20 datasets. The second series of measurements was performed at least 2 weeks after the first series to prevent recollection bias.

Statistical analysis

Statistical analysis was performed using SPSS/PC version 15.0 (SPSS Inc., Chicago, IL, USA). Two-sided P < 0.05 was considered to be statistically significant. The same statistics were used to compare the virtual reality and conventional 3D measurements, as well as to determine inter- and intraobserver variability. The mean difference (95% CI), limits of agreement (mean difference ± (1.96 × SD)) and intraclass correlation coefficients (ICC) were calculated. Bland–Altman plots were used to determine whether the difference was influenced by the magnitude of the measurements20–22. An ICC of 0.81–1.00 was considered to reflect excellent reliability23, 24.

Results

The baseline clinical characteristics of all the women who underwent pelvic floor ultrasound imaging are presented in Table 1.

Table 1. Baseline characteristics of the 100 symptomatic women attending tertiary pelvic floor clinic and included in the study
CharacteristicValue
  1. Data are given as median (range) or n (%).

Age (years)57 (22–79)
Nulliparous7 (7)
Urinary incontinence (all types)21 (21)
Prolapse complaints13 (13)
Fecal incontinence19 (19)
Anal sphincter rupture during childbirth4 (4)
Fistula2 (2)
Evacuation problems2 (2)
Pain7 (7)
No urogenital complaints6 (6)
Combination of urinary incontinence, prolapse complaints, evacuation problems and fecal incontinence26 (26)

The mean levator ani hiatus volume measurements of all 100 patients on conventional 3D ultrasound and virtual reality were 21.65 ± 3.38 cm3 and 21.75 ± 4.79 cm3, respectively. Table 2 shows the results of the comparison between conventional 3D ultrasound and virtual reality levator ani hiatus volume measurements. The mean difference between the two measurement techniques was not significantly different from zero and good agreement was observed, with an ICC of 0.968. Figure 4 illustrates the relationship between conventional 3D ultrasound measurements and virtual reality measurements in all 100 patients, with the Bland–Altman plot showing no variation in differences between the two techniques in relation to the magnitude of the measurements.

Figure 4.

(a) Scatterplot showing relationship between conventional three-dimensional (3D) ultrasound measurements and virtual reality measurements in all 100 symptomatic patients attending a tertiary pelvic floor clinic. equation image, Line of equality. (b) Bland–Altman plot showing agreement between conventional 3D ultrasound and virtual reality levator ani hiatus measurements in all 100 patients. Mean (equation image) and 95% limits of agreement (equation image) are shown.

Table 2. Results of comparison between conventional three-dimensional (3D) ultrasound and virtual reality levator ani hiatus volume measurements and intra- and interobserver reliability and agreement of the two measurement techniques
ComparisonnMean (SD) (cm3)*Mean difference (95% CI) (cm3)95% limits of agreement (cm3)ICC (95% CI)
  • *

    Mean of three measurements.

  • Mean difference = first minus second measurement of L.S. in intraobserver analysis and mean difference = first measurement of L.S. minus measurement of A.B.S. in interobserver analysis.

  • Limits of agreement = mean difference ± (1.96 × SD).

Conventional 3D ultrasound vs virtual reality10021.70 (4.92)0.10 (–0.15 to 0.35)− 2.36 to 2.560.968 (0.952–0.978)
Intraobserver repeatability of 3D ultrasound measurements2022.16 (5.76)− 0.90 (−1.69 to − 0.10)− 4.23 to 2.430.948 (0.851–0.980)
Intraobserver repeatability of virtual reality measurements2020.56 (4.07)0.12 (−0.21 to 0.45)− 1.26 to 1.510.986 (0.965–0.994)
Interobserver agreement of 3D ultrasound measurements2022.06 (5.79)− 0.71 (−1.38 to − 0.44)− 3.52 to 2.090.964 (0.899–0.986)
Interobserver agreement of virtual reality measurements2020.49 (4.04)0.27 (−0.13 to 0.67)− 1.40 to 1.940.977 (0.944–0.991)

The intra- and interobserver reliability and agreement of the two measurement techniques (n = 20) are also shown in Table 2, with good agreement observed for all comparisons. The intra- and interobserver ICCs for conventional 3D ultrasound levator ani hiatus volume measurements in the 20 randomly chosen datasets were both > 0.94 and those for virtual reality measurements were > 0.97.

Discussion

This study indicates that there is an excellent ICC value for levator ani hiatus volume measurements performed with conventional 3D ultrasound and with virtual reality. This investigation demonstrates that the conventional 3D ultrasound measurements do not differ significantly from the virtual reality volume measurements during contraction. We also conclude that there is excellent intra- and interobserver reliability of both conventional 3D ultrasound and virtual reality levator ani hiatus volume measurements. The mean intra- and interobserver differences of conventional 3D ultrasound levator ani hiatus volume measurements show some bias, but as indicated by the ICCs this does not affect reliability. The means of the second volume measurements in conventional 3D ultrasound of L.S. are smaller than the means of the first, and the volume measurements of A.B.S. are smaller than the first measurements of L.S. This bias maybe explained by the fact that A.B.S. is a more experienced investigator of the levator ani and can therefore better determine the tissue/hiatus interface of the levator ani; over time L.S. became more experienced, explaining the smaller measurements the second time.

We believe that the small mean inter- and intraobserver differences of 0.71 cm3 and 0.90 cm3 that we found in this study for conventional 3D ultrasound do not have any clinical significance, as our mean hiatal dimension was 21.7 cm3 (14.47 × 1.5). These results are comparable with those of previous studies. Shek and Dietz25 found a mean hiatal dimension during contraction of 11.6 ± 2.24 cm2 in 296 nulliparous women and Braekken et al.26 showed a mean hiatal dimension at contraction of 14.70 cm2 in 17 healthy volunteers.

It is notable that the ICC values for both the intra- and interobserver 3D ultrasound measurements are lower than those ICC values for the virtual reality measurements, indicating a better reliability for measurements with virtual reality. This may be explained by the fact that discrimination of the pubic bone and the inside margins of the levator ani hiatus are better visualized in virtual reality. In contrast to subjective selection of the margins of the levator ani using conventional 3D ultrasound, in virtual reality a large portion of the levator ani hiatus was selected by ‘the computer system’ automatically planting a seed point. It is questionable whether these differences between ICC values are clinically relevant.

To study the reliability and agreement of both measuring techniques we chose to select volumes obtained on maximal levator contraction, because these are usually well defined volumes. Also, we considered that in contraction the convex–concave shape of the levator hiatus would be more clearly outlined. Because of this, we expected to obtain a larger difference between conventional 3D ultrasound and virtual reality measurements. However, this study shows only small differences, which were not systematic and not statistically significant. So, although we had assumed that we would detect larger differences between conventional ultrasound and virtual reality it might be that the shape of the levator hiatus in contraction is not the best volume for detecting these differences.

That our study only focused on volumes obtained during contraction could also be considered a limitation. For clinical use, measurements of volumes obtained on Valsalva maneuver are of more importance, i.e. for assessing the risk of developing prolapse and determining the risk of developing recurrent prolapse3, 27. Therefore further investigation is necessary to see whether volumes obtained on Valsalva might show larger differences between the two methods. Another factor to be considered is that the results found in this study reflect the fact that the current measurements performed using ultrasound display a good representation of the non-Euclidean shape of the levator hiatus.

Previous studies for determining the reliability of measurements during contraction of the levator hiatus in conventional 3D ultrasound have been done. Majida et al.24 showed an interobserver ICC of 0.92 with limits of agreement of − 4.05 to 3.13 in 17 healthy women. Chen et al.28 found an interobserver ICC of 0.807 (95% CI, 0.581–0.918) in 96 patients for the levator hiatus area during contraction on conventional 3D ultrasound. Braekken et al.26 showed an intraobserver ICC of 0.79 (95% CI, 0.50–0. 92) of the levator hiatus area during contraction on conventional 3D ultrasound. Our interobserver ICCs and limits of agreement are comparable with those found by Majida et al.24, but better than those reported by Chen et al.28. Our intraobserver ICCs are a lot higher than those of Braekken et al.26. This may be explained by the small number of women investigated.

No previous studies comparing conventional 3D ultrasound with virtual reality for levator ani hiatus volume measurements during contraction have been reported. In addition, no previous studies to determine the reliability and agreement of levator ani hiatus volume measurements during contraction using virtual reality have been performed. A drawback of this study is that, unfortunately, CAVE-like virtual reality systems are currently only available in a very limited number of research centers throughout the world. For this type of measurement to become clinically feasible, smaller, low-cost desktop virtual reality systems will have to be introduced in the hospital environment. Further investigation is also needed to observe possible differences at rest and during Valsalva maneuver, in women with abnormal levator ani anatomy, and their relationship with pelvic floor symptoms.

Acknowledgements

We would like to thank W.C.J. Hop, who provided advice concerning statistics and interpretation of results.

Ancillary