Image quality evaluation of ultrasound imaging systems: advanced B‐modes

Abstract The Quality assurance of ultrasound clinical imaging systems is essential for maintaining their performance to the highest level and for complying with the requirements by various regulatory and accrediting agencies. Although there is no standardization yet, most of the quality assessment procedures available in literature are proposed for B‐mode and Doppler imaging. However, ultrasound imaging systems offer a variety of advanced imaging modes, besides B‐mode and Doppler, which are primarily aimed at improving image quality. This study presents computer‐based methods for evaluating image quality for the advanced imaging modes of ultrasound imaging systems: harmonic imaging, spatial compounding imaging, adaptive speckle reduction, and tissue aberration correction. The functions and parameters proposed for evaluating image quality are: grayscale mapping function, image contrast, contrast‐to‐noise ratio (CNR), and high‐contrast spatial resolution. We present our computer‐based methods for evaluating image quality of these modes with a number of probe and scanner combinations, which were employed to image targets in ultrasound phantoms. The functions and parameters here proposed in image quality performance evaluation are: grayscale mapping function, image contrast, CNR, and high‐contrast spatial resolution. We show that these quantities could be useful in developing standardized methods for evaluating the advanced ultrasound imaging modes, especially when the advanced mode resulted in subtle visual differences.


European Federation of Societies for
Ultrasound in Medicine and Biology (EFSUMB), 6 and Institute of Physics and Engineering (IPEM). 7 These tests usually include: visual inspection of the components of the US system (scanner/probes), display monitor performance, image uniformity, sensitivity (maximum depth of visualization, signal-to-noise ratio), geometric accuracy, spatial resolution, and contrast resolution. In USA, QA of personnel qualifications and equipment consent hospitals to gain accreditation or re-accreditation of their practices by an accrediting agency (e.g., ACR, AIUM). In addition to B-mode and Doppler imaging, US imaging systems offer now a variety of advanced imaging techniques which are primarily aimed at improving image quality. These techniques include: harmonic imaging (HI), [17][18][19][20][21] spatial compound imaging (SCI), [21][22][23][24] adaptive speckle reduction (SR), 25,26 and tissue aberration correction (TAC), 26,27 also known as speed-of-sound correction.
Emerging clinical imaging modes include: elastography 28 and 3D/4D imaging. 29 For the accreditation process, no specific QA tests are currently required for Doppler imaging and for the advanced and emerging imaging modes. Therefore, it is not surprising that these tests are not normally performed in a routine QA of medical US equipment.
The difficulty in their implementation, likely due to a lack of scientific literature and to the high costs, is followed by lack of norms, standardization, and accreditation programs. Nevertheless, since the US technology is widespread, there is a great need to develop effective and reproducible QA tests also for the advanced modes and emerging imaging modes.
In a previous paper, 14 we discussed our computer-based QA tests for basic B-mode imaging. In this paper, we present examples of computer-based QA implementation for some US advanced modes. Since the advanced modes are aimed at improving image quality, image contrast, and high-contrast spatial resolution (HCSR) are usually evaluated for these modes. The image contrast test is aimed at establishing the ability of the US imaging system to detect subtle differences in the echogenicity of two targets. Image contrast is usually assessed using targets of known nominal contrast in a phantom [ Fig. 1(a)] at fixed ultrasound settings and is affected by both the operator-controlled settings and the subject contrast. HCSR is defined in Ref. [1], as the minimum distance resolvable between two identical point targets, which produces, for a given gain setting, a higher level of backscattering than their surrounding medium, permitting their individual identification. HCSR is usually tested using filaments within a tissue mimicking material embedded in an ultrasound phantom [ Fig. 1(b)].
Very few papers have addressed the issue of objective, computer-based methods for the advanced modes. 10,15,16 Computer based tests for assessing the difference in image quality of HI vs. B-mode imaging were published by Van Wijk and Thijssen. 15 Using a commercial phantom, they found no significant difference in image contrast between the B-mode and HI mode but they observed an improvement in lateral spatial resolution. Browne et al. 10 also investigated image quality for the HI and SCI modes using computerbased tests and a commercial phantom. They found that HI improves lateral spatial resolution and slice thickness as a function of depth but had no effect on contrast resolution and low-contrast spatial resolution (anechoic target detection). In a follow-up study, 16 they used a subcutaneous pig fat layer with a commercial phantom to introduce aberration clutter and saw an improvement in anechoic target detection with HI mode as compared to B-mode imaging but no improvement in image contrast resolution. In the same study, they also investigated the SCI mode. SCI was found to significantly improve image contrast resolution and anechoic target detection as compared to B-mode imaging but produced limited improvements in spatial resolution.

2.A | Phantoms
A general-purpose phantom (Model 040GSE, CIRS, Norfolk, VA, USA) and in-house manufactured aberration layer were used for the tests. The commercial phantom was made of a material (Zerdine, solid elastic water-based polymer) which mimics the average acoustic properties of soft tissue: average speed of sound (1540 m/s) and attenuation either 0.5 or 0.7 dB cm −1 MHz −1 ). Embedded in the Zerdine material, were targets of known depth and size, which were designed to test the probe plus scanner combination performance.
The grayscale (contrast) targets were used to determine image contrast [ Fig. 1(a)]. They are cylinders of known diameters and location, having known (nominal) contrast, as compared to the surrounding background material, of: −9, −6, −3, 3, 6 dB, and a hyperechoic target (>15 dB), with an accuracy usually no lower than ±1 dB. The background contrast had a range of ±1 dB. The point spread function (PSF) targets were horizontal and vertical filaments of polymeric material (nylon) with nominal diameter 0.1 mm which were employed to determine the spatial resolution. The commercial phantom cannot mimic image degradation observed in images of real tissue. Therefore, we created a phase aberration layer using silicone elastomer 2 (Sylgard 184; Dow Corning) to test the TAC mode. This layer is shown in Fig. 2. Its thickness was made to vary between 0.9 and 2 mm in a sinusoidal pattern. This pattern repeats three times over its length. To create the layer a mold was fabricated which consisted of a 10 cm × 10 cm acrylic tray with removable sides. The lower surface of the mold was fitted with thin plastic pieces that were cut to create an undulating upper surface using a laser cutter.
The mold was sprayed with a release agent (CRC Industries) to allow removal of the layer after setting. The base and curing agent of the elastomer were mixed (10:1 ratio) and degassed to remove visible bubbles. The mixture was then slowly poured into the mold to avoid entraining air and allowed to set at room temperature. After setting,

2.B | US equipment and settings
Examples of our QC tests are illustrated with the GE LOGIQ E9 system for SCI, HI, SR modes, and with the Philips iU22 system for the TAC mode. With the LOGIQ E9 ultrasound scanner, we used the 9L-D probe which is a linear array with bandwidth of 3.33-10.0 MHz, and the convex probe C1-6-D with bandwidth 1-6 MHz. With the iU22 system, we employed the high-frequency linear probe L17-5 probe with bandwidth 5-17 MHz. Both the contrast targets and the PSF filaments were at first imaged in B mode and then in sequence, each advanced mode was turned on, while keeping the same settings as the ones established for B mode. With the GE system, we chose the prostate preset. This is because among the display maps available for this preset, there is map E, which is closest to linear, assigning the different brightness levels (representing the echo strength of each target) equally throughout the range of pixel values (0-255). With the Philips system, we chose the breast preset and map 5, which is also closest to linear. In addition, we kept the image processing to the minimum. Features such as frame average (or persistence, temporal filter that averages frames together), rejection (selects a level below which echoes will not be amplified) and suppression (suppresses the noise in the image) were kept to zero and line density was kept to four. The output power percentage was kept to 100, and only one focus was chosen with focus width 1.
The remaining parameters: depth, frequency, dynamic range, and gain were adjusted for best visualization of the contrast targets and the filaments as summarized in Table 1.
The weighted mean and weighted SD values of the pixel values,

2.C.2 | Contrast-to-noise ratio
Another important parameter of performance is the Contrast-tonoise ratio (CNR) 8,32 where hE T iand σ hET i are the weighted mean and SD for the target as defined in Eqs. (1) and (2), hE B iis the weighted mean of a region of F I G . 7. The slope (contrast) of the GMF curves shown in Fig. 5 for the convex probe. background in the phantom of same size and depth as the target ROI, and σ hEBi is the corresponding weighted SD. Special care has to be taken to select the background ROI. This is because the background weighted mean hE B i changes significantly from the center location of the image to the border. For best accuracy, it is recommended to choose the background ROI next to the target. 1 Since there is not enough background space between targets in the phantom, we selected a ROI for both target and background a bit smaller than the one we selected for contrast determination. For the linear probe, we have chosen an ellipse of size 57 × 52 pixels. An example is shown in Fig. 3(b). The CNR data were then plotted as a function Therefore, the obtained CNR has limited statistics validity.

2.C.3 | High-contrast spatial resolution
The HCSR is determined in the scan plane along the beam propaga-

3.B | CNR
The CNR (Eq. 3) is shown in Fig. 8 for the linear probe and in Fig. 9 for the convex probe. For both the linear and convex probes, CNR is better for the negative contrast targets than the positive ones for all the modes, with the worst CNR for +3 dB target. This is clearly visible with the naked eye and the CNR analysis reproduces it correctly.
The comparison among modes is however limited by low statistics.
Nevertheless, the CNR plots suggest the following. For the linear probe, SCI mode has a better CNR, especially for the negative contrast targets. The CNR for the convex probe is worse than for the linear probe. As may be seen in Fig. 9, the CNR values for the convex probe are closer than the corresponding ones for the linear probe for all the modes. The HI mode has a somewhat better CNR than SCI except in the region where contrast is highest for SCI mode.

3.C | HCSR
For the HCSR test, it is recommended to select the filament closest to the elevation focus. However, since the depth of the elevation focus was not established at acceptance testing for our probes, we have chosen for the linear probe, the vertical filament located at the depth of about 2 cm and for the convex probe, the one located at the depth of about 4 cm. For each mode, images of the chosen filament were acquired five times for the linear probe and ten times for the convex probe as the data were noisier. The filament highest pixel value was determined and the lateral and axial pixel lines passing through it were selected as described in Section 2.C.3. Representative plots of the pixel values for these lines may be seen in Fig. 10(a) for the lateral direction and Fig. 11(a)

3.D | Tissue aberration correction mode
For evaluating the TAC mode, we performed the HCSR test using the Philips iU22 scanner with the high-frequency linear probe L17-5.
The pixel size was estimated to be 0.03 mm. As described earlier, the image processing settings were kept to the minimum. Five images of the filament located at a depth of 2 cm were acquired in B and TAC modes with and without the aberration layer. The results for the FWHM are summarized in Fig. 15. At first, filament images were obtained in B mode without the aberration layer and then with the layer. Owing to the high frequency of the probe, the resolving power of the US system is high, as may be seen in Fig. 15(b), where the FWHM along the axial direction is close to the actual size of the filament, which is 0.1 mm. Then, the layer was placed on top of the phantom with ultrasound gel on the top and bottom of the layer to ensure the best coupling; care was taken to avoid trapped air bubbles between the layer and phantom. As may be seen in Fig. 15, the presence of the layer only slightly increases the FWHM in the axial and lateral directions, with the effect being stronger in the axial direction than in the lateral one. Next, the TAC 2 mode was turned on and the filament image appeared strongly distorted with the image of the filament cross-section squeezed along the axial direction and enlarged along the lateral direction [ Fig. 14(a)]. As a result, the FWHM along the axial direction is smaller than the corresponding one in B mode and the FWHM in the lateral direction is larger. Then, the layer was placed on top of phantom. The presence of the layer, although still defocuses the filament image, corrects somewhat the filament distortion [ Fig. 14(b)]. This may also be seen in Fig. 15, where the FWHM along the axial direction is increased, correcting in part for the squeezing of the filament, and along the axial direction the FWHM is decreased. This effect may arise because the speed of sound in the layer is closer in magnitude to the speed of sound assumed in the TAC 2 mode.

| DISCUSSION
We have presented computer-based methods for estimating the fol- The examples provided in this study are aimed to illustrate our computer-based QA method. The preliminary results of these examples suggest that the GMF for the linear probe is a sigmoid curve with the highest contrast in the most linear portion of the curve.
This remains true also for the convex probe; however, for this probe the linear portion of the GMF is greater. The above results suggest that the selected parameters can be useful in a quality assessment protocol of HI, SCI, SR, and TAC due to their sensitivity to ultrasound image characteristics and distortions. Moreover, a phase aberration layer could be used in addition to commercial ultrasound phantoms for testing how well the ultrasound system corrects for artifacts. The goodness of the system may be related to the characteristics and number of aberration layers whose overall distortion effect cannot be properly compensated in the diagnostic image. To this aim, a threshold on the tolerable distortion may be established. In this regard, future studies can be focused on the design and the development of a set of standardized phase aberration layer to be applied on a commercial ultrasound phantom. Nevertheless, addition studies are required in order to determine the reliability and robustness of the proposed QA method. For example, we know that the above parameters of performance are affected by the scanning conditions and for this reason we kept these conditions as similar as possible. However, we have not investigated in a systematic way to which degree the scanning conditions affect our results. It also important to investigate the inter-observer reproducibility and this aspect was not considered at all in this study. In addition, the results of this paper are based on a few measurements (5-10 for each mode) and for improving the statistics, a feasible option is to automate data analysis so that many more images for each mode can be quickly analyzed. Analyzing many images using automation could be also useful in reducing the effect of variability associated with slightly different scanning conditions and the effect of inter-observer variability.

| CONCLUSION S
Performances of advanced US imaging modes currently available in US imaging systems such as HI, SCI, adaptive SR, and TAC modes, can be quantified by evaluating parameters of image quality, such as GMF, image contrast, CNR, and HCSR. In this paper, a first evaluation of the aforementioned parameters is proposed and some preliminary image analysis have been shown without and with a phase aberration layer applied on the scanning window of a commercial ultrasound phantom. Despite the limitations of this study, some general conclusions can be drawn: the adaption and application of B-mode image quality parameters to ultrasound advanced imaging is promising and feasible due to their good sensitivity to the ultrasound image characteristics and because they are well known in the scientific community. Moreover, a cheap and user-friendly tool such as a phase aberration layer, may be used in addition to other common test objects, i.e., commercial ultrasound phantoms, for testing the ultrasound system capability to correct for artifacts. This may be useful to clinical medical physicists and technicians in testing the performance of US imaging systems before new algorithm or acquisition modes are employed in clinics.

CONFLI CTS OF INTEREST
No Conflict of interest.