Automated quality control in nuclear medicine using the structured noise index

Abstract Purpose Daily flood‐field uniformity evaluation serves as the central element of nuclear medicine (NM) quality control (QC) programs. Uniformity images are traditionally analyzed using pixel value‐based metrics, that is, integral uniformity (IU), which often fail to capture subtle structure and patterns caused by changes in gamma camera performance, requiring visual inspections which are subjective and time demanding. The goal of this project was to implement an advanced QC metrology for NM to effectively identify nonuniformity issues, and report issues in a timely manner for efficient correction prior to clinical use. The project involved the implementation of the program over a 2‐year period at a multisite major medical institution. Methods Using a previously developed quantitative uniformity analysis metric, the structured noise index (SNI) [Nelson et al. (2014), \textit{J Nucl Med.}, \textbf{55}:169—174], an automated QC process was developed to analyze, archive, and report on daily NM QC uniformity images. Clinical implementation of the newly developed program ran in parallel with the manufacturer’s reported IU‐based QC program. The effectiveness of the SNI program was evaluated over a 21‐month period using sensitivity and coefficient of variation statistics. Results A total of 7365 uniformity QC images were analyzed. Lower level SNI alerts were generated in 12.5% of images and upper level alerts in 1.7%. Intervention due to image quality issues occurred on 26 instances; the SNI metric identified 24, while the IU metric identified eight. The SNI metric reported five upper level alerts where no clinical engineering intervention was deemed necessary. Conclusion An SNI‐based QC program provides a robust quantification of the performance of gamma camera uniformity. It operates seamlessly across a fleet of multiple camera models and, additionally, provides effective workflow among the clinical staff. The reliability of this process could eliminate the need for visual inspection of each image, saving valuable time, while enabling quantitative analysis of inter‐ and intrasystem performance.


| INTRODUCTION
In nuclear medicine (NM), a robust quality control (QC) program is essential for detecting detrimental changes in camera performance, allowing for remediation prior to clinical involvement. [1][2][3] Currently, one of the most valuable assessments is the uniformity evaluation, routinely performed on a daily basis prior to patient imaging. Many of the most significant performance issues will produce system nonuniformities which are effectively depicted in the uniformity evaluation image. Early identification of performance issues provides the operator a chance to initiate appropriate corrective action prior to patient imaging, preventing suboptimal clinical studies. However, in order to benefit from this valuable quality control evaluation, a suitable and reliable process must be in place.
Typically the traditional uniformity evaluation process involves image acquisition, followed by analysis comprised of a critical visual inspection, in addition to pixel value-based analysis. [4][5][6] While a critical visual inspection is considered the gold standard and should be performed prior to pixel-based analysis, 7 several downfalls exist.
First, it is time consuming, especially during the busy morning when the technologist is preparing for patients, potentially resulting in an insufficient inspection. In addition, it is also subjective; relying heavily on the expertise of the reviewer to determine which nonuniformities may have a clinical impact. In order to make the evaluation process more objective and to provide an additional perspective on image uniformity, a computer analysis program typically accompanies the visual analysis. However, the traditional pixel value-based programs often fail to adequately identify subtle structure and patterns, potentially biasing the reviewer to perhaps erroneously pass an image with visual nonuniformities.
In a previous project, a new uniformity analysis metric was developed, the structured noise index (SNI), which reports the uniformity of an image based on the image noise texture. 8 It was found that this metric outperforms currently established pixel value-based analysis methods for identifying image nonuniformities and additionally, correlates closely with expert visual analysis possibly reducing the need for visual assessment.
In this current project, we evaluate the practicality of integrating the SNI metric into the daily uniformity QC program, where oftentimes a qualified medical physicist is unable to visually analyze the quality of each flood image prior to patient studies. The goal of this project was to develop and implement a robust QC metrology for NM that is effective and reliable in identifying nonuniformity issues, effective in reporting issues in a timely manner for effective problem correction, and to characterize the program over a 2-yr period in an academic medical center setting.

2.A | Structured noise index
The SNI was developed in a previous project for quantifying nuclear medicine flood-field uniformity images with regard to the presence of nonuniformities, detailed in an earlier publication. 8 In summary, the SNI is based on frequency-based two-dimensional (2D) noise power spectrum (NPS), where I x i ; y j À Á is the image intensity at pixel location x i ; y j À Á , I is the global mean intensity, u and v are the spatial frequencies conjugate to x and y, N x and N y are the number of pixels in the x and y direc- Both the input NPS and structured noise NPS are further filtered with a 2D human visual response function using the equation where r is the radial spatial frequency and c is a scale factor selected to yield the maximum for the function at four cycles per degree at a typical viewing distance of 150 cm and typical image display size of 6.5 cm. The input flood SNI value is the ratio of the filtered NPS of the structured noise to the filtered NPS of the input image as given by the following equation The SNI metric was validated through an observer study, and further compared against some traditional pixel value-based unifor-

2.B | Automated analysis process
To effectively put the SNI analysis into clinical use, an automated process was developed to transmit, receive, analyze, report, and archive the daily uniformity images.

2.B.1 | Transmission
A physics network database was first created within the medical center network. This database was then entered as a "send" destination on each NM imaging system. After acquisition, each system sends the uniformity image to the physics database in DICOM format where they are then sorted based on physical and acquisition attributes contained in the DICOM header (i.e., station name, study type, number of counts, etc.). This sorting process helps avoid images which are not uniformity QC images from being analyzed by the program.

2.B.2 | Analysis
After the received images are properly sorted, the uniformity images are then analyzed using the SNI metric described above. As part of the SNI analysis, a four-quadrant figure shown in Fig. 1 is also gener-

2.B.3 | Reporting
Results of the SNI analysis as well as image acquisition attributes (DICOM header information) are populated into a database assess-

2.B.4 | Archiving
After the uniformity images are analyzed, they are automatically archived in separate folders based on acquisition station name.
These archived images provide the physicist the ability to view and further analyze the images via remote access. The four-quadrant jpeg images are also stored in separate folders based on station name.
Archival allows interrogation of the recent history of a detector for gradual developments.

2.C | Clinical validation
The utility of the program was tested for a period of 21 months at an academic medical center utilizing nine nuclear imaging systems The SNI program was also further upgraded to provide email alerts if the SNI value exceeded a user defined threshold or if daily QC images were not received. Similar to the first two phases, the NM technologists continued to visually inspect each uniformity image, but discontinued manual analysis using the GE UA program. However, they did record the system-generated uniformity percentage reported during the GE NM Daily QC Procedure.

2.D | Statistical analysis
The performance of the automated SNI uniformity analysis QC program was evaluated quarterly by calculating the sensitivity. A true positive was defined as an instance when the SNI metric returned a value exceeding the upper-level threshold and service was performed on the system due to image quality issues.
Instances when the threshold was exceeded and it was determined servicing was not necessary is a false positive. The sensitivity using the IU metric (generated from the GE UA program, or GE NM Daily QC program) was also calculated and reported for comparison purposes.
Variation in flood image quality was also evaluated using the coefficient of variation (CV) of the SNI which was calculated for each system on a quarterly basis. An overall average CV for each quarter was then calculated by dividing the average standard deviation of all systems by the average mean of all systems during the entire quarter. The SNI proved to be a better predictor of image quality issues than the IU metric determined by the sensitivity in identifying image quality issues at consistently low false positive rates. During the first phase of our SNI trial, the SNI successfully identified 100% of the seven image quality issues, while the IU metric identified only 43% of the issues. During the entire 21 months of our trial, the overall average SNI sensitivity was 92%, compared with 31% for the IU metric. The complete results of all three phases delineated by quarter are reported in Fig. 3.

| RESULTS
The three phases of the structured noise index clinical validation.  Although the majority of clinical imaging is performed with Tc-99m radiopharmaceuticals, traditionally daily quality control is performed using the more convenient Co-57 sealed sheet source.  (a) Structured noise index (SNI) and system generated Uniformity % for a single detector over a 6 month period. The red horizontal line indicates the threshold used for both metrics (5.0% for Uniformity % and 0.50 for SNI). Instances where service was performed due to visual image quality issues are represented by hollow markers. This particular system uses isotope specific uniformity correction maps. When a Tc uniformity map was acquired, the clinical image (using Tc-99m isotopes) quality became acceptable. However, the quality control (using Co-57) continued to exceed limits until the Co uniformity correction was able to be acquired. (b) Flood images corresponding to locations indicated in (a) with enhanced window/level settings to better visualize non-uniformities.

NELSON AND SAMEI
| 85 texture within the image in the area deemed unacceptable by visual inspection, however, the SNI value did not exceed the facility established trigger threshold. One may consider adjusting the trigger threshold to be more sensitive to image nonuniformities, however, this adjustment will lead to an increase in the false positive rate.
Our evaluation of the SNI metric in a clinical quality control setting was based on comparison with a pixel value-based metric (IU%).
Although our evaluation demonstrates instances where monitoring the SNI metric proved to be a better predictor of uniformity, it should be mentioned that monitoring the IU% or other QC strategies (e.g., energy peak or energy resolution) will likely complement each other in order to provide a more complete assessment of quality in the nuclear medicine operation. This topic merits further investigation, but is beyond the scope of this paper.

| CONCLUSION
Visual evaluation of NM daily uniformity QC images is time consuming, subjective, and prone to transcription and oversight (incomplete visual inspection) errors. Alternatively, the SNI provides a robust quantification of the NM performance of gamma camera uniformity in a more objective and quantitative fashion. Implementing across a large academic institution, it operates seamlessly across a fleet of multiple camera models. The automated alert process provides enhanced workflow between physicists, technologists, and clinical engineers. The reliability of this process paired with the high sensitivity of the SNI has made it the preferred platform for NM uniformity analysis, and could eliminate the need for visual inspection of each image.

ACKNOWLEDGMENTS
The authors are grateful to Olav Christianson for the early development of the methodology.

CONFLI CT OF INTEREST
No conflict of interest.