Clinical importance of re-interpretation of PET/CT scanning in patients referred to a tertiary care medical centre

re-interpretation


Introduction
Patients referred to our tertiary care medical centre for treatment frequently present with imaging studies that were obtained elsewhere, and the referring physicians or surgeons often request an 'in-house' interpretation of these 'outside' imaging studies.
At our Department of Clinical Physiology, Nuclear Medicine and PET at Rigshospitalet, these 'in-house' interpretations or re-interpretations of outside PET/CT scans are named 'second opinions'.When a 'second opinion' is requested, the images are required including a copy of the initial local report to ensure that no finding mentioned in the initial report is overlooked in our re-interpretation.Second opinions are typically requested before surgery in patients with lung cancer, mesothelioma, gynaecological tumours, liver tumours and prior to treatment of lymphoma.We have noted important differences in the in-house and outside image interpretations, and therefore, we initiated this study.
First we compared 159 clinical re-interpretations of external PET/CT scans to the original reports and found 'major discordance' in 26% of the cases (unpublished data), and we evaluated our in-centre interobserver agreement on 100 internal PET/CT scans and found 'major discordance' in only 5% of the cases (unpublished data).Then, we decided to evaluate this properly, as the results of this open analysis were important and controversial, and our initial open non-blinded methodology could be criticized.The importance was also warranted by the fact there are no reports in the literature on second opinions on PET/CT.
The aim of this study was to evaluate in a controlled prospective manner with double-blind read whether there are differences in PET/CT image interpretations of PET/CT scans at our tertiary medical centre, Rigshospitalet, compared to the external hospitals.We used the same original images and the same clinical information, and we did not use the initial PET/ CT interpretation for our in-house interpretation.Histopathologic analysis, surgical findings and clinical follow-up served as standard of reference.

Patients
From March 2012 through June 2012, 90 consecutive patients referred to our department who had an external F-18 fluorodeoxy-glucose (FDG) PET/CT scan were included in this study.Patients from abroad were excluded due to follow-up issues.The local ethics committee classified the study as Quality Control & Quality Assurance and therefore waived the requirement for informed consent.
The characteristics of the study population regarding indications for referral are summarized in Table 1.This material reflects the heterogeneity in our clinical practice.The study population included patients with malignancy or suspicion of malignancy in all but three cases where the indication for referral was infection (N = 2) and sarcoidosis (N = 1).The most frequent malignancy was colorectal cancer (N = 23).The mean age of the patients was 65 years (range 24-91 years).The PET/CT scans had been performed at nine different hospitals in Denmark, with 28 studies performed at a nearby hospital and only one study from the two most distant hospitals.Seventeen PET/CT scans (19%) were performed only with lowdose CT and the remaining 73 (81%) were PET/CT scans with diagnostic CT quality, and the vast majority of the scans were contrast enhanced.

Study design
In the daily clinical routine, a copy of the original PET/CT report is always available to our nuclear medicine physicians and radiologists at the time of re-interpretation of the scan to ensure that no finding noted in the initial report is overlooked.Sometimes, histopathology, pertinent imaging or clinical data are also available.To avoid this information bias in our study, we engaged an independent medical student, hereafter called data controller (DC).The DC made sure that only the information that had been available at the time of the original interpretation at the external hospital was available at our re-interpretation.This included the removal of the original report and all information that was not available when the original report had been performed.The DC also made sure that the nuclear medicine physician and radiologist doing the re-interpretations had not been involved in the previous clinical re-interpretation of the scan.

PET/CT image interpretations
Teams with one radiologist and one nuclear medicine physician working side by side, all experienced in reporting PET/ CT scans, performed the re-interpretation in consensus, identical with our daily routine clinical practice.In total, five nuclear medicine physicians with a minimum of 5 years of experience of PET/CT reporting and five radiologists were involved in the reporting (Our department performs 7000 PET/CT studies annually).The nuclear medicine physician viewed the cases with TrueD software on a Leonardo workstation (Siemens, Forchheim, Germany), while the radiologist used an Agfa Impax 5.2 (Agfa-Gevaert NV, Mortsel, Belgium) PACS client.

Patient diagnosis
Two oncologists and in selected cases also a clinician within another relevant specialty subsequently and independently compared the original reports with the reinterpretation reports and graded these as 'accordant', 'minor discordance' or 'major discordance'.'Major discordance' was defined as findings which would affect clinical management.For example, a lung lesion initially interpreted as benign but re-interpreted as highly suspicious for lung metastasis (Fig. 1).'Minor discordance' included findings unlikely to alter patient care or irrelevant for the further clinical course, for example, when an adrenal metastasis is overseen in a patient with disseminated disease (Fig. 2).In case of 'major discordance' between the original report and the re-interpretation, the oncologists assessed the respective reports validities by obtaining histologic analysis of tissue obtained from biopsy or surgery, surgical findings, consequent imaging studies and clinical follow-up.Mean follow-up time was 588 days.
All cases were part of our clinical routine for 'second opinions', and therefore, a final re-interpretation was performed using all relevant clinical information and the initial PET/CT report to secure the patient an optimal treatment.These final 'official' re-interpretations were given to the clinicians and were the basis for the actual clinical management and were not part of this study.Exact binomial 95% confidence intervals (CIs) were used to evaluate the reliability of the rates of change.
Among the 17 cases, in which 'major discordance' occurred, the following diagnoses were the most common; pulmonary cancer (3), colorectal cancer (3), sarcoma (2) and lymphoma (2).In 11 (65%) of the 17 cases graded as 'major discordance', it was possible, after reviewing clinical information including operation descriptions, pathology report, subsequent imaging studies and clinical follow-up, to determine which report that was most correct.In six cases, this was not possible to determine; four patients died within less than 4 months postscan date, one patient had no operation or biopsy due to subsequent dissemination to other sites, and for one patient, histopathology report was inconclusive, and no additional biopsy was performed.Results from those 11 cases where it were possible to determine which report that was most correct are summarized in Table 2.In 9 of the 11 cases (82%), the re-interpretation was most correct, and in one case (9%), the original report was most accurate.In the last one case (9%), both interpretations were incorrect, meaning that both differed significantly from the final diagnosis.

Discussion
In tertiary referral hospitals, official re-interpretations of externally obtained imaging studies are common within both radiology and nuclear medicine to reduce rate of repeat imaging (Lu & Tellis, 2012).At Rigshospitalet, a tertiary referral hospital in Copenhagen, Denmark, re-interpreting of PET/CT scans (i.e.second opinion) is typically required in the context of a multidisciplinary team meeting.Frequently, the change in interpretation in this setting results in a change of patient management.
In the literature, we have not found any study of second opinions on PET/CT, but there are several studies on the difference between the original report and second opinion in radiology.G. J. Loughrey found a major difference in interpretation in 34% of 536 CT and MRI scans, which was assessed by an oncology specialist radiologist (Loughrey et al., 1999).Specialist radiology review changed radiological staging in 19% of cases.Loevner et al. (2002) assessed the clinical value of re-interpreting cross-sectional imaging studies of patients with head and neck cancer, in the setting of a multidisciplinary cancer centre.One hundred and thirty-six patients' CT and MRI scans were re-interpreted by a neuroradiologist.In 56 (41%) cases, there was a change in interpretation, which ultimately altered treatment in 55 (40%).Another study on second-opinion consultations in neuroradiology found a 7Á7% rate of clinically important discrepant interpretations (347 of 4534 studies) (Zan et al., 2010).When a definitive diagnosis was obtainable, the second-opinion consultation was more accurate in 84% of studies.A study on 773 secondopinion interpretations by specialty radiologists at a tertiary care children's hospital in an unselected paediatric population The lung lesion was initially interpreted as benign but re-interpreted as highly suspicious for lung metastasis.Subsequent imaging and clinical follow-up confirmed that the lesion represents a lung metastasis.
Figure 2 Positron emission tomography/CT shows a metastasis to the right adrenal gland (arrow) in addition to multiple lung and bone metastases in a 72-year-old female patient 5 years after radical surgery for colorectal cancer.The metastasis to the right adrenal gland was overseen at the initial report, but this had no influence on this patient's further clinical course.noted major disagreement rates of 14Á3% and 32Á6% for neurologic and body examinations, respectively (Eakins et al., 2012).Gollub et al. (1999) studied the clinical importance of re-interpretation of 143 CT scans obtained elsewhere on cancer patients referred for care at a tertiary cancer centre.They found 'major disagreement' in 24 patients (17%) and 'minor disagreement' in 29 patients (20%), and these results are consistent with ours.These studies are example of studies in which investigators have shown that re-interpretation of imaging studies by subspecialized radiologists can affect staging, management and potentially the prognosis in cancer patients.
In some of these studies, the original report had been available to the radiologist at the time for re-interpretation and, in many instances, the initial readings were likely reported without the benefit of the complete clinical history and sometimes newer findings on physical examination, including results from more recent histopathology reports.In this study, we investigated whether there are differences in expertise in interpreting PET/CT at a tertiary referral hospital compared to outside hospitals by removing all information bias, that is the same clinical information was available to the radiologist and nuclear physician as it was at the time for the initial reading.
Although we did not find any studies of second opinions on PET/CT, there are a few on interobserver variability.Hofman et al. (2009) examined the interobserver variability of PET/CT for staging of lymphoma and found that experienced observers at a centre in London had a high level of agreement using PET/CT for lymphoma staging.In another study by Barrington et al. (2010), four PET centres in Europe (including Rigshospitalet in Copenhagen) participated in an evaluation of PET/CT scans of 50 patients with Hodgkin's lymphoma before and after two cycles of chemotherapy.The scans were reported using a five-point scale to score the remaining activity.When score 1-2 was classify as 'negative' and 3-5 as 'positive', Barrington et al., found an agreement in 44 of the 50 patients (88%) at all four centres.
Our analysis has shown a difference in the quality of the PET/CT interpretations between the primary hospital and Rigshospitalet in Copenhagen with the same clinical information available.In 19% of cases, we noted a 'major discordance' meaning a discrepancy of a magnitude that would significantly affect clinical management.We find this surprisingly high.This figure could be compared to the results from the National Oncologic PET Registry (NOPR), which overall showed that physicians changed their intended management in 36Á5% (95% CI, 35Á9-37Á2) of cases after PET (Hillner et al., 2008).An important cornerstone for ordering a diagnostic scan must be that the patients and the treating physicians could be certain that the interpretation of a PET/CT is correct, independent of where it has been undertaken.
The fact that the re-interpretation at Rigshospitalet was most correct in 82% of the cases where it could be determined, excluding six cases where it could not, and that only one case (9%) showed the original report to be most correct, indicates that quality improvements should be considered, with special focus on departments who have problems with the validity of their PET/CT interpretations.
There are a few limitation of our study to generally apply these results to all PET/CT reports.First, only reports in which an official second opinion was requested were included meaning that in most cases there had been significant and relevant findings in the original report.Therefore, the discrepancy rates may be higher than if the study had included all PET/CT reports.Second, reports from nine outside hospitals with very varying experience both in throughput and years of performing PET/CT were included.For example, one hospital only had a 1-year experience with PET/CT and performed at the time of the study less than 1000 scans annually.At Rigshospitalet, all readers in this study had a minimum of 5 years of experience of PET/CT reporting.The individual readers' experience at the outside hospitals, which most probably influences the quality of the reports, is not investigated due to the number of readers.
The higher rate of correct interpretations at Rigshospitalet is probably due to a combination of close contact between clinicians and nuclear medicine physicians/radiologists through multidisciplinary team meetings and research, in addition to the high level of competence and throughput with more than 7000 PET scans a year for several years.At most Danish hospitals, PET/CT scans are read by a team consisting of a nuclear medicine physician and a radiologist, but often this dual reading is performed first separately and then a consensus is formulated.At Rigshospitalet, the team read simultaneously, and it is possible that this better facilitates the discussion and knowledge exchange within the team.At the same time, up to three teams can be working side by side in the same reading room which makes expertise easy accessible for discussion.We also believe that our everyday case discussions could influence on the quality.This scenario is not always possible to set up in a hospital with few scans and readers.
Although there has been a tradition of attending Rigshospitalet before starting with PET/CT, we see this study as an argument for setting up a continuing educational system with focus on PET/CT interpretation of common diagnoses.
After this study was performed, there has been taken an initiative to harmonize PET/CT reporting in Denmark.In addition to this, all hospitals in the region are in 2015-2016 implementing a mutual regional PACS (picture archiving and communication system), simplifying access to images and reports.This study has encouraged us to advocate for routine feedback to the outside hospitals, and a mutual regional PACS makes this feasible.

Conclusion
In this study of 90 PET/CT scans performed at external hospitals, we found major discordance in 19% of cases when re-interpreted at Rigshospitalet, a tertiary referral hospital.The reinterpretations were performed without knowledge of the findings reported in the initial local readings or any other information not available at the time of the initial reading.Discordances were frequently seen in patients with lung cancer, colorectal cancer and sarcoma.Clinical follow-up showed that when there was major discordance, the re-interpretation at Rigshospitalet was most correct in 82%, indicating that there is a difference in expertise in interpreting PET/CT at a tertiary referral hospital compared to primary local hospitals.

Figure 1
Figure1Positron emission tomography/CT shows a lung lesion (arrow) in a 30-year-old female patient surgically treated for sarcoma.The lung lesion was initially interpreted as benign but re-interpreted as highly suspicious for lung metastasis.Subsequent imaging and clinical follow-up confirmed that the lesion represents a lung metastasis.

Table 1
Indication for referral versus accordance/discordance.
a Includes one patient each with ovarian cancer, prostate cancer, hepatic cancer, small intestine cancer, cholangiocarcinoma, breast cancer, melanoma, gastric cancer and sarcoidosis.© 2015 The Authors.Clinical Physiology and Functional Imaging published by John Wiley & Sons Ltd on behalf of Scandinavian Society of Clinical Physiology and Nuclear Medicine.37, 2, 143-147 Importance of reinterpretation of PET/CT, J. L€ ofgren et al.

Table 2
Cases with 'major discordance' where it was possible to determine which report was deemed most correct.
© 2015 The Authors.Clinical Physiology and Functional Imaging published by John Wiley & Sons Ltd on behalf of Scandinavian Society of Clinical Physiology and Nuclear Medicine.37,2, 143-147Importance of reinterpretation of PET/CT, J. L€ ofgren et al.