Department of Surgery, University of California San Francisco, San Francisco, CA
Address reprint requests to John Paul Roberts, M.D., Division of Transplantation, Department of Surgery, University of California San Francisco, 505 Parnassus Avenue, Box 0780, San Francisco, CA 94143-0780. Telephone: 415-353-8783; FAX: 415-353-8709; E-mail: email@example.com
If you can't find a tool you're looking for, please click the link at the top of the page to "Go to old article view". Alternatively, view our Knowledge Base articles for additional help. Your feedback is important to us, so please let us know if you have comments or ideas for improvement.
Adult-to-Adult Living Donor Liver Transplantation Cohort Study
donor risk index
incidence rate ratio
Model for End-Stage Liver Disease
Organ Procurement and Transplantation Network
Scientific Registry of Transplant Recipients
United Network for Organ Sharing
The Organ Procurement and Transplantation Network (OPTN)/United Network for Organ Sharing (UNOS) database contains information on all transplants occurring in the United States since 1986, and it is currently the most comprehensive collection of liver transplantation data available for analysis in the United States and arguably in the world because few data on a center-specific level are available in other countries. Transplantation centers must submit forms detailing yearly organ recipient follow-up, including posttransplant outcomes such as malignancies and causes of death, with the requirement that these forms be completed in a timely manner. The content of these forms, however, is not validated except by individual comparison with the Social Security Death Master File. Additionally, although the most recent UNOS policy puts forth stringent guidelines for reporting information about a recipient's explanted hepatocellular carcinoma (HCC), it contains no mandated follow-up protocols. Thus, there is no formal incentive for centers to report recurrence data accurately and completely, and this leads to the question whether some centers are underreporting and, therefore, exhibiting recurrence rates lower than the actual rates.
The past few years have seen efforts to assess the quality of data in OPTN/UNOS databases. In particular, a 2007 comparison with private insurance claim data sets demonstrated some reporting discrepancies in patient characteristics. Claim comparisons, however, are confounded by the fragmented nature of the private insurance sector: it is difficult to get a data set representative of the patients in the UNOS database.
A more recent analysis compared the OPTN/Scientific Registry of Transplant Recipients (SRTR) database to the Adult-to-Adult Living Donor Liver Transplant Cohort Study (A2ALL) database, which is an independent collection of source data from 9 major transplant centers that has been validated by comparison with national outcomes. Several parameters were found to be significantly different between the 2 data sets; perhaps the most striking was the cold ischemia time. HCC recurrence, however, was not found to be significantly misreported. For patients who received transplants for a diagnosis of HCC (n = 111), 9.0% of the recurrence data were missing from the OPTN database but were present in the A2ALL database, 3.6% were missing from the A2ALL database but not from the OPTN database, and 2.7% were missing from both. When values were present in both data sets, they were inconsistent 8.1% of the time: 7% were recorded as yes in the A2ALL database and as no in the OPTN/SRTR database, and 1% were recorded as no in the A2ALL database and as yes in the OPTN/SRTR database. The study concluded that the discrepancy was symmetric between the OPTN/SRTR and A2ALL databases and suggested that although OPTN data were less carefully collected, HCC recurrence was not systematically underreported.
Because the OPTN database is the best source of data available for analyzing current transplantation practice on a multicenter basis, it is important to establish the validity of the outcome data found within it. In this study, we evaluated the reliability of posttransplant HCC recurrence data in the OPTN/UNOS database by comparing observed recurrence rates to expected rates calculated with a risk-adjusted Poisson model.
PATIENTS AND METHODS
Adults who were listed for primary liver transplantation with an initial exception granted for an HCC diagnosis meeting policy 184.108.40.206 criteria (stage T2) between January 1, 2006 and September 30, 2010 and who underwent transplantation within the same time period were identified from the UNOS Standard Transplant Analysis and Research files (created on March 2, 2012). Patients who died after transplantation because of cholangiocarcinoma were excluded from the analysis because HCC was likely misdiagnosed during the initial evaluation (n = 4).
HCC recurrence, defined as a posttransplant HCC-related death or a diagnosis of HCC recurrence, was determined by a physician's review (J.P.R.) of primary and contributory causes of death or an indication of recurrence in the malignancy follow-up data. The follow-up time after liver transplantation was defined as the number of years from liver transplantation to the first of HCC recurrence, death, or last follow-up. For patients subsequently receiving a second or third liver transplant, the follow-up time was evaluated from the date of first transplantation to the first event after retransplantation (HCC recurrence, death, or last follow-up after retransplantation). The posttransplant vital status and follow-up date were updated when valid data from the Social Security Death Master File were available.
Hepatoma was designated as the primary diagnosis for 34% of the patients. To identify the underlying liver disease for patients with a primary diagnosis of hepatoma, the secondary diagnosis at listing and the diagnosis at transplant (when a secondary diagnosis was unavailable or was also hepatoma) were evaluated. Patients with only a diagnosis of hepatoma and evidence of viral hepatitis (seropositive for hepatitis C virus or positive for hepatitis B virus surface antigen) were categorized by their viral hepatitis diagnosis.
Frequency distributions and medians and interquartile ranges (IQRs) for recipient, donor, and tumor characteristics were described for the total population and by outcome (HCC recurrence versus no reported recurrence). We calculated the tumor volume in cubic centimeters as the volume of a sphere (4/3 × π × tumor radius3; the tumor radius was half of the reported tumor size). For patients with multiple tumors, we summed the volumes of all tumors. The donor risk index (DRI) was calculated in accordance with Feng et al.
To predict the expected number of HCC recurrences by transplant center, we developed a risk-adjusted Poisson model with patients as the unit of analysis, random effects for transplant centers, and years of follow-up as an offset. To select risk adjustment variables, we first estimated single predictor Poisson models for the effects of recipient and tumor characteristics and DRIs on HCC recurrence. Variables with P < 0.1 were evaluated in the multivariate model. All variables remained statistically significant (P < 0.05) except for recipient sex, which was removed from the model. Incidence rate ratios (IRRs) and 95% confidence intervals (CIs) for each covariate were calculated from the multivariate model coefficients. Next, the ratio of observed HCC recurrences to expected recurrences for each transplant center was estimated with the best linear, unbiased prediction of its random effect. This approach differentially shrank the ratio for small centers toward 1, which reflected imprecision due to small numbers. The exponentiated random effect comprised a shrinkage estimate of the ratio of observed recurrences to expected recurrences for each center. We computed both 95% and 90% CIs for the shrinkage observed/expected ratios. Centers with upper confidence limits less than 1 would be identified as potentially underreporting HCC recurrence.
To further investigate the possibility of HCC recurrence underreporting, we imputed the expected value of the outcome for observations with non-HCC deaths on the basis of the first run of the model. This identified patients likely to have experienced unreported HCC recurrence and reclassified the expected proportion of these patients at each center to the HCC recurrence outcome group. We then reran the Poisson model with these imputed values, and we rechecked for centers with shrinkage observed/expected ratios significantly less than 1.
The data manipulation and analysis were completed with SAS 9.3 (SAS Institute, Inc., Cary, NC). Poisson regression modeling was completed with Stata/IC 11.1 (StataCorp, College Station, TX). This study received approval from the committee on human research at the University of California San Francisco.
HCC liver transplant recipients (n = 5034) were primarily male, white, and nondiabetic. Hepatitis C virus was the most common diagnosis (62.1%). At transplant, the patients had a median age of 57 years (IQR = 53-62 years) with a laboratory Model for End-Stage Liver Disease (MELD) score of 12 (IQR = 9-16; Table 1). When patients were granted an HCC exception, the median tumor volume for all tumors combined was 9.2 cm3 (IQR = 5.6-17.2 cm3), with 43.3% of the patients undergoing ablative therapy and 5.9% having an alpha-fetoprotein (AFP) level greater than 500 ng/mL. Multiple tumors were identified in 34.8% of the patients (Table 2).
Table 1. Recipient and Donor Characteristics for HCC Liver Transplant Recipients by Posttransplant Outcomes
Patients were followed for a median of 2.1 years (IQR = 1.0-3.6 years) after liver transplantation. Death due to HCC recurrence or a diagnosis of HCC recurrence was identified for 6.8% of the patients at a median of 1 year (IQR = 0.5-1.8 years) after transplantation (Table 2).
After adjustments for covariates, the IRRs for HCC recurrence were increased for liver transplant recipients with 1 tumor ≥ 2 cm (IRR = 1.63, 95% CI = 1.10-2.41, P = 0.02) or 2 to 3 tumors ≥ 2 cm (IRR = 1.82, 95% CI = 1.04-3.20, P = 0.04) versus recipients with >1 tumor, all < 2 cm; with ablative therapy versus none (IRR = 1.44, 95% CI = 1.15-1.79, P = 0.001); with an AFP level > 500 ng/mL versus an AFP level ≤ 500 (IRR = 3.00, 95% CI = 2.21-4.09, P < 0.001); and with an increasing DRI (IRR = 1.97, 95% CI = 1.52-2.57, P < 0.001). HCC recurrence IRRs were decreased for black race versus white race (IRR = 0.61, 95% CI = 0.39-0.95, P = 0.03) and for noncholestatic cirrhosis versus hepatitis C virus (IRR = 0.48, 95% CI = 0.26-0.89, P = 0.02; Table 3).
Table 3. Multivariate Adjusted Poisson Regression Model for the Risk of HCC Recurrence or HCC-Related Death
IRR (95% CI)
Tumor number and size
>1 tumor, all < 2 cm
1 tumor ≥ 2 cm
2-3 tumors ≥ 2 cm
Ablative therapy at exception
AFP > 500 ng/mL at exception
Hepatitis C virus
Hepatitis B virus
The shrinkage estimates of the ratio of observed HCC recurrences to expected HCC recurrences by center ranged from 0.6 to 1.76 (median = 0.97; Fig. 1). Although the model provided strong evidence (P = 0.001) that adjusted recurrence rates varied by center, no transplant center had a shrinkage observed/expected ratio significantly less than 1 on the basis of 95% CIs. Additionally, no transplant center was identified with a significantly higher than expected HCC recurrence rate. Using narrower 90% CIs did not change these results. After we imputed outcomes for patients potentially experiencing unreported HCC recurrence, the center-specific shrinkage ratios ranged from 0.72 to 1.39 (median = 0.98), and no center had shrinkage ratios significantly different from 1.
The evaluation of center-level effects was limited by the small numbers of patients at some centers. In this case, the expected number of recurrences was low, so that even when no recurrences were reported, the CIs for the shrinkage observed/expected ratios did not exclude 1.0.
As the transplant community continues to improve the liver transplantation guidelines for HCC, it is vital to identify and appropriately analyze the data on HCC recurrence. Single-center studies are generally limited by small sample sizes. The UNOS/OPTN database is the largest collection of US liver transplant data available. It has been demonstrated, however, that these data are often incomplete and sometimes inaccurate; a systemic misreporting of HCC recurrence would heavily skew any analysis of recurrence based on this data set. Publications on HCC using data from the OPTN/SRTR database have, therefore, exclusively relied on overall survival as the only outcome measure.[7-11] In fact, overall mortality has been used as a surrogate for tumor recurrence in evaluating the total tumor volume, AFP,[7, 9-11] and pretransplant local regional therapy as prognostic factors. Our study clearly shows that recurrence rates vary by center, even after we account for recipient, donor, and tumor characteristics. However, no centers could be conclusively singled out as having lower or higher than expected recurrence rates.
Shrinkage estimators are used in health quality research to examine center outcomes because they provide more stable and conservative center rankings than traditional alternatives. In brief, the shrinkage rate estimators are weighted averages of the center-specific observed/expected ratios and the ratio for all centers (by construction, 1.0), with weights primarily determined by the sample size at each center. Although the relatively precise estimates for large centers are almost unchanged by this averaging, estimates for smaller centers increasingly shrink toward 1. This decreases the effect of random variation due to small sample sizes on small centers' ratios and prevents them from being unfairly singled out. Mukamel et al. recently provided a cogent discussion of shrinkage estimators in health policy.
Although this analysis demonstrates unexplained variation among centers in recurrence rates, no particular center could be clearly identified as reporting fewer than the expected number of recurrences. Using data for a longer period would increase center sample sizes, but this might also obscure changes in reporting practices over time. Factors other than inaccurate reporting, including mismeasurement or the omission of important risk adjustment variables, could underlie the unexplained variation, although A2ALL comparison data demonstrate that the variables used by us are highly concordant. The absolute accuracy of center HCC reporting is not known and would require a comparison of the OPTN database with one of known quality with respect to HCC recurrence and mortality in the same patient data set. However, the A2ALL comparison demonstrates that even this approach is limited by variations in reporting between data sets.
A limitation is that our primary analysis provides no information about the cause of the differences between observed and expected recurrences reflected in the shrinkage ratios. In particular, it cannot distinguish underreporting from other unmeasured factors that might decrease HCC recurrence, such as earlier transplantation or different ablative techniques. However, our sensitivity analysis imputing recurrences among patients who died without recurrence may more directly capture underreporting. Specifically, the shrinkage relative risk (RR) for a center that underreports recurrences should increase numerically with respect to the estimate without imputation if our supposition is correct that recurrences are primarily underreported among patients who have died. This analysis would be most informative if we had identified centers with RRs significantly below 1.0 in the main analysis and found that their RRs moved toward or even above 1.0 in the sensitivity analysis.
Our inability to identify center underreporting or overreporting opens up the OPTN database to further research. We suggest further research efforts to evaluate whether the quality of reporting remains stable over time. Repeating this type of analysis every few years may be necessary to ensure the reliability of research using the UNOS/OPTN database.