Inconsistent identification of reasons for removal from the liver transplant waiting list by Organ Procurement and Transplantation Network (OPTN) regions may contribute to regional variability in wait-list death rates. We analyzed OPTN and Social Security Administration (SSA) reported deaths of 103 364 liver transplant candidates listed May 8, 2003–April 17, 2011, and determined regional variability in risk of death attributable to differences in use of OPTN removal codes. Only 26% of candidates removed as “too sick” died within 90 days of delisting; 6335 deaths after delisting were not reported to OPTN. The ratio of number of candidates removed as “too sick” to number who died on the waiting list varied by region from 0.23 to 0.94, indicating substantial variability in use of removal codes. Including SSA-reported deaths within 90 days of delisting reduced regional variability in risk of death by 48% compared with deaths on the list alone, and by 35% compared with deaths plus the “too sick” designation. Codes for delisting liver transplant candidates are inconsistently applied among OPTN regions, spuriously elevating estimated regional variability in risk of wait-list death. This variability is ameliorated by including SSA- reported deaths within 90 days of delisting.
The Organ Procurement and Transplantation Network (OPTN) contractor, the United Network for Organ Sharing (UNOS), does not require transplant programs to report outcomes of patients who are removed from the liver transplant waiting list for reasons other than undergoing transplant, dying, or transferring to another center. Some studies [1, 2] simply censor (treat as survivors) patients who have neither undergone transplant nor died. However, mortality after removal from the waiting list is generally believed to be high among candidates removed for the reasons “too sick to transplant”, “medically unsuitable” or “refused transplant”. Censoring such patients in analyses of death on the waiting list could substantially bias results. Therefore, rather than using only wait-list deaths identified by OPTN as an outcome, many studies of wait-list mortality include as deaths patients removed from the list as “too sick to transplant” [3, 4]. Traditionally, candidates delisted for other reasons are not treated as deaths. However, these delisting categories are not defined, and may not be consistently applied. As a result, it is unclear to what extent inconsistent policies regarding patient delisting and use of removal codes exaggerate or understate differences among OPTN regions in mortality after registration on the liver transplant waiting list.
Ascertaining death on the waiting list is the cornerstone of the evidence used in making OPTN liver allocation policy changes, comparing center performance, and determining the extent of geographic, racial, and other inequities in wait-list deaths. Yet the consistency and accuracy (sensitivity and specificity) of death ascertainment using the traditional endpoints for death (death alone, death + “too sick”) have not been studied. The objectives of this study were (1) to establish, using the Social Security Administration Death Master File (SSADMF), the rate of death among patients removed from the liver transplant waiting list for reasons such as “too sick” or “medically unsuitable”, (2) to determine if there are regional differences in how removal codes are applied and to what extent such differences affect ascertainment of death, (3) to establish the extent to which regional variability in use of removal codes distorts the apparent geographic disparity in risk of wait-list death and (4) to establish an optimal definition of “death on the waiting list”.
Data source and institutional review board review
Analyses were based on UNOS data (Liver Star Files) as of June 3, 2011, and included all 103,364 patients active on the OPTN liver transplant waiting list at any time from May 8, 2003, through April 17, 2011. The University of Iowa institutional review board does not require review and approval of analysis of publically available deidentified data if a data use agreement has been signed protecting patient privacy.
Ascertainment of deaths and liver transplants
Deaths were ascertained by searching the OPTN files by patient identification code for reports of any death on the waiting list (DEATH_DATE) or posttransplant (PX_STAT_DATE with PSTATUS = 1), and by searching the SSADMF (SSMDF_DEATH_DATE). Liver transplant at any time after candidate registration on the waiting list was ascertained by locating the earliest TX_DATE (if any) after the listing date (INIT_DATE), including deceased and living donor liver transplants performed at the listing center or elsewhere.
The disposition of candidates removed from the waiting list with codes other than “death” or “transplant” was determined from the removal code and classified as follows: “medically unsuitable” (REM_CD 5), “refused transplant” (REM_CD 6), “transferred to another center” (REM_CD 7), “other” (REM_CD 9), “improved, transplant not needed” (REM_CD 12), “too sick to transplant” (REM_CD 13), “removed in error” (REM_CD 16).
Selection and imputation of baseline covariates
To permit analysis of the impact of the OPTN region on outcomes adjusted for important baseline covariates, we constructed a set of proportional hazards (Cox) models based on the entire OPTN set of patient listings for deceased donor liver transplant at any US liver transplant center from May 8, 2003, to April 17, 2011. Values for baseline covariates were ascertained at the time of initial listing, except that for patients already on the waiting list as of May 8, 2003, time-dependent covariates were updated to their values as of May 8, 2003. To retain all records in the final analyses, missing values were imputed. Further details of the statistical methods used to impute missing values and to select covariates for inclusion in the final models are available in an on-line Statistical Supplement available with this article at www.amjtrans.com.
The following baseline covariates in the UNOS data set excluding region were considered for inclusion in the models:
Demographic variables: patient age at listing, sex, race and ethnicity, citizenship, education, employment status, primary insurance, listing date and ABO blood group.
Liver disease status: liver diagnosis including liver tumor history and type; first versus subsequent liver transplant; history of peritonitis, muscle wasting, portal vein thrombosis, variceal bleeding, ascites, encephalopathy, transjugular intrahepatic portosystemic shunts, previous abdominal surgery and initial model of end-stage liver disease (MELD) status (active MELD, pediatric end-stage liver disease [PELD], status 1, or inactive).
Medical comorbid conditions: body mass index (BMI; kg body weight/meter2 height), diabetes, history of previous malignancy, hypertension, angina, cerebrovascular disease, peripheral vascular disease or ulcer; functional status, medical condition and life support and ventilator status.
Initial laboratory data: initial laboratory MELD score, serum albumin and serum sodium.
Competing risks analysis
To avoid bias in survival analysis due to correlation of the “risks” for transplant and for death on the waiting list, we used methods for determining the actual cumulative incidence of each of the competing outcomes of transplant in the place of Kaplan–Meier methods, and a reweighting method for the Cox analyses that permits calculation of the impact of covariates on the actual hazard associated with the cause-specific incidence of each outcome. We developed mixed Cox models using the individual data adjusted for each patient's individual baseline risk and with region entered as a random effect to estimate the variance of the impact of region on these outcomes. Further details are available in the on-line Statistical Supplement.
Selection of alternative definitions of “death on the waiting list”
In testing alternative definitions of “death on the waiting list”, we considered the desirability of choosing a definition that would be completely ascertainable within a reasonable time after removal from the waiting list. We therefore compared two traditional definitions for ascertaining death with three new definitions: Traditional definitions:
List only: deceased donor transplant from the list at the listing center and death on the list.
List + “too sick”: deceased donor transplant from the list at the listing center and death on the list plus delisting as “too sick”.
Ninety days: any death on the list or within 90 days of delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere).
One year: any death on the list or within 365 days of delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere).
Ever: any death on the list or ever after delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere).
We determined the impact of the method of ascertaining transplant and death by comparing the estimated variances of OPTN region as a random effect on the risk of transplant or death after listing, using definition 3 as the reference. Since the individual candidate baseline risks and the impact of other differences among the regions remain constant in all the models, the extent to which the estimated variance of region on outcomes differs according to the specific definitions of transplant and death estimates the extent to which the variability in these outcomes by region is inflated by the differences in method for ascertaining the outcomes.
Of 103 364 initial listings, 21 176 candidates were delisted for reasons other than liver transplant or death on the waiting list (Table 1). The most common reason for delisting was “other” followed by “too sick to transplant”, and “improved, transplant not needed”; smaller numbers of candidates were delisted for other reasons.
Table 1. Numbers of candidates removed from the Organ Procurement and Transplantation Network liver transplant waiting list for reasons other than transplant or death
Reason for removal
Percentage of total
Too sick to transplant
Improved, transplant not needed
Transferred to another center
Removed in error
Mortality among patients delisted with a code of “too sick” was substantially worse than among patients delisted with other codes, but was only 26% overall at 90 days after delisting, and only 47% at 1 year (Figure 1). Mortality was also substantial among patients with removal codes “other” and “refused”.
Further, variability by OPTN region was substantial regarding risk of death after removal from the list as “too sick”; 90-day mortality varied by region from 15% to 35%, and 1-year mortality from 34% to 60% (Figure 2; Table 2, “Unadjusted” columns). The ratio of number of candidates removed from the list as “too sick” to number who died on the waiting list varied from 0.23 to 0.94 by region (Table 3).
Table 2. Regional mortality rates, unadjusted and adjusted for completeness of reporting in the Social Security Administration Death Master File
OPTN = Organ Procurement and Transplantation Network.
Table 3. Ratio of number of candidates removed from the OPTN liver transplant waiting list as “too sick to transplant” to number who died on the waiting list, by OPTN region
Wait-list deaths, n
Delisted as “too sick”, n
OPTN = Organ Procurement and Transplantation Network.
Mortality after delisting with the “too sick” code was lower and more variable by OPTN region than expected, raising questions about how completely postdelisting deaths are reported. Because OPTN does not collect information about vital status of patients after removal from the waiting list, ascertainment of these deaths depends on the SSADMF, which is known to be incomplete and to vary in completeness from state to state due to different state policies regarding forwarding death information. We therefore crosschecked how completely deaths were ascertained in OPTN and SSADMF data among liver transplant recipients in the first year posttransplant. Ascertainment of death during the first year posttransplant should be essentially complete, since OPTN policy requires these data to be reported and audits centers for compliance. Indeed, 99% of the deaths reported by the SSADMF were reported in OPTN data. Conversely, only 86.1% of the deaths reported in OPTN data were reported in the SSADMF, with some variability in completeness by OPTN region (Table 4).
Table 4. Ascertainment of death: completeness of reporting to SSADMF and OPTN for patients who died in the first year after liver transplant
SSADMF OPTN deaths, n
OPTN = Organ Procurement and Transplantation Network; SSADMF = Social Security Administration Death Master File.
If one assumes that the SSADMF ascertains deaths after removal from the waiting list as completely as it ascertains deaths in the first year after liver transplant, the above data permit estimation by OPTN region of an “adjusted” risk of death at 90 days and at 1 year after delisting as “too sick” by dividing the unadjusted risk of death by the fraction of deaths reported in the SSADMF (Table 2, “Adjusted” columns). This adjustment increased the estimated total fraction of deaths in the 90 days after delisting as “too sick” only from 26% to 30%, and did little to reduce the variability of death by region (data not shown).
We then determined the impact of the method of defining transplant and death on the estimated variance of OPTN region as a random effect on the risk of transplant or death after listing.
With definition 1 as the reference, definition 2 (death on the list plus removals as “too sick”) adds no new transplants (Table 5). Definition 3 (90 days) adds 5291 transplants to the deceased donor liver transplants at the listing institution, either from living donors or from deceased donors at centers other than the listing center. Relatively few transplants are added with longer periods of postremoval follow-up (definitions 4 and 5). Deaths added to definition 2 include all patients removed from the list as “too sick”, and therefore include many patients who have not in fact died and exclude deaths of patients removed from the list for reasons other than “too sick”. Definition 3 adds 5333 deaths, ascertained from the SSADMF, that occurred within 90 days of delisting for any reason other than transplant or death while on the list. There were relatively few additional deaths with further follow-up (definitions 4 and 5).
Table 5. Total numbers of transplants and deaths by specific definition, and added transplants (% increase) and deaths over definition 1
Definitions: (1) list only: deceased donor transplant from the list at the listing center and death on the list; (2) list + “too sick”: deceased donor transplant from the list at the listing center and death on the list plus delisting as “too sick”; (3) 90 days: any death on the list or within 90 days of delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere); (4) 1 year: any death on the list or within 365 days of delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere); (5) ever: any death on the list or ever after delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere).
Includes patients removed as “too sick” as deaths, irrespective of their actual outcomes.
Table 6 shows the estimated variance in risk of death and transplant by OPTN region for definitions 1, 2, 4 and 5, compared with variance for definition 3 (90-day definition). Each of these estimated variances is highly significantly different from 0; i.e. irrespective of definitions of transplant and death used, highly significant differences remain among the OPTN regions in time to transplant and time to death on the waiting list. But the estimated regional variance in the risk (chance) of undergoing any liver transplant at the listing center or elsewhere within 90 days of removal from the list (definition 3) is 38% lower than the variance in risk of undergoing a deceased donor transplant specifically at the listing hospital (definition 1). There is relatively little change in the variance due to region with longer follow-up periods, since there were few additional transplants. Conversely, the estimated regional variance of the risk of death for definition 3 is only 52% as large as the estimated variance for definition 1, and only 65% as large as the variance for definition 2.
Table 6. Model variance for impact of region on risk of transplant and death
Endpoint definitions: (1) list only: deceased donor transplant from the list at the listing center and death on the list; (2) list + “too sick”: deceased donor transplant from the list at the listing center and death on the list plus delisting as “too sick”; (3) 90 days: any death on the list or within 90 days of delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere); (4) 1 year: any death on the list or within 365 days of delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere); (5) ever: any death on the list or ever after delisting, censoring for any transplant (living or deceased donor, at the listing hospital or elsewhere).
1. List only
2. List + “too sick”
3. 90 day
4. 1 year
Since the models based on these different definitions hold constant the underlying individual baseline risks of death and all regional factors other than method of ascertaining death, any difference in estimated regional variance in risk of outcomes among the models is attributable purely to differences in the method of ascertaining death (i.e., to how and when patients are delisted for reasons other than transplant and death), and not to differences in actual risk of death or transplant. That is, variance using definition 1 (list-only) and definition 2 (list + “too sick”) is inflated by 92% and 54%, respectively, due simply to regional differences in how delisted patients are ascertained as deaths. Further reduction in the estimated variance in risk of death without transplant was relatively minor with follow-up after 90 days.
There are major concerns about the variability among OPTN regions in the risk of death of patients listed for a deceased donor liver transplant. The Final Rule stipulates that allocation policies should aim to reduce geographic disparities in organ allocation . In addition, OPTN is considering including wait-list deaths when assessing center-specific performance. These considerations would suggest that the definition and ascertainment of death after listing should be carefully defined and consistently applied. However, OPTN provides no guidance regarding when patients on the liver transplant waiting list should be temporarily inactivated or permanently removed from the list, nor direction regarding which removal code should be used, nor a definition of “too sick to transplant”. Many patients are removed from the waiting list for “other” reason (which is undefined), and some die after delisting. This problem is exacerbated by the fact that OPTN has no mechanism for ascertaining deaths that occur after removal from the waiting list. We hypothesized that inconsistency in the definition of “death on the waiting list” might increase the apparent variability in the risk of death on the waiting list among OPTN regions.
To minimize some of the potential bias in overlooking deaths that occur after waiting list removal, some studies have suggested including patients delisted as “too sick to transplant” as events equivalent to death. However, this approach has its pitfalls; we found using the SSADMF that only 26% of patients removed from the OPTN waiting list during our study period had died within 90 days of delisting, and only 47% had died within the year. Including patients removed as “too sick” overestimates the correct number of deaths (1101 excess deaths). Equally important is the observation that including only “too sick” patients as deaths misses a significant number of deaths in patients delisted for other reasons (Figure 1).
Alternatively, deaths after removal from the OPTN waiting list may be ascertained by reference to US National Death Databases. The National Death Index is the gold standard for ascertaining death in the United States, but high cost and an additional lag of up to 2 years in reporting deaths limit its usefulness . The SSADMF is an online database of deaths reported to the Social Security Administration of people with Social Security numbers. It is sensitive for ascertaining death [6-9], albeit not as sensitive as the National Death Index. The SSADMF may be used to evaluate the vital status of patients removed from the OPTN liver transplant waiting list. OPTN and the Scientific Registry of Transplant Recipients (SRTR) have used the SSADMF data in their analyses for some time, and SRTR studied the completeness of the SSADMF in ascertaining deaths in a report in 2001 . But no study to our knowledge has reported how to use the SSADMF information to optimize consistency in analyzing outcomes of listing for liver transplant.
Using the SSADMF data, we found substantial variability by OPTN region in use of removal codes. The ratio of the number of candidates removed from the waiting list as “too sick” to the number who died on the list varied by region from 0.23 to as high as 0.94, and risk of death at 90 days among patients removed as “too sick” was more than double in some regions than in others (Table 3). We considered that some of this variability might be due to incomplete ascertainment of death in the SSADMF. To evaluate this, we determined how completely the SSADMF ascertains deaths of patients known to OPTN to have died in the year after liver transplant. The SSADMF appears to identify about 86% of the OPTN deaths; adjustment for this underreporting explains little of the regional variability in risk of death after delisting (Table 3). Lack of OPTN guidance regarding how to handle listed patients who are no longer reasonable candidates for transplant, and multiple factors in the listing and allocation process, such as severity of illness of the listed population (as reflected in mean MELD at transplant), ratio of donors to recipients, number of patients undergoing transplant with MELD exception points, center stances about delisting for futility and so forth, could account for this variability.
We used the SSADMF data to develop models for the outcomes of listing for liver transplant, with estimated variance of region as an optimization criterion. We considered administrative censoring at varying times after delisting (90 days, 1 year, and ever). We considered it biologically plausible that the impact of an event leading to delisting would most likely affect a candidate's survival within 90 days after the event [11-14]. We also thought it desirable that the outcome be completely ascertainable within a short period after delisting. We found only minor changes in performance of the 1 year and “ever” models (models  and  over the 90-day model (model .
The estimated regional variance in ascertainment of risk of transplant after listing is 38% lower using definition 3 (90 day) compared with definition 1 (deceased donor liver transplant from the list). The optimal choice of a definition for ascertaining risk of transplant will depend on the exact question raised, whether one is interested, for example, in the impact of a factor on an individual's chance of undergoing any transplant or alternatively in the impact of a factor on the chance of an individual being allocated a deceased donor liver at the listing center.
Compared with definition 1 (death on the list alone), definition 2 (death on the list + “too sick”) reduces the regional variance by 20%. But including all SSADMF-ascertained deaths within 90 days of delisting (definition 3) reduces the regional variance by 48% compared with definition 1 and by 35% compared with definition 2. There is relatively little further reduction in regional variance with additional follow-up time. Thus, definition 3 appears to be optimal for ascertaining death after listing for most purposes. But the optimal definition may depend on the question being asked. For example, hepatocellular cancer patients delisted as “too sick” because of progressing beyond Milan criteria may not die within 90 days.
We have shown that inconsistency in use of removal codes accounts for considerable regional variance, but this analysis does not indicate that there are no differences in allocation process risks. Indeed, regional variation in the allocation process may lead to as much as a 30% difference (data not shown) in the relative risk of death between the OPTN regions with highest and lowest risk, independent of individual patient baseline risks and differences in the method for ascertaining deaths. These differences are highly statistically significant. Yeh et al.  recently documented evidence of important regional differences in access to liver transplant as assessed by median waiting time to transplant and average MELD score at the time of transplant. Their measures are not influenced by the methods used to assess the rate of death on the liver transplant waiting list. It will be important to determine the reasons for these real differences in access to liver transplant. However, use of definition 3 (90-day outcome) would avoid exaggerating the regional differences in wait-list death rates that are due to methodological differences only.
In summary, there is marked variability among OPTN regions in the way candidates are removed from the liver transplant waiting list and in the removal codes used. This variability in the use of removal codes results in overstating the true regional variability in wait-list deaths. Our analysis shows that ascertaining all deaths up to 90 days after delisting, using the SSADMF and OPTN notifications of death, reduces estimated interregional variance by as much as 48% and is the optimal endpoint for death.
This work was supported in part by Health Resources and Services Administration contracts 234-2005-370011C and HHSH250201000018C. The content is the responsibility of the authors alone and does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US Government. The authors thank Scientific Registry of Transplant Recipients colleague Nan Booth, M.S.W., M.P.H., E.L.S., for manuscript editing.
The authors have no conflicts of interest to disclose as described by the American Journal of Transplantation.