Outcomes monitoring has existed for liver transplantation for many years. Initially, center outcomes were used to designate centers of excellence by insurance companies. The Organ Procurement and Transplantation Network began using center-specific, risk-adjusted patient and graft survival models produced by the Scientific Registry of Transplant Recipients to flag centers whose observed outcomes were less than expected. Centers submit data on recipients and living donors, and organ procurement organizations submit data on donors. Multivariate Cox models are created to compare expected patient and graft outcomes based on the collection of all submitted data (national data) to a center's observed outcomes based on individual patient characteristics.1 The Scientific Registry of Transplant Recipients publicly reports these center-specific outcomes every 6 months, and they are used by both payers and patients to decide the relative performance of transplant centers. The Membership and Professional Standards Committee of the Organ Procurement and Transplantation Network has a peer review process to help flagged centers to improve their outcomes. Figure 1 provides an example of the public report format available from the Scientific Registry of Transplant Recipients Web site.2
The Centers for Medicare and Medicaid Services began to use Scientific Registry of Transplant Recipients data to flag centers for corrective action in 2007. This has led to both voluntary and involuntary closures of centers, although these primarily have been kidney transplant programs. There is evidence that centers have changed their behavior because of concerns about the potential effects of being flagged. Figure 2 demonstrates this effect in a survey of transplant centers.3
Although the processes of both the Centers for Medicare and Medicaid Services and the Organ Procurement and Transplantation Network have required center site surveys and corrective action plans, many centers have had observed outcomes significantly lower than the expected outcomes for many years. Figure 3 presents an examination of the reports posted between January 2005 and July 2010, which included 10 reporting periods. As can be seen, many centers had multiple periods in which graft survival was significantly less than expected, and 1 center had outcomes worse than expected in all periods. These data suggest that the process does not rapidly change a center's results.
One reason for this lack of change is that the reporting periods are 2.5 years in length. This means that only 20% of the patients in a report turn over in any reporting period, so it is difficult for the Organ Procurement and Transplantation Network, the Centers for Medicare and Medicaid Services, and the center to decide whether or not corrective action plans have been effective. This slow report turnover also creates uncertainties for centers that may wish to take action on outcomes before they are flagged.
One possible solution is to use an outcomes signaling measure: the cumulative sum (CUSUM). The CUSUM tool meets many of the requirements for an ideal monitoring tool in that it can be timely, is easily interpretable, and can be risk-adjusted in the same manner in which center outcomes are reported. The CUSUM is a graphical representation of the changes in risk-adjusted outcomes over time. Changes in the plot line suggest improving or declining outcomes.
Figures 4 and 5 present CUSUM plots for 2 kidney transplant programs: one with declining performance and another with improving performance.4 If a center updates its data on a frequent basis, it can act early on deviations of observed outcomes from expected outcomes and try to improve reported outcomes.
Improvements in a center's outcomes may be effected by changes in specific care processes at the center. It is more difficult to improve a center's observed outcomes in comparison with expected outcomes from the Scientific Registry of Transplant Recipients risk-adjusted model through changes in donor or recipient characteristics.5 The following is a partial list of the donor and recipient characteristics included in the model in the order of their importance.
Recipient factors: retransplantation, life support, malignant neoplasms other than hepatocellular carcinoma, functional status, portal vein thrombosis, recipient age ≥ 65 years, hepatitis C virus, recipient age of 60 to 64 years, abdominal surgery, creatinine, and albumin.
Donor factors: donation after cardiac death, split liver, donor age > 70 years, ischemic time ≥ 12 hours, ischemic time of 9 to 11 hours, race, and cause of death.
A center could try to improve its observed outcomes by making changes in the recipients through the elimination of transplantation for patients older than 65 years. This could result in an improvement in the center's observed outcomes, but because this is contained in the risk adjustment model for recipients, the expected outcomes would also improve, and a change in the difference between expected survival and observed survival might not occur. If the center actually had better outcomes for patients older than 65 years in comparison with other centers in the nation, the difference could actually increase. Similarly, if the transplant center used deceased cardiac donors who were significantly older or had longer warm ischemia times in comparison with the national average, the center's observed results could be worse than expected.
Because not all clinically important factors are included in the risk adjustment models, a center may be able to improve its observed outcomes by minimizing transplantation for patients with these clinically significant factors, such as cardiovascular disease. Other potential areas for risk mediation are nutritional status, smoking, and steatosis on donor liver biopsy samples.
The potential effects of the flagging of worse-than-expected outcomes on innovations in transplantation are of great concern.7 Liver transplantation has always been at the cutting edge of medical therapy, and the field cannot move forward without some risk taking by transplant centers with patients, donors, and therapies. Recent examples of innovations in liver transplantation include the down-staging of hepatocellular cancers to the Milan criteria, transplantation for human immunodeficiency virus–positive patients, and the use of left lobe grafts with inflow modulation. Trials of new therapies may turn out badly and result in decreased patient and/or graft survival. If the trials involve a significant number of a center's patients, the center's outcomes may be flagged.
How do we allow for innovation within the program-specific report framework? One potential method is to allow the designation of a therapy as an innovation and exclude these patients from the risk adjustment models. Reaching consensus with the Organ Procurement and Transplantation Network and the Centers for Medicare and Medicaid Services is going to require a significant amount of work. Important decisions need to be made about the process for the inclusion of protocols, patients, consent processes, and so forth. It is hoped that progress can be made to allow innovations to move the field of liver transplantation forward.