Telephone: 415-353-1590; FAX: 415-353-8709
Impact of outcomes monitoring on innovation and risk in liver transplantation†
Article first published online: 26 OCT 2012
Copyright © 2012 American Association for the Study of Liver Diseases
Supplement: Reaching the Summit: Challenges and Obstacles in Liver Transplantation
Volume 18, Issue Supplement S2, pages S59–S63, November 2012
How to Cite
Roberts, J. P. (2012), Impact of outcomes monitoring on innovation and risk in liver transplantation. Liver Transpl, 18: S59–S63. doi: 10.1002/lt.23539
Potential conflict of interest: Nothing to report.
- Issue published online: 26 OCT 2012
- Article first published online: 26 OCT 2012
- Accepted manuscript online: 18 AUG 2012 03:35AM EST
- Manuscript Accepted: 7 AUG 2012
- Manuscript Received: 9 JUL 2012
1. The reporting of liver transplant center outcomes is required by the final rule of the Department of Health and Human Services. The reported patient and graft survival outcomes are risk-adjusted for specific donor and recipient factors, and the observed survival is compared to the expected survival. Both the Centers for Medicare and Medicaid Services and the Organ Procurement and Transplantation Network flag programs for corrective action when the observed survival is significantly less than the expected survival. Both agencies can take action up to the closure of a center. In the last 5 years, the Organ Procurement and Transplantation Network has not taken an adverse action that required the closure of a liver transplant center because of outcomes.
2. Center survey data suggest that centers may try to select donors and recipients to minimize poor outcomes. This strategy may not be effective if centers stop accepting donors or recipients according to factors that are included in the risk adjustment model. For example, limiting recipients to those less than 65 years old may improve the observed outcomes, but the expected outcomes will also improve because a recipient 65 years or older is included in the model's risk adjustment.
3. For factors such as cardiovascular risk that are not included in the model, it may be reasonable to exclude patients in an attempt to improve the observed outcomes without affecting the expected outcomes. Other examples of these types of factors are smoking, nutritional status, and donor liver biopsy findings.
4. Currently, there is no exemption for patients undergoing experimental protocols. Down-staging for hepatocellular carcinoma, transplantation for human immunodeficiency virus–positive recipients, and the use of left lobe grafts with inflow modification are relatively recent areas of innovation in liver transplantation. Because innovation is frequently associated with a learning curve and, therefore, poor outcomes, the inclusion of patients in innovative protocols potentially could lead to centers being subjected to an adverse action by the Organ Procurement and Transplantation Network or the Centers for Medicare and Medicaid Services. Active consideration is being given to the exclusion of patients in innovative protocols from center-specific outcomes. Liver Transpl, 2012. © 2012 AASLD.
Outcomes monitoring has existed for liver transplantation for many years. Initially, center outcomes were used to designate centers of excellence by insurance companies. The Organ Procurement and Transplantation Network began using center-specific, risk-adjusted patient and graft survival models produced by the Scientific Registry of Transplant Recipients to flag centers whose observed outcomes were less than expected. Centers submit data on recipients and living donors, and organ procurement organizations submit data on donors. Multivariate Cox models are created to compare expected patient and graft outcomes based on the collection of all submitted data (national data) to a center's observed outcomes based on individual patient characteristics.1 The Scientific Registry of Transplant Recipients publicly reports these center-specific outcomes every 6 months, and they are used by both payers and patients to decide the relative performance of transplant centers. The Membership and Professional Standards Committee of the Organ Procurement and Transplantation Network has a peer review process to help flagged centers to improve their outcomes. Figure 1 provides an example of the public report format available from the Scientific Registry of Transplant Recipients Web site.2
The Centers for Medicare and Medicaid Services began to use Scientific Registry of Transplant Recipients data to flag centers for corrective action in 2007. This has led to both voluntary and involuntary closures of centers, although these primarily have been kidney transplant programs. There is evidence that centers have changed their behavior because of concerns about the potential effects of being flagged. Figure 2 demonstrates this effect in a survey of transplant centers.3
Although the processes of both the Centers for Medicare and Medicaid Services and the Organ Procurement and Transplantation Network have required center site surveys and corrective action plans, many centers have had observed outcomes significantly lower than the expected outcomes for many years. Figure 3 presents an examination of the reports posted between January 2005 and July 2010, which included 10 reporting periods. As can be seen, many centers had multiple periods in which graft survival was significantly less than expected, and 1 center had outcomes worse than expected in all periods. These data suggest that the process does not rapidly change a center's results.
One reason for this lack of change is that the reporting periods are 2.5 years in length. This means that only 20% of the patients in a report turn over in any reporting period, so it is difficult for the Organ Procurement and Transplantation Network, the Centers for Medicare and Medicaid Services, and the center to decide whether or not corrective action plans have been effective. This slow report turnover also creates uncertainties for centers that may wish to take action on outcomes before they are flagged.
One possible solution is to use an outcomes signaling measure: the cumulative sum (CUSUM). The CUSUM tool meets many of the requirements for an ideal monitoring tool in that it can be timely, is easily interpretable, and can be risk-adjusted in the same manner in which center outcomes are reported. The CUSUM is a graphical representation of the changes in risk-adjusted outcomes over time. Changes in the plot line suggest improving or declining outcomes.
Figures 4 and 5 present CUSUM plots for 2 kidney transplant programs: one with declining performance and another with improving performance.4 If a center updates its data on a frequent basis, it can act early on deviations of observed outcomes from expected outcomes and try to improve reported outcomes.
Improvements in a center's outcomes may be effected by changes in specific care processes at the center. It is more difficult to improve a center's observed outcomes in comparison with expected outcomes from the Scientific Registry of Transplant Recipients risk-adjusted model through changes in donor or recipient characteristics.5 The following is a partial list of the donor and recipient characteristics included in the model in the order of their importance.
Recipient factors: retransplantation, life support, malignant neoplasms other than hepatocellular carcinoma, functional status, portal vein thrombosis, recipient age ≥ 65 years, hepatitis C virus, recipient age of 60 to 64 years, abdominal surgery, creatinine, and albumin.
Donor factors: donation after cardiac death, split liver, donor age > 70 years, ischemic time ≥ 12 hours, ischemic time of 9 to 11 hours, race, and cause of death.
A center could try to improve its observed outcomes by making changes in the recipients through the elimination of transplantation for patients older than 65 years. This could result in an improvement in the center's observed outcomes, but because this is contained in the risk adjustment model for recipients, the expected outcomes would also improve, and a change in the difference between expected survival and observed survival might not occur. If the center actually had better outcomes for patients older than 65 years in comparison with other centers in the nation, the difference could actually increase. Similarly, if the transplant center used deceased cardiac donors who were significantly older or had longer warm ischemia times in comparison with the national average, the center's observed results could be worse than expected.
Because not all clinically important factors are included in the risk adjustment models, a center may be able to improve its observed outcomes by minimizing transplantation for patients with these clinically significant factors, such as cardiovascular disease. Other potential areas for risk mediation are nutritional status, smoking, and steatosis on donor liver biopsy samples.
The potential effects of the flagging of worse-than-expected outcomes on innovations in transplantation are of great concern.7 Liver transplantation has always been at the cutting edge of medical therapy, and the field cannot move forward without some risk taking by transplant centers with patients, donors, and therapies. Recent examples of innovations in liver transplantation include the down-staging of hepatocellular cancers to the Milan criteria, transplantation for human immunodeficiency virus–positive patients, and the use of left lobe grafts with inflow modulation. Trials of new therapies may turn out badly and result in decreased patient and/or graft survival. If the trials involve a significant number of a center's patients, the center's outcomes may be flagged.
How do we allow for innovation within the program-specific report framework? One potential method is to allow the designation of a therapy as an innovation and exclude these patients from the risk adjustment models. Reaching consensus with the Organ Procurement and Transplantation Network and the Centers for Medicare and Medicaid Services is going to require a significant amount of work. Important decisions need to be made about the process for the inclusion of protocols, patients, consent processes, and so forth. It is hoped that progress can be made to allow innovations to move the field of liver transplantation forward.
- 2Scientific Registry of Transplant Recipients. US hospitals with liver transplant centers. http://www.srtr.org/csr/current/Centers/TransplantCenters.aspx?organcode=LI. Accessed August 2012.
- 3Significant alterations in reported clinical practice associated with increased oversight of organ transplant center performance. Prog Transplant 2010; 20: 279-287., , .
- 6Donor and recipient risk aversion in liver transplantation [oral presentation]. Hepatology 2011; 54( suppl 1): 70A., , , .