The AJT Report


  • Sue Pondrom


This month, “The AJT Report” covers transplant centers that have developed their own tools to evaluate performance and outcomes data. Also this month, we look at a new documentary on the organ trade.

News and issues that affect organ and tissue transplantation

Good Measure

With SRTR Program-Specific Reports unavailable, many transplant centers have developed their own tools to analyze performance and predict outcomes

You can't manage what you can't measure. In transplantation, if you don't know your own data, it's very difficult to know when you're at risk.

“In my experience, a lot of programs want to be proactive [in data gathering and analysis], but SRTR [Scientific Registry of Transplant Recipients] data is at least a year old by the time we see it,” says Ed Zavala, administrator at the Vanderbilt Transplant Center in Nashville, Tenn. “What I hear from colleagues is that they want to develop and evaluate their own reports with real-time data. It may not be through the risk adjustment methodology that SRTR uses, but it provides them with a finger on the pulse of what is going on in their organization.”

Throughout the U.S., several transplant centers have developed tools to measure performance, while others have utilized SRTR spreadsheets and formulas with their own center tables to predict expected outcomes. This became especially important after SRTR announced in 2012 that it was delaying its Program-Specific Reports (PSRs), which show a program's outcomes to be “as expected,” “higher than expected” or “lower than expected.” The most recent PSRs were distributed in July 2012 and then halted because the Social Security Administration decided for privacy reasons that it would not share its protected death data with SRTR.[1] The death data were essential to the assessment of patient survival outcomes. As a result of the Social Security Administration decision, transplant centers have not been able to use PSRs to track program outcomes and anticipate future results.

Centers Become Proactive

Even before July 2012, many transplant programs had complained that data in the PSRs were one to three years old by the time they saw it. The Emory Transplant Center in Atlanta, for example, decided in 2007 to develop their own data-gathering tool to review actual survival prospectively on a monthly basis to make sure they were on track with expected outcomes.

“While SRTR PSRs are retrospective data, we replicate the same cohort methodology as the SRTR, with 30 months of data per report divided into five six-month rolling cohorts,” says Jennie P. Perryman, director of Emory's Policy and Outcomes Management. “We graphically trend in a stair-step fashion our data prospectively, with six months of data per stair, projecting observed (actual) to expected survival rates into future SRTR PSRs. As one six-month cohort of data (first stair) closes and rolls off, thus [marking] the end of an SRTR reporting period, another stair (sixth stair) rolls into the data and the next SRTR reporting period continues to be tracked.” While data from transplants performed from June through December 2013 will not appear in SRTR reports until January 2015 through January 2017, their data are already populating Emory's tracking tool.

“The one caveat to this tool is that the expected to which we compare our actual is a program's last expected survival published by SRTR,” she adds. “However, our expected survival rates have proven to be rather consistent over time. Thus when the SRTR reports are published, there are no surprises relative to our center's various programs' one-month, one-year and three-year survival rates.”

Brigitte Sullivan, administrative director at Johns Hopkins Comprehensive Transplant Center in Baltimore, relies on a report her center has used for almost three years now. “We've always done it because we like to get the information before the SRTR report comes out. We've found that most centers now use their own internal data to monitor survival in a more real-time fashion,” she says, adding that every transplant center should be able to produce the information themselves, based upon what they report to the United Network for Organ Sharing (UNOS). “We can't calculate the expected rate ourselves, but we can look at the formulas SRTR was using on their last published report, and we can mimic those formulas,” Sullivan says. “We can tell if we're going to be OK, below or above the expected.”

At Barnes-Jewish Hospital (Washington University in St. Louis), Gene Ridolfi, who is program director of the institution's Organ Transplant Center, Mechanical Circulatory Support Program, and his team have utilized several different tools in support of transplant center performance. “We have replicated the SRTR Center Specific Report released every six months by utilizing the SRTR data tables and populating donor and recipient data in a more real-time basis,” he says. “This allows us to predict pending PSR and actually project future PSRs out four to five reporting cycles.”

He notes that the challenges with the process are data gathering and people resources. “The limitation is that the predictive modeling is based on available data that is part of that cohort and will obviously change over time,” he says. “In addition, we now push out our CUSUM [SRTR's cumulative sum chart] report on a monthly basis. Although there may or may not be direct correlation with your PSR, it does allow you to identify potential flags and evaluate process.”

The CUSUM Charts

The transplant program at the University of Kansas in Kansas City has its own report, which is based on SRTR formulas and program-specific data in the monthly CUSUM chart, says Tracy Giocoma, the program's transplant administrator. “We transfer the CUSUM data table into the SRTR spreadsheet that allows you to calculate your expected survival rates,” she notes. “Every program can now input their data into the SRTR spreadsheet in less time.” Before CUSUM charts were made available in July 2013, the Kansas team used a rolling survival report, which was a spreadsheet they created, based on their last expected outcomes.

The CUSUM charts were developed as a result of recommendations from the February 2012 Consensus Conference on Transplant Program Quality and Surveillance jointly hosted by SRTR and the Organ Procurement and Transplantation Network (OPTN). According to SRTR, CUSUM is a quality-control method used in statistical analysis. CUSUM charting techniques are used to detect a change in a process and can be used to monitor quality in as near to real time as feasible. As used by the SRTR, a CUSUM chart is constructed by plotting a transplant program's performance over time with a line from points plotted for each calendar day. When no patients die, the CUSUM line goes down; with a death, the line goes up. If patients die at the expected rate, the CUSUM chart hovers around zero. If the death rate is higher than expected, the line on the chart goes up over time. Thresholds can be set to indicate when a program should react to the data in the chart, such as initiating an internal review. SRTR spokespeople say the CUSUMs are meant to be process control charts that show individual events over time, while the PSR tables on posttransplant outcomes provide the average performance over a specific time period.

RAPID: Real-Time Analytics and Process Improvement Dashboard

Sometime this year, following beta testing and trials by selected centers, the American Society of Transplant Surgeons (ASTS) will make available free of charge a new software platform to provide quality, process improvement and outcomes dashboards and reporting to individual transplant centers. ASTS says that the Real-Time Analytics and Process Improvement Dashboard (RAPID) will be relevant to UNOS and SRTR reporting and is intended to be easily extensible and sufficiently generic to support any future evolution in center-specific or SRTR reporting.

The software has been developed by Bing Ho, MD, assistant professor of medicine and surgery and a transplant nephrologist in Northwestern University's Division of Transplantation in Chicago. “The real-time dashboards I developed provide PSR data and other process improvement quality dashboards that you will be able to run on demand,” Dr. Ho says. “The software will be installed locally and will load UNOS and SRTR data for each center. Northwestern has no control over or access to it. Once loaded, there will be standard dashboards or reports you can run which can project your PSR reports for the next three PSRs.”

Noting that the SRTR PSRs will not reflect any changes in organ selection or patient care until four reports from the date of change, Dr. Ho says “the power of the new platform is that you'll know how you're going to do in the next year or so, and it will also give you predictions for incomplete cohorts for the fourth cohort and further out. In fact, you'll have partial information out until the 2017 report.”

Dr. Ho notes that RAPID, which will be updated quarterly, is not limited to performing the same analysis as SRTR. “If you are so inclined, you can take this software and customize it to do more advanced analysis of your transplant outcomes.”

Next PSR Set for March

SRTR has announced that the full public release of the next PSR will be on March 20, 2014, and will contain three cycles of reports (January 2013, July 2013 and January 2014). According to SRTR spokespersons, OPTN has identified sources of death information outside the Social Security Administration. Additionally, OPTN implemented a new process for verifying with transplant programs the current status of transplant candidates and recipients.

HBO Documentary Looks at Black-Market Organ Trade

Tales from the Organ Trade,” the lead-off production in HBO's Fall Documentary series, was a look at the flourishing black market in organs. Filmed over three years in eight countries on three continents, the documentary follows donors, recipients and physicians in an attempt to put a human face on illegal organ trafficking.

“There's something interesting in this world, Westerners who are desperate to live and people in developing countries who are desperate for money,” says the film's producer and director Ric Bienstock. “There's no question I went in with a bias that there is something repulsive about the organ trade. But when I started meeting the people, I had no judgment. This film took me on a real ethical journey, more than any other film I've done. I started thinking: Wow, if I was desperate, even though I think the idea of buying a kidney is repulsive, I might do the same thing.”

Francis Delmonico, MD, who viewed the documentary at the request of “The AJT Report,” called the film “a contrived showcasing of an opinion of a producer and writer that fails to explain sufficiently the reasons why the World Health Organization, the Council of Europe and prominent governments such as China and the United States prohibit the sale of organs. They are mistaken to summarily conclude that such sales can be ‘regulated.”’

Arthur Matas, MD, sees the film differently. “I thought it presented both perspectives. And it was interesting how they showed the families discussing this and bringing up the pros and cons.”

“Tales from the Organ Trade” is available on HBO On Demand and HBO Go, the network's streaming video site.