SEARCH

SEARCH BY CITATION

Keywords:

  • Compliance;
  • Medicaid;
  • Medicare;
  • regulatory issues;
  • transplant

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Concerns about a Punitive Approach
  5. Risks to Innovation
  6. Clinical Experience and Citation for Outcome Deficiencies
  7. References

Two concerns expressed by the American Society of Transplant Surgeons (ASTS) are that (1) the new Medicare regulations for transplant hospitals take a ‘punitive’ approach and that (2) the outcome requirement may thwart innovation by not including certain risk factors into the risk adjustment used to calculate expected outcomes. This article explains efforts by the Centers for Medicare & Medicaid Services (CMS) to encourage quality improvement. CMS limits outcomes-related enforcement to situations where failure rates exceed certain substantial ‘tolerance limits’, ensuring opportunity for quality improvement to be effective prior to enforcement. Transplantations involving a disproportionate share of risk factors not incorporated into the risk-adjustment methodology can also be raised through CMS’‘mitigating factors’ process. Of the 22 mitigating factor requests completed through March 10, 2009, 7 raised issues of risk adjustment (none involved experimental protocols). Four of the seven requests were approved for other reasons (evidence of effective program changes and improved outcomes). CMS concluded that none of the seven made a persuasive case based on risk factors. The early data indicate that program deficiencies may outweigh risk adjustment issues. CMS agrees to consider the ASTS suggestions for future action and continues to monitor the situation in case a different pattern emerges.


Abbreviations: 
ASTS

American Society of Transplant Surgeons

CMS

Centers for Medicare & Medicaid Services

HRSA

Health Resources and Services Administration

MF

mitigating factors

OPTN

Organ Procurement and Transplantation Network

SRTR

Scientific Registry of Transplant Recipients

UNOS

United Network for Organ Sharing

Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Concerns about a Punitive Approach
  5. Risks to Innovation
  6. Clinical Experience and Citation for Outcome Deficiencies
  7. References

The remarkable noncollapsing (1) U.S. health care system continues to attract criticism that it is wasteful (2), prone to error (3), financed mainly by fee-for-service models that promote volume over value and so fragmented that it barely merits the label ‘system’ (4). However, at one-sixth of the nation's economy and growing in cost at a rate 2–3% faster than the gross domestic product, plans for change abound. Some proposals would address fundamental questions of value—not simply cost, but the quality of outcomes achieved in relation to each dollar spent. In turn, a focus on quality requires not only outcome measures but also methods to adjust for risk factors so as to yield the expected value.

Embedded in some discussions of health care are calls for ‘accountable care systems’ that can implement organized processes for improving quality and be held accountable for quality and costs (5–8). Where might we look for examples of systems that take responsibility not only for the outcomes of discrete procedures but also for the well-being of individuals over time, and that track outcomes against risk-adjusted expected values?

The transplant community has successfully struggled with many of these challenges. Insofar as graft survival is affected by overall health status of the individual, transplant programs are forced to address the whole well-being of patients over time. Insofar as both patient and graft survival are affected by many things outside of the physician's or the transplant center's immediate control (such as patient adherence to immunosuppressive drug regimens), transplant programs inherently depend on an ability to extend the reach and influence of their care (e.g. through education and follow-up) and to partner with the patient and other health care professionals in the patient's own community.

Equally important, the transplant community created a national quality improvement framework composed of (a) on-going, mandatory reporting to populate a national database (9) that (b) enables an impressive system for calculating risk-adjusted, expected outcome values against which observed outcomes can be compared. These data are (c) publicly reported for every transplant program in the United States. Outcome measures include not just 1-month or 1-year survival but also up to 3-year survival measures. The risk-adjusted data support (d) a peer review process organized under auspices of the Health Resource Services Administration (HRSA) and contracts with the United Network for Organ Sharing (UNOS) to operate the Organ Procurement and Transplantation Network (OPTN). This quality improvement framework has contributed to a pattern of consistent improvement in transplant outcomes.

Concerns about a Punitive Approach

  1. Top of page
  2. Abstract
  3. Introduction
  4. Concerns about a Punitive Approach
  5. Risks to Innovation
  6. Clinical Experience and Citation for Outcome Deficiencies
  7. References

The American Society of Transplant Surgeons (ASTS) authors contrast the above peer review, quality improvement process with what they characterize as the more ‘punitive’ approach in the Medicare regulations implemented by the Centers for Medicare and Medicaid Services (CMS). The concern is worthy of attention, as it is always worthwhile to be vigilant against a regulatory body that could become unhinged from beneficent purposes. CMS’ survey and certification efforts therefore include a number of design features intended to reinforce rather than undermine the quality improvement framework.

First, the CMS regulations include ‘tolerance limits’ that permit considerable opportunity for a hospital's internal quality improvement system to detect poor outcomes, identify causes and make adjustments prior to CMS even becoming aware of an issue. Before a transplant program is at a risk of termination from Medicare participation due to poor outcomes, 1-year patient death or graft failures must exceed the expected value by 50% or more, the difference between the observed and expected deaths or graft failures must be statistically significant based on a one-sided p-value < 0.05 and the number of observed minus expected deaths or graft failures must be greater than 3.0.

Second, poor risk-adjusted outcomes may start a Medicare termination process but do not conclude the process. During the initial 3-year phase-in of the new regulation, CMS delays the effective date of the prospective termination for 210 days with respect to outcome deficiencies. Compared to the 90-day effective date for other deficiencies that do not constitute immediate jeopardy, the longer 210-day period permits more time for quality improvement efforts to succeed and become manifest in the data, as well as time for mitigating factors (MF) to be considered. Table 1 portrays these time periods for the 1-year patient death statistic.

Table 1.  Quality improvement opportunities compared with timeline of potential medicare termination
1-year deaths observed/expected
 June 28, 2007 to CMS survey dateSurvey date + 210 daysOver 210 days from survey
 150% or moreQuality improvementMitigating factors (MF) considerationMedicare participation terminated (if MF is denied)
<150% or p > 0.05Quality improvementQuality improvementQuality improvement

Third, as an essential component of the certification process, CMS considers ‘mitigating factors (MF)’ that may have adversely affected outcomes, such as natural disasters or other factors beyond control of the transplant center. In the initial 3-year phase-in of the regulation, we purposefully restrained from restricting the types of MF that transplant programs might request CMS to consider. We include consideration of evidence that the transplant program made improvements to such an extent that it is in full compliance with outcome expectations, except that data lag may have prevented the improved results from appearing in the outcome data used in the regulation. The MF process is described on the CMS website at: http://www.cms.hhs.gov/CertificationandComplianc/20_Transplant.asp.

CMS regulations require each transplant program to have its own Quality Assurance and Performance Improvement (QAPI) system. The MF process permits a transplant program to benefit from its internal quality improvement efforts, but only if such efforts are effective, and are put in place early enough to provide clear evidence of improvement in graft and patient outcomes prior to the effective date of a prospectively scheduled Medicare termination.

To what extent are the above design features having the desired effects? As of March 10, 2009 we had completed action on MF requests from 22 transplant programs. Almost all programs facing termination of Medicare participation requested MF consideration. Table 2 shows the major reasons for which hospitals requested MF. Eleven programs that requested MF consideration had a level of clinical experience below the threshold specified in the regulation (10 transplants per year). We approved 2 of these 11 programs due to a unique organizational arrangement by the hospital system in question (i.e. two transplant programs with separate certifications performed multiorgan transplants jointly). No other requests from low-volume programs have been approved. Additional information about CMS action on all MF requests will be posted on the CMS website.

Table 2.  Mitigating factor (MF) decisions through March 10, 2009
Mitigating factor categoryMF requests by major deficiency typeValidity confirmed + MF approved
Clinical experienceOutcomeTotalClinical experienceOutcomeTotal
Access to care 9 918000
Improvement + SRTR data lag 01010077
Risk adjustment 0 7 7000
OPTN/OPO changes 6 3 9000
Negative impact on hospital 2 2 4000
Natural disaster 1 0 1000
Other 2 0 2202
Total no. of mitigating factors203151279
Total no. of programs111122279

An additional 11 programs requested MF consideration after receiving a deficiency citation for statistically significant outcomes that exceeded the CMS thresholds. Of the 11 outcomes-related requests, 7 were approved. Each of the seven outcomes-related, approved requests fulfilled all five of the following: (1) substantial program improvements (2) addressed root causes of prior, poor outcomes and were (3) implemented and institutionalized on an on-going basis with (4) substantial evidence of improved outcomes subsequent to the quality improvement changes, and the evidence (5) indicates present-day compliance with CMS outcome requirements (when trended forward), except for the time lag inherent in the 1-year survival data set. For example, one transplant program conducted a root cause analysis just prior to the CMS survey, identified substantial personnel and protocol changes, recruited a new Medical Director and transplant surgeons with track records of good outcomes, implemented the changes, tracked outcomes following the changes and provided statistical evidence of the improved outcomes in a ‘before and after’ comparison using data that did not require a 1-year wait. Examples of such intermediate data included posttransplant complication rates, posttransplant readmission within 30 days, acute rejection rates and red blood cell utilization during transplant.

Risks to Innovation

  1. Top of page
  2. Abstract
  3. Introduction
  4. Concerns about a Punitive Approach
  5. Risks to Innovation
  6. Clinical Experience and Citation for Outcome Deficiencies
  7. References

A second ASTS concern is that the absence of adjusters for certain risk factors will penalize innovation. We are pleased to see numerous OPTN workgroups laboring to evaluate additional risk adjustments that can be incorporated into the methodologies used in the Scientific Registry of Transplant Recipient (SRTR) reports. In the meantime, the ASTS offers constructive ideas, such as the potential for a prospective OPTN registration of patients undergoing transplantation under experimental protocols so that the outcomes for such patients may be analyzed separately. We are discussing these ideas with our colleagues at the Health Resources Services Administration (HRSA) and will provide a more substantive response to the ASTS proposal once we advance further in our dialogue.

Although the CMS survey and certification process is ongoing, early indications from the programs that have faced the prospect of Medicare termination are that the outcome requirement—so far—is having the effect of improving transplant programs rather than causing innovation to decline.

First, only 2 of the 32 MF requests received through March 10, 2009 involved the use of experimental protocols, and none of the 22 completed MF requests involved the use of such protocols. Second, the tolerance limits (previously discussed) provide some margin to accommodate the fact that no risk adjustment methodology will be perfect or that conscious decisions to limit the administrative burden of clinical data submission will limit the extent of risk adjustment possible.

Third, programs cited for outcome deficiencies were all also cited for many other program deficiencies. Table 3 compares the most common deficiencies found through the first 99 surveys of all transplant programs, and compares those with the deficiencies identified in programs cited for poor outcomes for which the MF review has been completed. These early data indicate that programs cited for outcome deficiencies were also cited at about twice the rate as other transplant programs in key areas, such as patient care (45% vs. 24%), multidisciplinary planning (49% vs. 21%), quality assurance and program improvement (27% vs. 7%) and patient selection (27% vs. 10%).

Table 3.  Most frequent deficiency citations—first 99 surveys
CategoryNo. of transplant programs citedPercent of programs cited (%)
All programsPrograms cited for outcomes1All programsPrograms cited for outcomes1
  1. 1Programs cited for outcome deficiencies includes only those for whom the MF request process had been completed by March 10, 2009.

  2. 2Many programs had more than one deficiency.

ABO verification-organ receipt53 55445
Informed consent26 22618
Patient/living donor care24 52445
Multidisciplinary planning21 52145
Quality assurance + performance improvement 7 3 727
Patient selection10 31027
Total number of programs299119911

Fourth, the MF process (described above) offers an important medium to consider the effect of factors not included in the SRTR risk-adjustment that may be beyond the control of the transplant program. The MF process is not intended to substitute for the SRTR risk adjustment methodology, but does allow other factors to be considered. The burden rests with the transplant program to provide evidence to establish the validity of each proposed MF.

Table 4 describes the results of the CMS review for the 7 out of 11 MF requests that involved both outcome deficiencies and concerns about risk-adjustment factors. Two of the requests made the case that cardiovascular risk factors were not included in the SRTR adjustments. We agreed that the SRTR risk methodology did not address the cardiovascular issues in question. However, we did not approve the MF requests on that basis because the programs offered little evidence that they had assertively addressed those risk factors during the SRTR reporting period. Pursuant to the CMS survey, the two programs fortunately recognized that these risk factors are subject to treatment effects and implemented recent program improvements targeted to cardiovascular conditions. We approved the MF requests for these two programs because of those improvements, combined with evidence of improved results sufficient to conclude that the programs were likely to be in present-day compliance but for the SRTR data lag. MF requests for two additional programs were approved for the same reason (while the risk adjustment concerns were not found to warrant MF approval, as explained in Table 4).

Table 4.  CMS action on risk adjustment mitigating factor requests through March 10, 2009
ProgramRisk adjustment concernReason for CMS denial of risk adjustment factor requestUltimate CMS action
A. Adult kidney only1. The program would be in compliance with the outcomes but for a suicide [because the (O-E) number would become less than 3.0 integers if the suicide were ignored]CMS did not accept this as a mitigating factor for several reasons: (a) suicide is a known risk factor posttransplant; (b) onsite surveyors found that the transplant program failed to provide support from social services during the transplant or posttransplant period and (c) all transplant programs are held to the same standard based on a comparison of national expected outcomes. A suicide in another program would also be reflected in its outcomes. (d) Even with the suicide removed from the data, the number of deaths would remain almost twice the expected number. (e) The MF process is not one in which we will manipulate the data to introduce a non-SRTR risk calculationValidity of MF not verified MF request denied
B. Adult kidney only1. The program transplants a high number of patients with cardiovascular disease who are not considered by the SRTR risk modelingThe program does not transplant a higher proportion of patients with cardiovascular disease than a comparison hospital serving a similar client profile and whose outcomes were in compliance with the regulation. The program also failed to provide evidence that it had implemented effective efforts to address the comorbidities during the SRTR report periodMF approved for other reasons: The program analyzed its poor outcomes, implemented substantial improvements, and showed evidence of recent, better outcomes
C. Adult kidney only1. SRTR does not adjust for time on the waitlistSRTR adjusts for ‘time on dialysis’, a risk adjuster found to be superior to ‘time on a waitlist’MF approved for other reasons: The program analyzed its poor outcomes, implemented substantial improvements, and showed evidence of recent, better outcomes
2. Cardiovascular risk factors are not included in the SRTRThe risk factor is influenced by treatment effects, and the transplant program failed to demonstrate that it had implemented effective methods to address comorbidities during the SRTR reporting period 
3. The program serves the unique patient population with a high percentage of minorities and hypertensive nephrosclerosis. The interaction of race and hypertensive nephrosclerosis is not in the SRTR modelThe SRTR methodology adjusts for race. SRTR also assessed the interaction between race and hypertensive nephrosclerosis and found that it was not a statistically significant predictor of survival. In addition, several nearby programs serve a comparable clientele and have acceptable outcomesImprovements focused on better management of cardiovascular risks
D. Adult heart only1. The program transplants individuals who tend to be older, who are minorities, who have a status of 1A and who are on devices compared to the national averageThe risk adjustment models used by the SRTR consider these patient characteristics to various degrees in calculating the expected survival rates for heart transplant patientsMF approved for other reasons: The program analyzed its poor outcomes, implemented substantial improvements, and showed evidence of recent, better outcomes
2. The program transplants a number of combined heart/liver patients who are not included in the risk adjustment models. These patients have better outcomesGiven the proportion of combined heart/liver patients compared to heart-only patients, the actual survival is sufficiently different from the expected survival that, even if heart/liver patients were included, the impact would be negligible and would not yield a finding of acceptable outcomes 
E. Pediatric kidney only1. A low number of transplants annually results in a single event having a large statistical adverse event on outcomesTransplant programs are held to the same standard regarding outcomes. A low number of transplants is not evidence of mitigating factors. CMS has built in other safeguards to prevent low-volume centers from being adversely affected by this factor [e.g. p < 0.05 and (O-E)>3 must both be satisfied before outcomes are found deficient]MF approved for other reasons: The program implemented substantial improvements, and showed evidence of recent, better outcomes
F. Adult heart only1. Prior to January 2009, the SRTR did not account for risk adjustment for patients supported by mechanical circulatory support devices. Nearly one-half of the program's deaths in the July 2008 report were in patients supported by a mechanical device prior to transplantThe July 2008 and January 2009 SRTR reports do consider whether or not the patient was supported by a mechanical circulatory support device. These reports adjust the survival rates for patients supported by a mechanical deviceValidity of MF not verified MF request denied
2. It is unclear if SRTR adequately stratifies risks or accounts for institutional selection biasesThe program did not provide any evidence support to the assertion that the risk adjustment is inadequate or that the risk adjustment calculation has an undue negative impact on the program outcome data 
G. Adult kidney only1. The program serves a high proportion of HIV-AIDS patientsOnly a small percentage deaths involved HIV positive individuals; HIV status represented a minor factor in the differences between the actual and expected graft failuresValidity of MF not verified MF request denied
2. The program serves a high proportion of those age 65 and older, those whose donor was an expanded criteria donor (ECD) or had acute renal failureThe SRTR methodology adjusts for age, whether or not the donor was an expanded criteria donor and the functioning of the donor's kidneys 

All three remaining MF requests were denied in total. One program wished to have the effects of a suicide discounted from the results. Another stated that it served a higher proportion of HIV positive individuals, yet such individuals did not play a significant role in the excess graft failure in the SRTR report. The third program erroneously asserted that the SRTR did not adjust for mechanical devices. Unfortunately, none of the three was able to demonstrate significant program improvements and improved outcomes.

Finally, any risk to innovation that might be engendered by the Medicare outcome requirement must be weighed against the risk of poor outcomes-–and adverse results for patients—that might otherwise be unaddressed. Quality improvement alone sometimes fails to achieve the desired results. The CMS survey and certification process is therefore designed to safeguard recipients and donors by enforcing an outer boundary of consequences for those transplant programs that have failed to take full advantage of quality improvement opportunities. For example, we issued a final termination notice to a program for which we found: (a) statistically significant 1-year patient deaths more than twice the number of risk-adjusted expected deaths, (b) no internal quality assurance and performance improvement (QAPI) system (more than 1 year after the new CMS regulation made such QAPI systems mandatory), (c) the absence of any real root cause analysis and (d) a paucity of implemented plans for substantial improvement. A previous peer review and other quality improvement efforts in the transplant center had failed to produce significant results. Only after we issued a Medicare termination notice did the transplant program conclude that it should go inactive and try to reconstitute itself (though inactivity does not stop Medicare termination once the termination process has begun).

Clinical Experience and Citation for Outcome Deficiencies

  1. Top of page
  2. Abstract
  3. Introduction
  4. Concerns about a Punitive Approach
  5. Risks to Innovation
  6. Clinical Experience and Citation for Outcome Deficiencies
  7. References

The ASTS authors also analyze the relationship between CMS requirements for minimum clinical experience (volume) and the prospect for Medicare termination due to poor outcomes. This is an interesting analysis but not particularly useful. Considerable, more in-depth research has found a clear relationship between clinical experience and positive outcomes (10). A principal reason for the clinical experience requirement in CMS regulations is that the outcome requirement has little utility for low-volume programs due to regulatory and statistical reasons, particularly the improbability that all three CMS thresholds for the citation of an outcome deficiency will be crossed if a program has very low volume. This is particularly true with regard to the threshold for statistical significance (p < 0.05) and the threshold that observed minus expected 1-year death or graft failure values must exceed 3.0. Of the 11 requests for MF related to clinical experience, only one had statistically significant outcomes (expected 1-year patient survival was 91%, actual survival was 50% at p < 0.05). Since the (O-E) value was less than 3.0, however, not even this low-volume situation tripped all three of the CMS outcome thresholds. We therefore do not expect to find a positive relationship between very low volume and CMS citation of outcome deficiencies. For the same reasons, we cannot have much confidence in data showing good outcomes from a program that conducts a few transplants. Note the large number of low-volume programs in each of the ASTS graphs. And for transplant programs with higher volumes, the CMS clinical experience requirement is irrelevant. We proceed in such cases to a direct examination of actual outcomes. Since neither of the previously discussed ASTS concerns depends on the ASTS analysis of clinical experience, I will not comment further.

I will conclude by returning to the context of health care discussions that now engage the nation. Any appreciative inquiry ought to take note of the fact the transplant community has organized one of the better, national, risk-adjusted outcome measurement systems in all of medicine. HRSA and the OPTN also contribute an additional and vital second element: a national quality improvement framework. CMS regulations and onsite surveys add a third element in the form of quality assurance. CMS quality assurance, is designed to reinforce the quality improvement framework, establish outer boundaries for performance, and clarify that clear consequences follow failure to heed the call for outcomes within an acceptable range. These are hallmarks of an increasingly accountable system. Insofar as no methodology is perfect and may also have unintended effects, a MF review process is proving to be a useful element of the accountability system that uses risk-adjusted outcomes for transplant hospitals. Those involved in the health care debates may benefit from a close look at the accomplishments of the transplant community as they examine other areas of health care.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Concerns about a Punitive Approach
  5. Risks to Innovation
  6. Clinical Experience and Citation for Outcome Deficiencies
  7. References
  • 1
    Brown LD. The amazing noncollapsing U.S. health care system—Is reform finally at hand? N Engl J Med 2008; 358: 325327.
  • 2
    Institute of Medicine and National Academy of Engineering. Building a better delivery system: A new engineering/health care partnership. Washington : National Academies Press, 2005: 18.
  • 3
    Institute of Medicine. Crossing the quality chasm: A new health system for the twenty-first century. Washington : National Academies Press, 2001.
  • 4
    Corrigan J, McNeill D. Building organizational capacity: A cornerstone of health system reform. Health Aff 2009; 28: w205w215 (published online 27 January 2009; 10.1377/hlthaff.28.2.w205).
  • 5
    Shortell SM, Casalino LP. Health care reform requires accountable care systems. J Am Med Assoc 2008; 300: 9597.
  • 6
    Fisher ES, McClellan MB, Bertko J et al. Fostering accountable health care: Moving forward in medicare Health Aff 2009; 28: w219w231 (published online 27 January 2009).
  • 7
    Fisher ES, Staiger DO, Bynum JP, Gottlieb DJ. Creating accountable care organizations: The extended hospital medical staff. Health Aff 2007; 26: w44w57.
  • 8
    Casale AS, Paulus RA, Selna MJ et al. “ProvenCareSM”: A provider-driven pay-for-performance program for acute episodic cardiac surgical care. Ann Surg 2007; 246: 613623.
  • 9
    National Organ Transplant Act of 1984, P.L 98–507, especially Sections 372 and 373.
  • 10
    Weiss ES, Mequid RA, Nishant PD et al. Increased mortality at low-volume orthotopic heart transplantation centers: Should current standards change? Ann Thorac Surg 2008; 86: 12501260.