Does practice make perfect?


More than 25 years ago, Luft and colleagues noted that hospitals in which higher annual volumes of particular procedures have been performed had lower rates of perioperative mortality than lower-volume centers (1). Inverse associations between surgical outcomes and volume have been documented for a wide range of procedures, including coronary artery bypass grafting, coronary angioplasty, vascular surgery, a range of cancer surgeries, total hip and knee replacement, liver transplantation, pediatric cardiac surgery, and many others (2–4). These associations persist after adjustment for age, sex, comorbidity, and socioeconomic status of patients, at least to the extent that such adjustment can be made with administrative data. The volume–outcome relationship has become a major theme of national quality improvement efforts. Indeed, the Leapfrog Group (an organization dedicated to improving the quality and efficiency of health care for US workers) has identified selective referral to high-volume centers for cardiac and oncologic surgical procedures as 1 of 3 key indicators of hospital quality (5).

In this issue of Arthritis & Rheumatism, Ward examines the associations between both hospital and attending physician volume of admissions for systemic lupus erythematosus (SLE) and in-hospital mortality (6). Having examined previously the association between hospital volume and SLE outcomes (7, 8), Ward focuses on the role of physician volume in this report. He demonstrates that patients admitted to Pennsylvania or New York hospitals with a primary or secondary diagnosis of SLE were twice as likely to die in the hospital if their attending physician admitted <1 case of SLE per year, on average, than if the attending physician admitted >3 cases per year.

Before reaching substantive conclusions about this report, we review key methodologic and interpretative issues inherent in studies of volume and outcome. We ask: 1) Do administrative data permit valid study of volume and outcomes? 2) Are associations between volume and outcome causal? 3) Is volume the key factor or a proxy for other aspects of care that drive outcomes? If the latter, what are the unmeasured factors? 4) How do we interpret studies that distinguish the effects of hospital volume and physician volume? Similarly, what considerations arise in studies of the effects of volume on medical conditions, as opposed to surgical procedures? 5) Finally, what are the potential consequences, intended and unintended, of shifting care from low-volume to high-volume providers?

Are administrative data valid?

Most studies of surgical volume and outcome investigate infrequent adverse events such as mortality, and complications of care such as readmission, infection, myocardial infarction, and pulmonary embolus. Since these adverse events occur in just a small percentage of patients undergoing elective procedures or admissions, analyses of volume and outcome must include a large number of providers and patients. Administrative data suit these purposes, and are used in virtually all studies of volume and outcome. Administrative data include insurance claims from Medicare or Medicaid or private insurance companies, hospital discharge data, pharmacy program data, and other such sources.

In general, documentation of mortality and surgical procedures in administrative data is very accurate, whereas medical diagnoses are less accurate and highly diagnosis dependent (9–11). Only 63% of outpatients who had had ≥2 visits to an academic rheumatology center for which they received the International Classification of Diseases, Ninth Revision (ICD-9) code for SLE were subsequently confirmed by medical record review to meet the American College of Rheumatology updated criteria for classification of SLE (12, 13). Furthermore, only 69% were considered by their treating rheumatologists to have definite SLE (14). The specificity of ICD-9 coding for SLE may be even lower for inpatients, who often carry many unconfirmed diagnoses on their lists of discharge diagnoses. This is particularly important since Ward's study included patients with SLE coded as the primary diagnosis or as 1 of up to 14 additional diagnoses. Subanalyses restricted to patients with SLE or nephritis as primary diagnoses documented stronger associations between increasing physician volume and decreasing mortality. This finding suggests that using secondary diagnostic codes may inadvertently include non-SLE patients or patients not admitted for an acute manifestation of SLE in the cohort, biasing results toward the null.

Does volume beget outcomes or vice versa?

Assuming that the data are accurate, we ask whether documented associations between volume and outcome are causal (15). It is natural to assume that practice makes perfect, but also plausible to envision that a large volume of patients seeks out high-quality providers. This issue is not easily teased apart with cross-sectional data. While analytical approaches are available to distinguish causal direction (15), longitudinal studies are ultimately needed to assess which comes first—changes in volume or changes in outcome. This question is of vital importance to policymakers. A program that creates high-volume centers makes sense only if favorable outcomes will follow.

Does practice make perfect or is volume a proxy for other structures or processes of care that drive outcomes?

If we accept that high volume is associated with good outcomes, we must ask whether it is volume per se, or factors associated with volume that make the difference. In a study of total hip replacement outcomes, we showed that hospital characteristics explained a small amount of the effect of volume on perioperative outcomes (16). Another study of mortality following myocardial infarction showed that use of beta-blockers was more common in high-volume centers, and that this process of care explained some of the effect of volume on outcome (17). In Ward's study, it is not clear what constitutes the SLE “disease-specific experience of the attending physician” hypothesized to be responsible for the volume–outcome association. Past experience caring for patients with SLE may increase the index of suspicion for relatively rare but potentially fatal conditions, such as diffuse alveolar hemorrhage, antiphospholipid antibody syndrome, and vasculitis. Experience with immunosuppressive medications may lead to their more judicious use and monitoring for their common toxicities. Alternatively, high-volume providers may be more adept in maneuvering within “the system,” and obtaining a timely rheumatology consult, renal biopsy, magnetic resonance imaging study, or echocardiogram that could save an SLE patient's life.

The strongest effects of physician volume in Ward's study were seen among patients without private insurance, while those with private insurance had low risks of in-hospital mortality regardless of provider volume. This finding suggests that having a high-volume provider may overcome barriers that do not exist for those with private medical insurance.

Hospital versus surgeon volume; surgical versus medical admissions.

Initial studies of volume and outcome addressed the association between hospital volume of surgical procedures and perioperative mortality and complications. Subsequent analyses distinguished the role of hospital and surgeon volume on surgical outcomes. Yet another vein of research in this area, including Ward's study (6), investigates the association between volume and the outcome of care for nonsurgical conditions. These approaches have important distinctions, particularly in their implications for decisionmaking with regard to patient care.

Studies of hospital volume and the outcome of elective surgery help patients and their physicians make choices. If patients are aware of the hospital's procedure volume and of the association between volume and outcome, they can use this information to inform their choice of hospital. Because patients and their physicians typically do not incorporate volume into hospital choice (Losina E, et al: unpublished observations), providing this level of education should be a priority.

Numerous studies have looked beyond hospital volume to examine the association between surgeon volume and outcome as well. Because surgeon and hospital volume are correlated (18, 19), these analyses must be done thoughtfully. For example, studies of total hip and knee replacement (18, 19), coronary angioplasty (20), and cancer surgery (21) documented that hospital volume drives some outcomes, surgeon volume others, and still others are influenced by both hospital and surgeon volume. Such data on surgeon volume refine the patient's decision matrix. Since patients are often referred to surgeons and not necessarily to hospitals, the surgeon's volume is particularly critical for patients who are planning elective, volume-sensitive operations.

In the last decade, investigators have shown that patients at higher volume hospitals and of higher volume clinicians have lower mortality rates following admissions for a range of medical conditions, including myocardial infarction and human immunodeficiency virus infection (4, 17). Ward has performed prior important work in this arena, and has shown an association between higher hospital volume of SLE patients seen yearly and lower mortality rates for SLE patients in California hospitals (7, 8).

Study of medical conditions raises a number of issues. First, as noted above, medical conditions are coded less accurately than surgical procedures, creating the risk of including inappropriate patients in the sample. Second, medical admissions are often emergencies, in which patients do not have the luxury of time to consider their hospital and physician of choice. Third, and perhaps most vexing, the role and even the identity of the admitting physician is less well defined for medical admissions than is the role of the operating surgeon in an elective surgery. The admitting physician may be a hospitalist who is an expert at handling typical inpatient problems such as decompensation of cardiac, pulmonary, and other diseases and typical complications such as deep vein thrombosis. Alternatively, the admitting physician may be a specialist who is expert at handling the underlying diagnosis (e.g., a lupus expert) or a specialist expert in the typical complications (e.g., a nephrologist, pulmonologist, or infectious disease specialist). Or, the admitting physician may be the primary care doctor. The fact that so few of the admitting physicians were rheumatologists or nephrologists in Ward's study raises the question of just who they were. Data were available only on the attending physician of record, while many other physicians and consultants were likely involved in the care of these patients, in particular at teaching hospitals where nearly 60% of the patients were admitted. Without understanding the roles of primary care providers, specialist admitting physicians, hospitalists, physicians-in-training, and specialist consultants in these findings, we cannot distinguish between the advice to seek care from an expert in inpatient care or an expert in SLE.

Consequences of shifting care to high-volume providers.

If indeed we are convinced that patients would be best served by high-volume providers, we must examine the policy options. One is to use a combination of administrative rules and incentives to steer patients to high-volume centers or providers. Investigators have calculated the number of perioperative deaths that would be averted if patients operated upon in low-volume centers received surgery in high-volume centers instead (3, 22, 23). The Medicare program has developed Center of Excellence projects for cardiac surgery and for total hip and knee replacement surgery, in which the centers of excellence are high-volume hospitals. As noted above, the Leapfrog Group advocates high-volume hospitals for cardiac and cancer surgeries (5).

Thus, the policy of selective referral to high-volume centers has proponents, and has produced results. We urge caution, however, and a broader perspective before adopting such programs widely. In the total joint replacement scenario, we have shown that patients operated upon in low-volume centers are more likely to be elderly, poor, nonwhite, less educated, rural, and female than their counterparts who undergo the same procedure in high-volume centers (24). A policy of shifting these patients to high-volume centers might avert some complications, but might also prompt some (perhaps many) of these patients to avoid surgery altogether, and thus to have persisting advanced musculoskeletal disability. Until we better understand the tradeoffs between reduced perioperative mortality for some and persistent disability for others, we are not ready for mandatory programs. This issue would seem even more important for SLE, which affects vulnerable populations disproportionately. Moreover, if all of the patients in Ward's study had equal access to the highest quality health insurance, it is possible that the association with provider volume may not have been observed. We should recall as well that the public health impact of regionalization is a function of both the magnitude of the advantage afforded by high-volume centers and the frequency of the particular type of admission. Thus, regionalization of SLE admissions would have a relatively small effect on mortality at the population level.

Returning to Ward's report (6), we ask whether the conclusions are credible and how patients, physicians, and policymakers should respond. We accept the conclusion that patients admitted by physicians who cared for a greater number of SLE patients annually had a lower rate of inpatient mortality than patients admitted by lower-volume physicians. Death is generally coded accurately in administrative databases. Ward provides no information on the accuracy of the SLE diagnoses in this database, other than to say that in general the data are accurate. The role of hospital volume also remains uncertain. In Ward's earlier study (7), hospital volume was associated with mortality following admission for SLE. In the present study, hospital volume is treated as a dichotomous variable, leaving open the possibility that a threshold effect exists but was missed by the cutoff used in these analyses.

The use of inpatient mortality as primary outcome means that patients who are discharged from the hospital earlier are at risk for the outcome for fewer days than those who are discharged later; that is, a patient who stays in the hospital for 10 days has more opportunity for an inpatient death than a patient who stays 2 days. The only way to overcome this limitation is to use a data source that captures deaths occurring after discharge. The observation that mortality rates were identical across providers for the first 2 days strengthens the need to continue observations beyond this period.

In short, the finding of reduced inpatient mortality among patients cared for by physicians who have higher SLE inpatient volume raises many questions that will require additional study to answer. It makes good sense to urge our patients to seek care—for SLE and other conditions—from providers who are experienced. However, before implementing programs within hospitals or health plans to ensure that patients are admitted by high-volume physicians, we need to understand some of the fundamental questions raised here about the care provided to the patients studied by Ward.