SEARCH

SEARCH BY CITATION

Keywords:

  • radiation oncology;
  • randomized controlled trials;
  • evidence-based medicine;
  • health care policy;
  • comparative effective research

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. FUNDING SUPPORT
  8. CONFLICT OF INTEREST DISCLOSURES
  9. REFERENCES

BACKGROUND

Randomized controlled trials (RCTs) are commonly used to inform clinical practice; however, it is unclear how generalizable RCT data are to patients in routine clinical practice. The authors of this report assessed the availability and applicability of randomized evidence guiding medical decisions in a cohort of patients who were evaluated for consideration of definitive management in a radiation oncology clinic.

METHODS

The medical records of consecutive, new patient consultations between January and March 2007 were reviewed. Patient medical decisions were classified as those with (Group 1) or without (Group 2) available, relevant level I evidence (phase 3 RCT) supporting recommended treatments. Group 1 medical decisions were further divided into 3 groups based on the extent of fulfilling eligibility criteria for each RCT: Group 1A included decisions that fulfilled all eligibility criteria; Group 1B, decisions that did not fulfill at least 1 minor eligibility criteria; or Group 1C, decisions that did not fulfill at least 1 major eligibility criteria. Patient and clinical characteristics were tested for correlations with the availability of evidence.

RESULTS

Of the 393 evaluable patients, malignancies of the breast (30%), head and neck (18%), and genitourinary system (14%) were the most common presenting primary disease sites. Forty-seven percent of all medical decisions (n = 451) were made without available (36%) or applicable (11%) randomized evidence to inform clinical decision making. Primary tumor diagnosis was significantly associated with the availability of evidence (P < .0001).

CONCLUSIONS

A significant proportion of medical decisions in an academic radiation oncology clinic were made without available or applicable level I evidence, underscoring the limitations of relying solely on RCTs for the development of evidence-based health care. Cancer 2013;119:3092—3099. © 2013 American Cancer Society.


INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. FUNDING SUPPORT
  8. CONFLICT OF INTEREST DISCLOSURES
  9. REFERENCES

Modern evidence-based health care relies on the availability of high-quality data to guide decisions regarding individual patient care as well as allocation and payment for health services. Randomized controlled trials (RCTs) are commonly considered the most powerful research method available to assess the effects of an intervention while controlling for potential selection bias through randomization.[1] Since the development of the modern RCT by Sir Austin Bradford Hill in the 1940s, results from hundreds of thousands of RCTs have been published, forming the basis of modern evidence-based medicine.[1, 2]

Within oncology, a substantial evidence gap remains, and published consensus clinical practice guidelines frequently lack high-level evidence to support recommendations.[3, 4] This lack of evidence has been highlighted during recent debates regarding appropriate health care reform, and substantial funding has been allocated to support the conduct of comparative effectiveness research with the passage of the Patient Protection and Affordable Care Act of 2010.[5]

Consensus guidelines typically consider RCTs as the highest quality of medical evidence available to guide clinical decisions.[6, 7] However, in many clinical situations, it may be unethical or impractical to conduct prospective randomized trials.[8] RCTs may be further limited, in that the results from such trials represent outcomes of selected individuals undergoing treatment in a highly controlled environment.[9] Outcomes in real-world settings may differ from the prescriptive and tightly controlled environment of a classic RCT, thus limiting the external validity of such studies.[10] Although RCTs remain a critical tool for assessing the efficacy of treatments, several alternative methods of assessing efficacy and effectiveness are suitable for comparative effectiveness research, including pragmatic clinical trials and observational studies.[11-13]

Although the development of improved health care models based on randomized evidence is rational, the body of published RCTs may not be applicable to a substantial number of patients seen in real-world settings.[14, 15] Few studies have systematically evaluated the availability of randomized data in everyday clinical scenarios. The objective of the current study was to assess both the availability and the applicability of randomized evidence guiding medical decisions in a cohort of patients who were evaluated for consideration of definitive management in a radiation oncology clinic.

MATERIALS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. FUNDING SUPPORT
  8. CONFLICT OF INTEREST DISCLOSURES
  9. REFERENCES

Study Population

The medical records of consecutive, new patients who attended a consultation between January and May 2007 in the Department of Radiation Oncology at the University of North Carolina Hospitals were retrospectively reviewed. Patients who were evaluated for the management of benign, metastatic, pediatric, nonsolid malignancies or primary tumor palliation were excluded.

Data Collection

Patient characteristics were obtained from medical records and included age at diagnosis, disease stage (American Joint Committee on Cancer Cancer Staging Manual, sixth edition), sex, race, month of diagnosis, disease organ system and site, disease presentation (primary or recurrent), consultation setting (outpatient vs inpatient), and treatment intent. For the purposes of this study, each primary medical decision made with respect to an individual patient's radiotherapeutic care (eg, the addition of neoadjuvant chemoradiation to surgery for a patient with locally advanced esophageal cancer) was classified based on the availability of supporting evidence into 1 of 2 classification groups: Group 1 included medical decisions associated with available, relevant level I evidence; and Group 2 included medical decisions made without available level I evidence.

Level I evidence was defined as evidence obtained from at least 1 RCT. The selection of appropriate phase 3 RCTs was based on RTCs that were referenced as evidence supporting the recommended treatment in the National Comprehensive Cancer Network (NCCN) Guidelines (version 1, 2007) and/or National Cancer Institute (NCI) Physician Data Query (PDQ) guidelines.[3, 16]

Medical decisions categorized in Group 1 were then assessed individually based on whether the respective medical decision fulfilled eligibility criteria within each appropriate RCT. For the purposes of this analysis, if an RCT existed for which the patient had full eligibility, then we assumed that all aspects of that treatment delivery were defined in the parameters for that clinical trial. For example, if a patient was eligible for a postlumpectomy radiation therapy trial, then all of the dose, time, volume, and other treatment parameters would be scored as being defined by that trial. In cases where there was a separate and independent, additional medical decision for the same patient (eg, postlumpectomy radiation boost in a patient with breast cancer who received breast-conservation therapy), that second medical decision was assessed separately. RCT eligibility was classified as follows: Group 1A, patients who fulfilled all eligibility criteria; Group 1B, patients who did not fulfill at least 1 minor eligibility criterion; and Group 1C, patients who did not fulfill at least 1 major eligibility criterion.

Major criteria were defined as those that were considered critical to the core integrity of the studied patient population in the RCT (eg, stage, histology). Minor criteria were defined as those deemed by the authors unlikely to alter recommended management in a routine clinical practice setting, such as age, history of clinically irrelevant prior or synchronous cancers (eg, history of prostate cancer in a patient with active, locally advanced lung cancer), or prior nonradiation treatment. Laboratory values and comorbidities could be considered minor or major violations, depending on specific circumstances. For example, neutropenia in a patient with rectal cancer who was being evaluated for a concurrent chemoradiation trial was considered a major violation; whereas mild, acute renal compromise in a patient with prostate cancer who was being evaluated for postoperative radiation was deemed a minor violation.

The relevant supporting RCTs were recorded for each medical decision along with any eligibility violations. Each medical decision was assigned only 1 classification group (1A, 1B, or 1C). If a patient's medical decision had both minor and major eligibility violations within a RCT, then the major violations were favored, and those medical decisions were classified as 1C. If multiple RCTs were applicable to a medical decision (eg, multiple RCTs addressing radiation dose escalation in prostate cancer), then medical decisions were classified as 1A if they were considered fully eligible for at least 1 RCT. For cases in which a patient was not fully eligible for any RCT (classified as 1B or 1C), medical decisions were classified as 1B if at least 1 RCT was considered 1B.

Because multiple RCTs may be applicable to a specific medical decision (eg, postlumpectomy radiation in breast cancer), it was possible for patients to be eligible for 1 or more RCTs, but not others. To address this issue, we also analyzed the data by using an alternative method, classifying medical decisions according to a “majority rules” process, and we observed no significant differences in the results (data not shown).

Statistical Analysis

Contingency tables were used to examine the relations between pairs of categorical variables. Chi-square tests were used to compare percentages among groups, and the Jonckherre-Terpstra test was used to compare percentages across groups for ordinal data. Descriptive statistics (frequencies and percentages) were reported for the overall study population. Statistical analyses were performed using the SAS statistical software package (version 9.3; SAS Institute Inc., Cary, NC).

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. FUNDING SUPPORT
  8. CONFLICT OF INTEREST DISCLOSURES
  9. REFERENCES

In total, there were 586 new patient consultations during the study period (Fig. 1). Nearly all of these patients (93%) were evaluated in the outpatient setting rather than the inpatient setting (7%). There were 193 patients excluded from analysis because of metastatic disease (n = 110; 19%), nonsolid malignancies (n = 27; 5%), benign disease (n = 15; 3%), pediatric malignancies (n = 14; 2%), and primary tumor palliation (n = 27; 5%). Among the remaining 393 evaluable patients, 58 presented with multiple treatment decisions regarding overall radiotherapeutic management, culminating in 451 total evaluable medical decisions.

image

Figure 1. Profile diagram of medical decisions based on eligibility criteria for randomized controlled trials (RCTs).

Download figure to PowerPoint

Clinical characteristics of patients and medical decisions are displayed in Table 1. Malignancies of the breast (30%), head and neck (18%), and genitourinary system (14%) were the most common primary disease sites included in the study. In total, 66 RCTs were considered and applied for the purposes of classifying medical decisions (Table 2). Thirty-three percent (n = 22) of all RCTs that were considered for study purposes were conducted in patients who had malignancies of the prostate (n = 13) or breast (n = 9). Across all disease sites, medical decisions were most frequently made regarding postoperative radiation (25%) and definitive chemoradiation (17%).

Table 1. Patient Characteristics and Medical Decisions
Patient Characteristic, n = 393No. (%)Medical Decision, n = 451No. (%)
  1. Abbreviations: ADT, androgen deprivation therapy; CNS, central nervous system

  2. a

    Medical decisions with less than 5 events included chemotherapy (n = 4), intraperitoneal radioactive phosphorus (32P) (n = 2), postoperative chemotherapy (n = 2), prophylactic cranial radiation (n = 1), salvage chemoradiation (n = 2), salvage radiation (n = 2), and supportive care (n = 2).

Age, y ADT20 (4)
Median [range]59 [21-92]Radiation boost23 (5)
Sex Brachytherapy5 (1)
Men165 (42)Definitive chemoradiation77 (17)
Women228 (58)Definitive radiation27 (6)
Race Definitive surgery41 (9)
White270 (69)Radiation dose escalation26 (6)
Black107 (27)Hypofractionation11 (2)
Others16 (4)Intraoperative radiation13 (3)
Organ site Surveillance17 (4)
Breast116 (30)Postoperative chemoradiation26 (6)
CNS/eye22 (6)Postoperative radiation114 (25)
Gastrointestinal36 (9)Preoperative chemoradiation15 (3)
Genitourinary56 (14)Preoperative radiation7 (2)
Gynecologic40 (10)Reirradiation6 (1)
Head/neck70 (18)Trimodality8 (2)
Lung24 (6)Miscellaneousa15 (4)
Skin/soft tissue/thorax29 (7)  
Disease presentation   
Primary351 (89)  
Recurrent42 (11)  
Clinical setting   
Outpatient384 (98)  
Inpatient9 (2)  
Table 2. Randomized Controlled Trials
TrialDisease SiteMedical Decision Topic
  1. Abbreviations: ±, with or without; 5-FU, 5-fluorouracil; AHT, adjuvant hormone therapy; AST, androgen-suppression therapy; BCT, breast-conservation therapy; CALGB, Cancer and Leukemia Group B; CAO/ARO/AIO-94, German Rectal Cancer Study Group trial; CKVO, Dutch Cancer Society Committee on Clinical Comparative Research (Commissie Klinisch Vergelijkend Onderzoek); COMS, Collaborative Ocular Melanoma Study; CRT, chemoradiotherapy; CT, chemotherapy; DCIS, ductal carcinoma in situ; ECOG, Eastern Cooperative Oncology Group; EORTC, European Organization for Research and Treatment of Cancer; ESPAC, European Study Group for Pancreatic Cancer; GBM, glioblastoma multiforme; GLOT-GFPC, Lyon-St. Etienne Thoracic Oncology Group-French Lung Cancer Study Group; GITSG, Gastrointestinal Tumor Study Group; GOG, Gynecologic Oncology Group; GORTEC, Groupe d'Oncologie Radiotherapie Tete et Cou (Head and Neck Radiation Oncology Group); HOG, Hoosier Oncology Group; HU, hydroxyurea; IDC, invasive ductal carcinoma; INT, intergroup; LCCG, Lung Cancer Cooperative Group; MDACC, The University of Texas M. D. Anderson Cancer Center; MGH, Massachusetts General Hospital; MMC, mitomycin-C; MRC, Medical Research Council; MRM, modified radical mastectomy; MSKCC, Memorial Sloan-Kettering Cancer Center; NCAHT, neoadjuvant concurrent and adjuvant hormone therapy; NCHT, neoadjuvant and concurrent hormone therapy; NCI, National Cancer Institute; NCIC, National Cancer Institute of Canada; NPC, nasopharyngeal carcinoma; NRH, Norwegian Radium Hospital; NSABP, National Surgical Adjuvant Breast and Bowel Project; NSCLC, nonsmall cell lung cancer; PALN, para-aortic lymph node; PMRT, postmastectomy radiation therapy; PORT, prostate-only radiation therapy; PORTEC, Post Operative Radiation Therapy in Endometrial Carcinoma; RT, radiation therapy; RTOG, Radiation Therapy Oncology Group; START, Standardization of Breast Radiotherapy Trial; SWOG, Southwest Oncology Group; TROG, Trans-Tasman Radiation Oncology Group; UKCCCR, United Kingdom Coordinating Committee on Cancer Research; WJLCG, West Japan Lung Cancer Group; WPRT, whole pelvis radiation therapy.

NSABP B-17Breast DCISLumpectomy ± breast radiation
EORTC 10853Breast DCISLumpectomy ± breast radiation
NSABP B-06Breast IDCMRM vs BCT
EORTC 22881Breast IDCPostlumpectomy RT ± RT boost
CanadaBreast IDCPostlumpectomy standard vs hypofractionation RT
STARTBreast IDCPostlumpectomy standard vs hypofractionation RT
Danish 82b,cBreast IDCMRM ± PMRT
British ColumbiaBreast IDCMRM ± PMRT
EORTC 26981GBMDefinitive radiation ± chemotherapy
COMSChoroidal melanomaEnucleation vs plaque brachytherapy
UKCCCRAnal cancerDefinitive RT vs CRT
EORTC 22861Anal cancerDefinitive RT vs CRT
RTOG 87-04/ECOG 1289Anal cancerDefinitive CRT 5-FU ± MMC
RTOG 98-11Anal canalDefinitive CRT 5-FU + MMC vs cisplatin
RTOG 85-01EsophagusDefinitive RT vs CRT
INT 0123EsophagusDefinitive CRT dose escalation
EORTC 22001EsophagusSurgery ± preoperative CRT
CALGB 9781EsophagusSurgery ± preoperative CRT
INT 0116StomachSurgery ± postoperative CRT
GITSG 9273PancreasDefinitive RT vs CRT
GITSG 9173PancreasSurgery ± postoperative CRT
EORTC 40891PancreasSurgery ± postoperative CRT
ESPAC-1PancreasSurgery ± postoperative CT vs CRT
RTOG 97-04PancreasPostoperative CRT 5-FU vs gemcitabine
NSABP R-01RectumSurgery + observation vs CT vs RT
NSABP R-02RectumSurgery + CT vs CRT
SwedishRectumSurgery ± preoperative RT
DutchRectumSurgery ± preoperative RT
CAO/ARO/AIO-94RectumPreoperative vs postoperative CRT
MDACCProstateDefinitive RT dose escalation
MSKCCProstateDefinitive RT dose escalation
CKVO96-10ProstateDefinitive RT dose escalation
MRC RT01ProstateDefinitive RT dose escalation
RTOG 94-13ProstateWPRT vs PORT vs NCHT vs AHT
MGHProstateDefinitive RT ± AST
RTOG 86-10ProstateDefinitive RT ± NCHT
TROG 96-01ProstateDefinitive RT ± NCHT
RTOG 85-31ProstateDefinitive RT ± AHT
EORTC 22863ProstateDefinitive RT ± AST
RTOG 92-02ProstateDefinitive RT + NCHT vs NCAHT
EORTC 22911ProstateSurgery ± postoperative RT
SWOG 8794ProstateSurgery ± postoperative RT
GOG 85CervixDefinitive CRT cisplatin/5-FU vs HU
GOG 120CervixDefinitive CRT cisplatin vs cisplatin/5-FU/HU vs HU
RTOG 90-01CervixDefinitive WPRT + PALN RT vs CRT
SWOG 8797CervixPostoperative RT vs CRT
NRHUterusPostoperative brachytherapy ± WPRT
PORTECUterusSurgery ± WPRT
GOG 99UterusSurgery ± WPRT
GORTEC 9401OropharynxDefinitive RT vs CRT
EORTCHypopharynxSurgery vs definitive CRT
RTOG 91-11LarynxDefinitive RT vs CRT
US INTHead and neckDefinitive RT vs CRT
Bonner et al[17]Head and neckDefinitive RT ± cetuximab
RTOG 95-01Head and neckPostoperative RT vs CRT
EORTC 22931Head and neckPostoperative RT vs CRT
LCCG-EORTCNSCLCDefinitive RT vs CRT
Jeremic et al[18]NSCLCDefinitive RT vs CRT
HOGNSCLCDefinitive RT vs CRT
WJLCGNSCLCDefinitive sequential vs concurrent CRT
GLOT-GFPC NPC 95-01NSCLCDefinitive sequential vs concurrent CRT
CALGB 39801NSCLCDefinitive CRT ± induction CT
RTOG 94-10NSCLCDefinitive sequential vs concurrent CRT
NCICSarcomaPreoperative vs postoperative RT
NCISarcomaAmputation vs limb preservation + RT

Thirty-six percent (n = 165) of all medical decisions did not have RCTs available to guide clinical decision making and were categorized as Group 2 (Fig. 1). Reasons for Group 2 classification included: no available RCTs addressing the specific treatment topic (56% of decisions), rare tumor presentation/histology (32% of decisions), and recurrent disease (12% of decisions). Examples of medical decisions that were classified as Group 2 included stereotactic body radiation for definitive treatment of nonsmall cell lung cancer and postoperative radiation for malignant melanoma. Furthermore, 11% (n = 48) of medical decisions were categorized as Group 1C decisions, because patients had major eligibility violations when considering relevant RCTs. The most frequent reason for major ineligibility was overwhelmingly because of inappropriate tumor stage, comprising 48% of all major eligibility violations (Table 3). In total, 47% of all medical decisions lacked either available or applicable randomized evidence (Groups 1C and 2) to inform clinical decision making.

Table 3. Reasons for Patient Ineligibilitya
Minor Ineligibility ReasonsNo.Major Ineligibility ReasonsNo.
  1. a

    Note that patients may have had multiple simultaneous ineligibility violations.

History of invasive cancer24Stage53
Prior chemotherapy23Histology8
Age13Laboratory values8
Laboratory values10Tumor markers6
Surgical staging8Pathologic features5
Surgical staging technique6Disease extent4
Bilateral disease2Tumor location4
Radiation timing2Timing of surgery4
Comorbidity1Nutrition4
Tumor size1Synchronous cancer3
  Prior radiation3
  Performance status3
  Comorbidity2
  Surgery type2
  Tumor grade1

Forty-six percent (n = 205) of all medical decisions were made with available randomized evidence for patients who met all trial eligibility criteria and were classified as Group 1A (Fig. 1). In addition, 7% (n = 33) of medical decisions were made in patients who had only minor eligibility violations and were categorized as Group 1B. The most common reasons for minor ineligibility were a history of prior, clinically irrelevant, invasive cancer; prior chemotherapy; and age (Table 3). In summary, randomized evidence was both available and applicable in 53% of medical decisions (Groups 1A and 1B) that were evaluated for the purposes of the current study.

Various patient characteristics potentially associated with eligibility classifications were studied. Race, sex, and month of diagnosis were not associated with eligibility classifications (Table 4). Primary tumor diagnosis was the only clinical characteristic significantly associated with the availability of evidence (P < .0001). A detailed statistical analysis of the association of individual primary diagnosis categories with eligibility classifications was not feasible given the relatively few numbers of patients within each individual primary diagnosis category.

Table 4. Patient Characteristics and Association With Eligibility Classifications
 No. of Patients (%) 
CharacteristicGroup 1A+ 1BaGroup 1C+ 2aP
  1. Abbreviations: CNS, central nervous system.

  2. a

    Group 1 included decisions made based on available, relevant level I evidence (phase 3 randomized control trial [RCT]) supporting recommended treatments; Group 1A, patients who fulfilled all eligibility criteria; Group 1B, patients who did not fulfill at least 1 minor eligibility criterion; Group 1C, patients who did not fulfill at least 1 major eligibility criterion; Group 2, patients without available, relevant level I evidence (phase 3 RCT) supporting recommended treatments.

Race   
White129 (69)141 (69).83
Black52 (27)55 (27) 
Other8 (4)8 (4) 
Sex   
Men76 (40)89 (43).49
Women113 (60)115 (57) 
Month of diagnosis   
January28 (15)39 (19).90
February41 (22)36 (18) 
March40 (21)35 (17) 
April34 (18)39 (19) 
May46 (24)55 (27) 
Primary diagnosis   
Breast66 (35)50 (25)< .0001
CNS/eye11 (6)11 (5) 
Gastrointestinal21 (11)15 (7) 
Genitourinary28 (15)28 (14) 
Gynecologic19 (10)21 (10) 
Head and neck31 (16)39 (19) 
Lung10 (5)14 (7) 
Skin, soft tissue, thorax3 (2)26 (13) 

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. FUNDING SUPPORT
  8. CONFLICT OF INTEREST DISCLOSURES
  9. REFERENCES

We undertook this study to determine the availability and applicability of randomized evidence guiding medical decisions in a cohort of patients who were evaluated for consideration of definitive management in a radiation oncology clinic. We observed that nearly half of medical decisions were made without available or applicable randomized evidence to serve as guidance, indicating a substantial shortage of high-quality evidence to inform routine clinical decisions. Our findings also indicate that there is considerable variation in the availability of randomized evidence among selected patient groups based on primary tumor diagnosis.

To our knowledge, this is the only published study to evaluate the availability of evidence in a routine clinical setting for any medical specialty. We systematically evaluated the evidence supporting medical decisions made in a cohort of consecutive patients who attended a single academic institution. Our study sample was relatively large and included a wide variety of patient diagnoses. In addition, we conducted a detailed analysis of whether related RCTs were truly relevant for an individual patient based on the study eligibility criteria.

Our data are relevant in the current health care and economic environment. Further improvements in patient outcomes and the development of efficient health care delivery systems will depend on the development of high-quality data to inform changes with respect to care delivery, payment models, and policy reform. For example, clinical care pathways provide consensus guidelines for the delivery of health services and seek to improve the effectiveness and overall efficiency of health care delivery by improving provider education and reducing unexplained variation in the provision of medical services.[21-25] However, the rationale supporting the creation of appropriate care pathways critically depends on the availability of relevant and high-quality data regarding health care outcomes. Although there have been numerous publications of consensus guidelines for oncology care, our study suggests that the strength of such guidelines may be compromised in a substantial proportion of clinical situations by a lack of available or applicable level I evidence.

Furthermore, health policy makers and payers frequently require supportive evidence when making coverage decisions, which can have profound implications for the delivery of health services as well as an individual patient's finances.[26] For situations in which health care plans apply evidence-based approaches to insurance coverage decisions, our data become particularly germane. Evidence-based coverage policies frequently deny payment for health services that lack adequate evidence to establish efficacy.[26, 27] RCT data often are considered the proof of adequate evidence. In our study, we observed that a substantial proportion of patients seen in routine clinical settings lacked applicable level I evidence to guide their care. These results highlight the potential negative consequences of such evidence-based coverage policies, which may impede necessary and appropriate patient care. If the standard of adequate evidence in the form of RCTs often is not generalizable to patients, then the appropriateness of using randomized data as the basis for coverage of every patient comes into question.

We acknowledge that randomized evidence is not a reasonable goal for all clinical situations. There are a multitude of reasons for this, including the rarity of a disease, which may preclude timely accrual of an adequately powered RCT24; reluctance of patients or providers to enroll in RCTs that randomize patients to widely disparate therapies (eg, surgery vs radiation therapy); or randomization that may be unethical in certain clinical situations because of a lack of equipoise regarding the interventions in question (eg, proton radiation therapy for pediatric brain tumors). Overall, it has been estimated that only 5% of adult oncology patients are enrolled in clinical trials.[14, 17] Our study results draw attention to this issue and suggest that well designed observational studies and alternative clinical trial designs may play a central role in the continued development of evidence for medical decision making in select patient groups.

In this study, we observed substantial variation in the availability of level I evidence according to patient diagnosis. Although we were not able to perform valid statistical analyses on which specific individual tumor diagnoses were more likely to have supporting randomized evidence because of the small patient numbers in each primary diagnosis group, the reasons for this variation are likely multifactorial. It may be more practical to conduct large, prospective RCTs in patients with more common diagnoses, because trial accrual goals will be easier to meet, thereby ensuring the success of a resource-intensive clinical trial.

Furthermore, from a societal perspective, the allocation of limited resources to conduct clinical trials in the largest segments of the cancer patient population is certainly a rational approach.[18] However, in our study, we observed that, in many cases, there were multiple published RCTs available to address a single clinical question, providing an abundance of data that may be more than necessary to adequately inform decision making. For example, we identified 4 published RCTs evaluating the benefit of radiation dose escalation in patients with prostate cancer. In contrast, level I evidence was sparse when considering medical decisions for radiation dose escalation or for stereotactic body radiation therapy in patients with lung cancer, which is the second most common cancer diagnosis and the leading cause of cancer-related death in the United States. These findings highlight the potential utility of well designed RCTs in patients who have diagnoses for which there is a paucity of available level I evidence.

We acknowledge that our study has limitations. Our study cohort was obtained from a single radiation oncology department at an academic institution. Patients who attend an academic medical center may be more likely to have complicated or uncommon diagnoses; therefore, our study sample may differ significantly from the patients who attend community radiation oncology practices. In addition, a single-institutional sample may not be representative of patient characteristics in other geographic regions of the country. We attempted to minimize the potential risk of selection bias by studying a relatively large cohort of consecutive patients. We elected to exclude pediatric patients from the study because of the small number of pediatric patients at our institution. In addition, we excluded patients with stage IV disease who were under consideration for palliative radiation because of concerns regarding the clinical heterogeneity of this patient group. Future studies evaluating the availability of level I evidence in other clinical settings and patient populations would be potentially useful in determining the greatest needs for additional RCTs. Finally, although we identified RCTs for study purposes through a systematic online search, it is possible that there are additional, previously conducted RCTs that were not identified during this process. However, because they are not acknowledged by major groups evaluating therapeutic options, those trials are not available for medical decision-making analyses.

In conclusion, in our current study, we systematically evaluated the availability and relevance of published RCTs for medical decision making in routine radiation oncology care. We observed that a significant proportion of all medical decisions were made without available or applicable level I evidence. These findings underscore the limitations of relying solely on prospective RCTs for the development of evidence-based health care. In addition, the variation in availability of evidence identified in our study may encourage funding agencies and cooperative trial groups to allocate additional resources for new clinical trials in these areas of need. Our study highlights the need for ongoing systematic evaluations of available high-quality data and existing gaps in evidence that will be important for future improvements in the quality and value of oncology care.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. FUNDING SUPPORT
  8. CONFLICT OF INTEREST DISCLOSURES
  9. REFERENCES
  • 1
    Silverman WA, Chalmers I. Sir Austin Bradford Hill: an appreciation. Control Clin Trials. 1992;13:100-105.
  • 2
    Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637-639.
  • 3
    National Comprehensive Cancer Network (NCCN). NCNN Clinical Practice Guidelines in Oncology (version 1.2007). Available at: www.nccn.org/professionals/physician_gls/f_guidelines.asp#site. Ac-cessed March 15, 2008.
  • 4
    American Society of Clinical Oncology. Clinical Practice Guidelines. Available at: http://www.asco.org/ascov2/Practice+&+Guidelines/Guidelines/Clinical+Practice+Guidelines. Accessed March 20, 2012.
  • 5
    US Senate. Patient Protection and Affordable Care Act 2010. Available at: http://democrats.senate.gov/pdfs/reform/patient-protection-affordable-care-act.pdf. Accessed September 7, 2012.
  • 6
    US Preventive Services Task Force Grade Definitions. Available at: http://www.uspreventiveservicestaskforce.org/uspstf/grades.htm. Accessed September 7, 2012.
  • 7
    UK National Health Service. Oxford Centre for Evidence-based Medicine – Levels of Evidence. Available at http://www.cebm.net/index.aspx?o=1025. Accessed September 7, 2012.
  • 8
    Freedman B. Equipoise and the ethics of clinical research. N Engl J Med. 1987;317:141-145.
  • 9
    Jadad AR, Rennie D. The randomized controlled trial gets a middle-aged checkup. JAMA. 1998;279:319-320.
  • 10
    Brass EP. The gap between clinical trials and clinical practice: the use of pragmatic clinical trials to inform regulatory decision making. Clin Pharmacol Ther. 2010;87:351-355.
  • 11
    Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967;20:637-648.
  • 12
    Bekelman JE, Shah A, Hahn SM. Implications of comparative effectiveness research for radiation oncology. Practical Radiat Oncol. 2011;1:2-80.
  • 13
    Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:1624-1632.
  • 14
    Elting LS, Cooksley C, Bekele BN, et al. Generalizability of cancer clinical trial results: prognostic differences between participants and nonparticipants. Cancer. 2006;106:2452-2458.
  • 15
    Flather M, Delahunty N, Collinson J. Generalizing results of randomized trials to clinical practice: reliability and cautions. Clin Trials. 2006;3:508-512.
  • 16
    National Cancer Institute (NCI). NCI Physician Data Query (PDQ) Guidelines. Available at: http://www.cancer.gov/cancertopics/pdq/adulttreatment. Accessed March 15, 2008.
  • 17
    Bonner JA, Harari PM, Giralt J, et al. Radiotherapy plus cetuximab for squamous-cell carcinoma of the head and neck. N Engl J Med. 2006;354:567-578.
  • 18
    Jeremic B, Shibamoto Y, Acimovic L, Milisavljevic S. Hyperfractionated radiation therapy with or without concurrent low-dose daily carboplatin/etoposide for stage III non-small-cell lung cancer: a randomized study. J Clin Oncol. 1996;14:1065-1070.
  • 19
    Feinberg BA, Lang J, Grzegorczyk J, et al. Implementation of cancer clinical care pathways: a successful model of collaboration between payers and providers. J Oncol Pract. 2012;8(3 suppl):e38s-e43s.
  • 20
    Arrow K, Auerbach A, Bertko J, et al. Toward a 21st-century health care system: recommendations for health care reform. Ann Intern Med. 2009;150:493-495.
  • 21
    Maclennan SJ, Imamura M, Omar MI, et al. Urological cancer care pathways: development and use in the context of systematic reviews and clinical practice guidelines. World J Urol. 2011;29:291-301.
  • 22
    Lougheed MD, Olajos-Clow JG. Asthma care pathways in the emergency department. Curr Opin Allergy Clin Immunol. 2010;10:181-187.
  • 23
    Marrie TJ, Lau CY, Wheeler SL, Wong CJ, Vandervoort MK, Feagan BG. A controlled trial of a critical pathway for treatment of community-acquired pneumonia. CAPITAL Study Investigators. Community-Acquired Pneumonia Intervention Trial Assessing Levofloxacin. JAMA. 2000;283:749-755.
  • 24
    Maher J. A piece of my mind. Lack of data. JAMA. 2012;308:1331-1332.
  • 25
    Garber AM. Evidence-based coverage policy. Health Aff (Millwood). 2001;20:62-82.
  • 26
    Antman K, Amato D, Wood W, et al. Selection bias in clinical trials. J Clin Oncol. 1985;3:1142-1147.
  • 27
    Edwards SJ, Lilford RJ, Braunholtz DA, Jackson JC, Hewison J, Thornton J. Ethical issues in the design and conduct of randomised controlled trials. Health Technol Assess. 1998;2:i-vi, 1-132.