Demographic analysis of randomized controlled trials in bladder cancer

Authors


  • Wassim Kassouf is a recipient of a Research Scholar Award from the Fonds de la recherche en santé du Québec (FRSQ)

Correspondence: Wassim Kassouf, Division of Urology, McGill University Health Center, 1650 Cedar Avenue, Rm L8-315, Montreal, Quebec H3G?1A4, Canada.e-mail: wassim.kassouf@muhc.mcgill.ca

Abstract

What's known on the subject? and What does the study add?

  • Results from well designed randomized controlled trials usually provide the strongest evidence possible in favour of one medical intervention over another. For this reason, it is of paramount importance to conduct such trials in bladder cancer, where randomized trials are lacking, in particular to answer questions that have so far confounded us or to investigate the efficacy of new diagnostic tools or interventions.
  • This study provides a demographic analysis of randomized controlled trials published in bladder cancer between the years of 1995 and 2010, with only 238 articles identified. Less than one-third of these reported a statistical power calculation, and only 8% were double-blinded. With many publications inaccurately labelled as randomized trials, we reveal the scarcity of trials performed over the given time period, even compared with other cancers with similar incidence, and highlight the need for more well designed trials to be conducted.

Objective

  • To demographically examine randomized controlled trials (RCTs) that have been conducted in bladder cancer over a predefined time period.

Methods

  • Various techniques have been described to detect RCTs using different databases.
  • We searched the MEDLINE database by crossing the heading ‘Urinary bladder neoplasms’ with the MeSHs ‘Clinical trial$.mp. OR clinical trial.pt. OR random:.mp. OR tu.xs.’ between 1995 and 2010.
  • For the RCTs identified, analysis was performed on each RCT, placing particular emphasis on modality of intervention, cohort size, principal author, region, journal type, disease status, histology, blinding, number of centres involved, performance of a statistical power calculation, accrual status and trial support.

Results

  • Of 5002 RCT bladder cancer papers retrieved over the given period, only 238 represented actual RCTs after manual appraisal.
  • More than half of the RCTs investigated medical and surgical therapies (54.2%), and only half had a sample size of >100 patients.
  • A small percentage of studies were double-blinded (8.0%), and there was an almost equal distribution of multicentre vs single centre trials (54.6% vs 45.4%).
  • More studies were conducted in Europe (61.3%) than the rest of the world combined, with urologists principally the lead investigators in the majority (72.3%).
  • Most studies were conducted on patients with urothelial carcinoma (97.1%), with less than one-third reporting a statistical power calculation (31.5%).

Conclusions

  • Only 238 RCTs were published for bladder cancer between 1995 and 2010.
  • RCTs are under-utilized in bladder cancer.
  • More trials need to be designed with larger sample sizes in order to optimize diagnostic and treatment strategies for patients with bladder cancer.
Abbreviations
RCT

randomized controlled trial

SCC

squamous cell carcinoma

TCC

transitional cell carcinoma

Introduction

Randomized controlled trials (RCTs) are characterized by both the random allocation of patients to different treatment arms and preferably blinding both patient and trialist to prevent bias from preconceived views. Results from well designed RCTs usually provide the strongest evidence possible in favour of one intervention over another. For this reason, it is of utmost importance to construct such trials, in particular to answer questions that have so far confounded us or to investigate the efficacy of new diagnostic tools or interventions. We conducted a concise search of the MEDLINE literature to review all RCTs in bladder cancer over a predefined time period.

Methods

All publications on RCTs within the time period 1995–2010 were manually appraised and data were recorded for each one under 12 major categories: modality of intervention, cohort size, principal author, region, journal type, disease status, histology, blinding status, number of centres involved, performance of a statistical power calculation, accrual status and trial support.

Several search strategies have been described as providing the greatest sensitivity in searching the MEDLINE database for RCTs. Using the search string with the MeSH ‘Clinical trial.mp. OR clinical trial.pt. OR random:.mp. OR tu.xs.’ is described as having a sensitivity of 99.3% [1]. We applied a slight modification and used the MeSH ‘Clinical trial$.mp. OR clinical trial.pt. OR random:.mp. OR tu.xs.’ to possibly include more papers, such as for example those with the heading ‘clinical trials’ or ‘clinical trialist’. Papers specifically on bladder cancer were easily retrieved by exploding the medical subject heading ‘Urinary bladder neoplasms’, limiting our search to papers in English. We applied an additional year-by-year limit from 1995 to 2010 to come up with the total number of RCTs under urinary bladder neoplasms for every given year. An easily reproducible and detailed outline of our search strategy is presented in the Appendix. All abstracts were then individually appraised to distinguish actual RCTs from publications erroneously labelled as RCTs. Abstracts that presented preliminary data on ongoing trials were also included, as were different abstracts based on the same patient population.

The modalities of intervention included all possible treatment combinations in bladder oncology. For example, a trial comparing two different intravesical therapies after transurethral resection was labelled as a medical and surgical trial, since the resection is an integral part of the intervention. On the other hand, a trial comparing two different chemotherapeutic regimens for advanced disease was labelled as medical. Several trials also looked at different combinations of anaesthetics used during surgery, and these were included under ‘other’. Although it can be argued that such trials can be considered as comparing two different surgical approaches, we decided to categorize such trials separately to distinguish them from ones that really assessed operative differences on patients' outcomes. For example, a surgical trial would be one comparing two different techniques of ureteral implantation during radical cystectomy. With urology being a surgical specialty, all urology journals were categorized as surgical.

Results

From 1995 to 2010 there were 29 106 published papers on bladder cancer. The total number of RCTs in bladder oncology retrieved over the given time period was 5002 papers. Of these, only 238 (4.8%) were found to be actual RCTs after manual appraisal. Most excluded papers included comment letters, review articles and meta-analyses. Table 1 shows the percentage of papers retrieved every year that actually turned out to be RCTs after individual appraisal. It also shows a somewhat similar number of RCTs being published yearly in this domain.

Table 1. Number of RCTs before and after manual appraisal of MEDLINE
YearBefore manual appraisalAfter manual appraisalNumber of articles from MEDLINE search (%)
1995266228.3
1996288227.6
1997246135.3
1998226125.3
1999256155.9
2000319113.4
2001296144.7
2002311154.8
2003299124.0
200428393.2
2005322134.0
2006353133.7
2007359133.6
2008322185.6
2009411143.4
2010445224.9
Total50022384.8

The cumulative data for all 238 articles retrieved are shown in Table 2, and these data are broken down into a year-by-year basis in Table 3. More than half of the modalities of intervention included combined surgical and medical therapies (54.2%), and only half (54.6%) of the overall RCTs had cohorts of more than 100 patients. More trials were conducted in Europe (61.3%) than in the rest of the world combined, with urologists being the lead investigators in the majority (72.3%). More than half of the studies were published in surgical journals (59.2%) and were mostly on non-muscle- invasive disease (75.2%). Only series in which the predominant histology was squamous cell carcinoma (SCC) were labelled as so. This means that a certain number of studies with urothelial carcinoma as the main histology may have still included a small subset of patients with SCC or adenocarcinoma. Only a small percentage of studies were double-blinded (8.0%), and there was an almost equal distribution of multicentre vs single centre trials (54.6% vs 45.4%). Less than one-third of trials reported a statistical power calculation (31.5%) and an even fewer number reported completing planned accrual (23.5%). Lastly, only a handful of trials received industry support (14.3%).

Table 2. Cumulative data for all 238 articles over a 16-year period
Modality of interventionMedical2811.8%
Surgical208.4%
Radiation52.1%
Medical + surgical12954.2%
Surgical + radiation10.4%
Medical + radiation41.7%
Medical + surgical + radiation62.5%
Screening177.1%
Quality of life104.2%
Other187.6%
Cohort size10–504318.1%
51–1006527.3%
>10013054.6%
Principal authorUrologist17272.3%
Medical oncologist2811.8%
Radiation oncologist93.8%
Other2912.2%
RegionEurope14661.3%
USA2912.2%
Canada31.3%
Asia/Middle East4820.2%
Africa104.2%
South America  
Australia/New Zealand20.8%
Journal typeMedical5623.5%
Surgical14159.2%
Radiation oncology52.1%
Basic science41.7%
Other3213.4%
DiseaseNon-muscle-invasive17975.2%
Muscle-invasive4418.5%
Advanced/metastatic145.9%
Other10.4%
HistologyTCC23197.1%
SCC31.3%
Adenocarcinoma  
Other41.7%
Study typeClinical23699.2%
Pre-clinical20.8%
BlindingDouble-blind198.0%
Other21992.0%
CentreMulticentre13054.6%
Single centre10845.4%
Power calculationYes7531.5%
 No16368.5%
Accrual targetCompleted5623.5%
Stopped early177.1%
Not reported16569.3%
Trial supportCooperative group trial6828.6%
Industry sponsored198.0%
Both156.3%
Neither13657.1%
Table 3. RCTs in bladder cancer from 1995 to 2010
 Year (number of RCTs)
1995 (22)1996 (22)1997 (13)1998 (12)1999 (15)2000 (11)2001 (14)2002 (15)2003 (12)2004 (9)2005 (13)2006 (13)2007 (13)2008 (18)2009 (14)2010 (22)
Modality of interventionMedical1323223111241 11
Surgical11 1  1211111135
Radiation     2  111     
Medical + surgical1616966569756767612
Surgical + radiation  1             
Medical + radiation 1 11         1 
Medical + surgical + radiation 1 111 1    1   
Screening1   31 12 111222
Quality of life1 1   4     22  
Other2   2  1 12 1612
Cohort size10–504432314321332233
51–1009823532143242755
>1009108777811658699614
Principal authorUrologist2016127976129771110121116
Medical oncologist1311222122312122
Radiation oncologist 2 2 2  1   1  1
Other11 24 62  31 513
RegionEurope14157510861054106714916
USA3522 14 4  31112
Canada 1 1  1         
Asia/Middle East5124413514333333
Africa  2 1   11 12 11
South America                
Australia/New Zealand     1  1       
Journal typeMedical5413334461363154
Surgical1315125767756861011716
Radiation oncology0    1  111   1 
Basic science1   1 1        1
Other33 44124 111 611
DiseaseNon-muscle-invasive1913109126913868711171219
Muscle-invasive2822344232331122
Metastatic1111 11 11221  1
Other           1    
HistologyTCC2222131213111415128131112181322
SCC         1 1  1 
Adenocarcinoma                
Other    2      11   
Study typeClinical2122131215111415129131313181322
Pre-clinical1             1 
BlindingDouble-blind3 1 11 11 22 511
Other1922121214101414119111113131321
CentreMulticentre101167789135491065614
Single centre1211758352754371388
Power calculationYes4722565553435748
No18151110105910769108111014
Accrual targetCompleted34  463441234648
Stopped early13321 2 111 11  
Not reported181510101059117710108111014
Trial supportCooperative group trial9966452521423136
Industry sponsored1  11    1242214
Both    221111211111
Neither121375841199656714911

We also conducted a review of ongoing registered trials to determine what percentage of these were RCTs in bladder cancer [2]. Out of 115 636 registered trials being conducted in locations across 178 countries, only 166 (0.14%) were currently open trials in bladder cancer. We examined these trials and recovered only 34 RCTs. The cumulative data for these RCTs are presented in Table 4. Certain similarities and important differences can be noted with the data presented herein. Although urologists are still the primary lead investigators, we notice that more trials are currently being conducted in the USA than anywhere else. In addition, there is a tendency towards higher enrolment rates, probably due to more multicentre trials being conducted. All currently ongoing trials that are actively recruiting patients are on patients with urothelial carcinoma. Furthermore, there is a higher percentage of double-blinded RCTs being conducted than previously noted, presumably due to improved trial design.

Table 4. Cumulative data for the 34 ongoing RCTs in bladder cancer currently recruiting
Modality of interventionMedical1441.2%
Surgical514.7%
Radiation  
Medical + surgical1029.4%
Surgical + radiation  
Medical + radiation25.9%
Medical + surgical + radiation  
Screening12.9%
Quality of life25.9%
Other  
Estimated enrolment10–5012.9%
51–100926.5%
>1002470.6%
Principal investigatorUrologist1750.0%
Medical oncologist617.6%
Radiation oncologist25.9%
Other926.5%
RegionEurope1338.2%
USA1852.9%
Canada25.9%
Asia/Middle East12.9%
Africa  
South America  
Australia/New Zealand  
DiseaseNon-muscle-invasive1338.2%
Muscle-invasive1132.4%
Advanced/metastatic1029.4%
Other  
HistologyTCC34100%
SCC  
Adenocarcinoma  
Other  
Study typeClinical34100%
Pre-clinical  
BlindingDouble-blind926.5%
Other2573.5%
CentreMulticentre2264.7%
Single centre1235.3%
Trial supportCooperative group trial823.5%
Industry sponsored1132.4%
Both38.8%
Neither1235.3%

Discussion

Only a very small number of papers retrieved yearly actually turned out to be RCTs after manual appraisal (3.2%–8.3%). An almost identical result was shown in a recent demographic analysis by Wehbi et al. [3] on prostate cancer (5.4%–8.6%), although the absolute number of trials conducted yearly was considerably higher. In their study, 2.2% (517 out of 23 905) of all papers retrieved on prostatic cancer were RCTs. In our analysis, this percentage is as low as 1.4% (127 out of 8827) if we limit our time frame to that of Wehbi et al.'s (1997–2006). This is also in concordance with a study performed by Sauerland et al. [4] that showed, in general, the low number of RCTs being published yearly in both medical and surgical journals. This, however, is not surprising given the low number of RCTs conducted yearly. For example, in Canada only two trials were open for patients with bladder cancer in 2011, and the total number of patients accrued to both is still less than 100 (RTOG 0926, SPRUCE) [5-9]. In fact, the true number of unique RCTs is even lower than the 238 that we retrieved, given that 33 reviewed abstracts either presented preliminary data on ongoing RCTs that were later published in full or are different abstracts derived from the same patient population involved in one RCT. However, most of these were republished after a considerable follow-up interval beyond what was planned for the original study, and thus have been included as unique RCTs in our analysis. To investigate whether the low number of RCTs in bladder cancer was a problem particular to this organ site, we compared the number of RCTs in bladder cancer with that of another cancer (non-Hodgkin's lymphoma) that is known to have a similar incidence [10]. We performed a quick search of the same MEDLINE database by exploding the heading ‘Lymphoma, Non-Hodgkin’, limiting our search to RCTs published in English over the same time period noted above. Although considerably less extensive than our search for bladder cancer RCTs, it nevertheless provided some evidence for the under-utilization of RCTs in bladder cancer specifically. Out of 35 988 papers retrieved on non-Hodgkin lymphoma from 1995 to 2010, 495 were RCTs in our search. This was more than double the number of RCTs we retrieved on bladder cancer (238) over the same time period.

There are several challenges to setting up proper RCTs in bladder oncology including disease incidence and mean patient age. Based on recent Surveillance, Epidemiology and End Results (SEER) data from the National Cancer Institute in the USA, the age-adjusted incidence rate of bladder cancer was around 21.1 per 100 000 men and women per year with a median age at diagnosis of 73 years. In addition, 84% of mortality from this disease occurs in patients over 65 years of age [11]. Patient accrual is not easy because of the low incidence of disease. In addition, it is more difficult to ask an elderly patient to present for regular follow-ups because of age and the likelihood of multiple comorbidities that would render their journeys to and from hospital all the more cumbersome. These multiple comorbidities more often than not also preclude these patients from entry into most well designed RCTs that strive to reduce dropout rates as much as possible. Lastly, and particularly pertaining to second-line therapy for metastatic bladder cancer, poor accrual is mostly secondary to lack of tolerance to further therapy after receiving significantly toxic first-line chemotherapy. Furthermore, designing therapeutic trials based on molecular pathways in bladder cancer has been lacking, as the biology of this disease remains poorly clarified [12].

The importance of blinding cannot be overstressed as even the most dedicated clinician can be influenced by preconceived opinions. The impact of blinding was shown in a paper by Noseworthy et al. [13] in which the unblinded physician demonstrated a treatment advantage of one intervention over another when there was actually no statistically significant difference. In our analysis, only 8.0% of all RCTs reviewed were double-blinded, highlighting the challenge of setting up double-blinded trials.

The advantage of multicentre RCTs is the ability to accrue many more patients than would be possible with one single centre, particularly in a disease such as bladder cancer where the incidence is relatively low; however, the cost and organizational manpower needed to run such a trial makes setting them up a challenge [14]. This is also true given that research in this organ disease site is heavily underfunded. The essence of RCTs is administering the same treatment to patients within the same treatment groups. With multicentre RCTs, particularly those involving surgical arms, this can prove to be quite a challenge because of the inherent differences in surgical techniques and practices amongst surgeons not only from the same institution but also from different institutions and regions.

A recent paper by Paramasivan et al. [15] also highlighted several other key challenges to recruitment to RCTs that may partially explain the low number of RCTs that we noted in bladder cancer. These included difficulty in explaining trial design to potential participants, complicated recruitment pathways involving multiple staff from different specialities that often led to a breakdown in communication, use of impressionable phrases such as ‘gold standard’ that may lead to a biased presentation as well as inadvertent treatment preference presented by some recruiters [15].

Even after RCTs start patient recruitment, several other challenges arise. The reliability of conclusions derived from various RCTs depends to a great extent on quality control, in particular quality control in the implementation of the research protocol as well as building meaningful databases by quality data collection [16, 17]. Without these, it may be difficult to reach significant conclusions, which will only leave both researchers and patients feeling frustrated [18, 19]. There is evidence, however, that trial design and data collection have improved, owing to advanced automated data collection systems and strict adherence to trial protocol [20]. However, even when RCTs are completed, there is a lack of focus on clinical effectiveness, which may also serve as a barrier to subsequent RCT design. For example, RCTs in bladder cancer that demonstrated efficacy for neoadjuvant chemotherapy or postoperative immediate instillation of mitomycin have had very little uptake in the urological community. Lastly, despite challenges to the hierarchy of research design, it still firmly holds that results of RCTs provide the highest grade of evidence. At best, the results of other studies, including well designed observational studies, serve only as useful adjuncts to the results of RCTs [21, 22].

In conclusion, only 238 RCTs were published for bladder cancer between 1995 and 2010. RCTs are under-utilized in bladder cancer. More trials need to be designed with larger sample sizes in order to optimize diagnostic and treatment strategies for patients with bladder cancer.

Conflict of Interest

None declared.

Appendix: Appendix

Database: Ovid MEDLINE(R) <1948 to August Week 2 2011>

Search strategy

  1. exp Urinary Bladder Neoplasms/(40127)
  2. limit 1 to english (29106)
  3. “1995”.yr. (421480)
  4. “1996”.yr. (426037)
  5. “1997”.yr. (437307)
  6. “1998”.yr. (451567)
  7. “1999”.yr. (464275)
  8. “2000”.yr. (491011)
  9. “2001”.yr. (511844)
  10. “2002”.yr. (528556)
  11. “2003”.yr. (556005)
  12. “2004”.yr. (587070)
  13. “2005”.yr. (618837)
  14. “2006”.yr. (646084)
  15. “2007”.yr. (672190)
  16. “2008”.yr. (708018)
  17. “2009”.yr. (734935)
  18. “2010”.yr. (791444)
  19. 2 and 3 (741)
  20. 2 and 4 (766)
  21. 2 and 5 (742)
  22. 2 and 6 (692)
  23. 2 and 7 (779)
  24. 2 and 8 (905)
  25. 2 and 9 (889)
  26. 2 and 10 (884)
  27. 2 and 11 (926)
  28. 2 and 12 (956)
  29. 2 and 13 (984)
  30. 2 and 14 (1070)
  31. 2 and 15 (1111)
  32. 2 and 16 (1089)
  33. 2 and 17 (1162)
  34. 2 and 18 (1311)
  35. Clinical trial$.mp. or clinical trial.pt. or random:.mp. or tu.xs. [mp = protocol supplementary concept, rare disease supplementary concept, title, original title, abstract, name of substance word, subject heading word, unique identifier] (3644847)
  36. 19 and 35 (266)
  37. 20 and 35 (288)
  38. 21 and 35 (246)
  39. 22 and 35 (226)
  40. 23 and 35 (256)
  41. 24 and 35 (319)
  42. 25 and 35 (296)
  43. 26 and 35 (311)
  44. 27 and 35 (299)
  45. 28 and 35 (283)
  46. 29 and 35 (322)
  47. 30 and 35 (353)
  48. 31 and 35 (359)
  49. 32 and 35 (322)
  50. 33 and 35 (411)
  51. 34 and 35 (445)

Ancillary