Fax: (734) 763-7370
Under-representation of women in high-impact published clinical cancer research†
Version of Record online: 8 JUN 2009
Copyright © 2009 American Cancer Society
Volume 115, Issue 14, pages 3293–3301, 15 July 2009
How to Cite
Jagsi, R., Motomura, A. R., Amarnath, S., Jankovic, A., Sheets, N. and Ubel, P. A. (2009), Under-representation of women in high-impact published clinical cancer research. Cancer, 115: 3293–3301. doi: 10.1002/cncr.24366
Presented in preliminary form at the American Society of Clinical Oncology 44th Annual Meeting, Chicago, Illinois, May 30–June 3, 2008.
- Issue online: 15 JUL 2009
- Version of Record online: 8 JUN 2009
- Manuscript Accepted: 22 DEC 2008
- Manuscript Revised: 16 DEC 2008
- Manuscript Received: 8 AUG 2008
- University of Michigan Institute
Adequate representation of women in research has been deemed essential.
Cancer research published in 8 journals in 2006 was reviewed. The percentage of women among study participants was compared with the proportion expected from population-based estimates of sex-specific cancer incidence, using binomial tests. Differences were assessed in sex distribution of participants by funding source, author sex, and focus of research with the Student t test, and in a linear regression model controlling for cancer type.
A total of 1534 cancer research articles were identified, of which 661 (representing 1,096,098 participants) were prospective clinical studies and were analyzed further. For all 7 non-sex–specific cancer types assessed, the majority of studies analyzed included a lower proportion of women than the proportion of women among patients having cancer of that type in the general population. Among studies focusing on cancer treatment, women constituted a significantly lower overall proportion of the participants in the analyzed studies than expected for 6 of 7 non-sex–specific cancer types (P < .001). Among non-sex–specific studies, the mean percentage of participants who were women was 38.8%. Non-sex–specific studies reporting government funding had a higher percentage of female participants (mean 41.3% vs 36.9%; P = .005). In a regression model controlling for cancer type, lack of government funding (P = .03) and focus on cancer treatment (P = .03) were found to be independent significant predictors of a lower percentage of female participants.
Women were under-represented as participants in recently published, high-impact studies of non-sex–specific cancers. Studies that received government funding included a higher proportion of female subjects. Cancer 2009. © 2009 American Cancer Society.
In 2001, the Institute of Medicine issued a report emphasizing that sex was “an important basic human variable that should be considered when designing and analyzing studies in all areas and at all levels of biomedical and health-related research”.1 Indeed, sex may be a predictor not only of the incidence of disease, but also of the utility of diagnostic tests, preventive interventions, prognostic markers, and therapeutics. Not only may biologic differences between the sexes be important but also social and cultural influences based on sex, including gender differences in the approach of men and women with regard to their physicians and their own health, and how they communicate their health concerns.2 The National Institutes of Health (NIH) has similarly recognized the critical role of sex in medicine; its Revitalization Act of 1993 explicitly articulated the importance of including women in each clinical research project and noted that clinical trials should enroll adequate numbers of female participants for performing valid subanalyses of the effect of trial variables on women alone.3
Fifteen years after the enactment of the NIH policy, however, it is still unclear whether women are adequately represented as subjects of clinical research. Whereas some analyses have suggested that women are adequately represented, others have suggested otherwise. For example, a study of federally funded randomized trials reported in 9 medical journals in 2004 found that women constituted only 37% of participants in non-sex–specific studies and 24% of participants in drug trials.4
Several previous studies have considered participation in cancer research specifically and also yielded mixed results. A study analyzing enrollment in Southwest Oncology Group trials from 1993 to 1996 revealed significant disparities between the proportion of women among enrolled trial participants and the proportion of women among the general population of US cancer patients for 3 of 11 cancer types they examined.5 Another study of participation in cooperative group trials in 2000 through 2002 found that women were significantly less likely than men to enroll in trials for colorectal cancer (odds ratio, 1.30) and lung cancer (odds ratio, 1.23).6 Yet, a different study of accrual to cooperative group studies found that overall, women did not accrue at lower rates and in fact, between the ages of 40 and 69 years, accrued at even higher rates than men. However, this finding was driven largely by participation in breast and gynecologic cancer studies; when studies were limited to non-sex–specific cancers, fewer women accrued than men.7
Despite the compelling nature of these findings, the researchers who conducted these studies have themselves acknowledged that their results may not be representative because their studies focused exclusively on cooperative group trial participation. Indeed, whereas cooperative group trials are funded by government sources, studies funded by industry or private philanthropy may be less sensitive to the need to enroll and analyze female patients than federally funded studies subject to the mandates of the NIH. Furthermore, patterns of enrollment in trials studying the impact of treatments may differ from enrollment in studies seeking to examine mechanisms of cancer causation and prevention. Therefore, further investigation is warranted to define better the representation of women as subjects in the full range of high-impact, clinical cancer research published today. This is particularly important in light of changes in political priorities that have made the competition for scarce federal research funds increasingly intense, potentially fueling an even greater reliance on private support. Indeed, although several other studies have also examined women's representation among research participants, they too have generally focused on federally funded clinical trials8-12 or have been limited in other ways, such as focusing on studies published in only 1 journal.13 These studies have primarily focused on quantifying the representation of women rather than analyzing the role of various factors, such as funding source, in influencing their representation. The rare exceptions are dated.14 Therefore, in this study, we sought both to update and build upon this previous work.
In this study, we attempted to quantify the representation of women as subjects in clinical cancer research studies, including not only trials but also prospective cohort studies, that result in high-impact publications. In addition, we assessed whether studies funded privately are as likely to include female participants as those with federal funding.
MATERIALS AND METHODS
This study focused on original research published in 2006 in selected high-impact, English-language journals featuring original clinical oncology research. Journals were selected after consideration of journal impact factors, citation half-life, and readership. Only journals primarily publishing original clinical research were included; journals focused on publishing reviews or basic science investigation were excluded. Three journals catalogued by the Thomson ISI Journal Citation Index as general medical journals were included: New England Journal of Medicine, JAMA, and The Lancet. Five journals catalogued as oncology journals were also included: Journal of Clinical Oncology, Journal of the National Cancer Institute, The Lancet Oncology, Clinical Cancer Research, and Cancer. All articles listed as original investigations were identified by hand-searching (in Clinical Cancer Research, only articles under the subheading ‘Cancer Therapy: Clinical’ were included); special articles, editorials, and review articles were excluded.
All original oncology articles were then read to determine several objective attributes: cancer type(s), declared source(s) of funding, numbers of participants (by sex when reported), and sex of primary and senior (final) authors. Author sex was determined by inspection of the author's name; for names in which sex was ambiguous, Internet searching with the Google search engine was used in an attempt to determine sex, as described elsewhere.15 Each study was also subjectively coded for study type; prospective clinical studies were defined as prospective analyses of clinical trials and prospective cohort studies, not including meta-analyses of prospective studies. Data were entered by 2 medical student coders into a Microsoft Access database.
To ensure inter-rater reliability, 10% of the articles, distributed across all included journals, were assessed independently by both coders. All discrepancies in this subset were analyzed further to determine the nature and frequency of coding disagreements for each item. Discrepancies were resolved by the consensus of 2 additional individuals, and overall error rates for each item were thus determined. For all objective attributes assessed, error rates were low and well below 10%. Before analysis, we also applied error-trapping techniques to the entire data set, including verification of outlier cases, double-coding of the 5 largest studies for each cancer type, and simple arithmetic confirmation that total n = n female + n male + n unknown for each case. Two coding categories required subjective assessment on the part of the coders: classification of the study as a prospective clinical oncology study versus other type of study, and classification of the study focus on treatment or not. Regarding classification of the study as a prospective clinical oncology study, the 2 coders were found to disagree in 11% of the cases in the overlap sample. This corresponded to a kappa of 0.74 (95% confidence interval [95% CI], 0.65-0.83), indicative of good interobserver agreement.16 For classification of the study focus as treatment, the 2 coders were found to disagree in 16% of the cases in the overlap sample. This corresponded to a kappa of 0.63, which also met commonly accepted standards for good interobserver agreement.
Each article was analyzed as a separate case; with this design, studies that yielded multiple publications in the studied journals in 2006 would have a greater influence on the results than studies published once. Studies never published or those published in lower-impact journals would not have an impact on the results.
Statistical analysis was performed using the SPSS 14.0 statistical software package. The percentage of participants who were female was defined as the number of female participants divided by the sum of the number of female participants and number of male participants. Percentage female across all cases was summarized in 2 ways: first, as the overall percentage female when all participants were included (an estimate that is sensitive to the effects of larger studies); and second, as the mean percentage female (allowing each study to count equally, avoiding the possibility of the result being driven primarily by a few large studies).
The percentage of incident cancer cases occurring in women was determined from incidence data by cancer type for the year 2000, as reported by Greenlee et al,17 based on population data reported by the U.S. Bureau of the Census and age-specific cancer incidence rates collected by the National Cancer Institute's Surveillance, Epidemiology, and End Results program. For comparisons of percentage female among study subjects and the percentage of incident cases of each cancer type occurring in women, incidence rates were considered to be fixed; 1-sample binomial tests were used, and the unit of analysis was the individual study participant, as suggested by Hutchins et al.5 For these comparative analyses, percentage female among study subjects was limited to those studies with a focus on cancer treatment.
For analyses of associations between study characteristics (author sex, funding source, and focus of research) and the sex distribution of participants, articles considering cancers occurring exclusively in 1 sex (or almost exclusively, such as breast cancer) were excluded. For these analyses, the unit of analysis was the individual paper (rather than the individual participant), and differences were assessed using the Student t test and multivariate linear regression. The dependent variable in the regression model was percentage of female participants in any given non-sex–specific study, and the independent variables were single dummy variables for the characteristics of author sex (either first or senior author female vs not), focus of research (on treatment vs not), and funding source (government vs not), as well as a set of dummy variables for cancer type (8 dummy variables representing the non-sex–specific cancer types listed in Table 1, with studies that were not specific to 1 cancer type constituting the base category). Significance was defined at the P = .05 level.
|Clin Cancer Res||144||9.4|
|J Clin Oncol||565||36.8|
|J Natl Cancer Inst||123||8.0|
|N Engl J Med||31||2.0|
|Prostate, testis, penis||120||7.8|
|Head and neck||67||4.4|
|Not site specific||284||18.5|
|Nature of study|
|Prospective clinical study||661||43.1|
We identified 2701 original articles published in the selected journals in 2006, of which 1534 were oncology studies. The characteristics of these studies are detailed in Table 1. Of these, 661 were prospective clinical studies; these constituted the data set for further analysis in this study. These studies included the analysis of 1,096,098 participants in total (mean participants per study, 1658; median, 83).
Table 2 details the number of studies and sex distribution of participants by cancer type. For most cancer types, the mean percentage of female participants across all studies was similar to the overall percentage of females calculated from the sum of all participants in studies of that disease type. However, in some cases, the 2 differed substantially, indicating that the percentage female in a few large studies differed from the percentage female in other studies of that disease type. This was most noticeably the case for nontreatment studies of gastrointestinal tumors, which was the category that included the 3 largest cancer type-specific, non-sex–specific studies analyzed. These 3 large gastrointestinal studies were prospective cohort studies that included 36,282, 48,835, and 368,277 participants, respectively. The percentages of participants who were female in these studies were 100%, 100%, and 64.8%, respectively. All other cancer type-specific, non-sex–specific studies were considerably smaller, and none had more than 5000 participants.
|Type of Cancer*||No. of Studies Reporting Sex of Subjects||No. of Participants With Sex Known||Mean % Female†||Overall % Female‡|
|Prostate, testis, penis||31/31||8702||0||0|
|Head and neck||14/14||1138||18.8||19.9|
|Prostate, testis, penis||15/15||74,679||0||0|
|Head and neck||7/7||1241||19.5||20.0|
Figure 1 depicts the percentage of participants who were female in 316 cancer type-specific, non-sex–specific papers, compared with the percentage female expected from incidence data based on cancer type. In this figure, each study is represented by a separate point; the position of the point is determined based on a comparison of the percentage female in that study with the percentage female among incident cases of the cancer type studied. Thus, points falling above the midline represent studies in which the percentage of female participants was higher than expected; points below midline represent studies in which the percentage of female participants was lower than expected. As the figure illustrates, in the vast majority of the non-sex–specific studies we reviewed, the percentage of participants who were women was lower than that expected based on incidence data.
Figure 2 compares the percentage female among participants in the 227 cancer-type–specific studies focusing on treatment in our sample (representing 530,503 patients) with the percentage of incident cases occurring in women for several major non-sex–specific cancer types. For 6 of 7 cancer types analyzed, the overall percentage of women among treatment study participants was significantly lower than the percentage expected based on general population incidence data (each P < .001).
Sex of first author was identified in 95.5% of the studies and was female in 30.4% of these. Sex of senior (last) author was identified in 96.6% and was female in 18.3% of these.
Among the prospective cancer research studies analyzed, 177 focused on sex-specific conditions. Among these, 126 focused exclusively on cancers that occur primarily in women (breast and gynecologic cancers), and 48 focused on cancers that occur exclusively among men (prostate, testicular, and penile cancers). Three studies focusing on sex-specific cancers considered both male and female sex-specific conditions. Studies of breast and gynecologic conditions were more likely to have a woman as first or senior author than other studies (59.2% vs 40.8%; P < .001). There was no statistically significant difference noted in the likelihood that studies of prostate, testicular, and penile cancers had a female first or senior author than other studies (31.3% vs 44.3%; P = .08).
Among the remaining 484 non-sex–specific prospective studies, 457 (representing 603,472 participants) reported the sex distribution of participants. The mean percentage female among these was 38.8%. Of the 457 studies, 304 focused on treatment, and these had a lower percentage female (36.9%) than other studies (42.2%) on univariate analysis (P = .004). Univariate analysis also revealed that studies with either first or senior female authors had a higher percentage of female participants than studies with first and senior authors who were both male (mean, 41.3% vs 37.3%; P = .01).
Government funding was reported in 44.0% of the non-sex–specific prospective studies. On univariate analysis, among the non-sex–specific studies, the percentage of female participants was significantly higher in studies reporting government funding than others (mean, 41.3% vs 36.9%; P = .005).
In a regression model including dummy variables for author sex, government funding, focus on treatment, and cancer type studied, government funding (P = .03), focus on treatment (P = .03), and cancer type studied (P < .001) were found to be independently significant correlates of the percentage of female participants. Government funding was found to be positively associated with percentage of female participants (standardized β = .10). Focus on treatment was found to be inversely associated with percentage of female participants (standardized β = −.10). Sex of first or senior author was no longer found to be independently significant (P = .15). This model explained a modest but statistically significant proportion of the variance in the percentage of female participants (adjusted R2 = 0.16; P < .001).
The results of the current study indicated that women are still under-represented as participants in recently published, high-impact, clinical studies of several non-sex–specific cancer types. The findings further revealed that cancer research studies receiving federal government funding include a significantly higher proportion of female subjects.
The study of participation in clinical trials is complicated by the issue that the definition of appropriate representation is open to debate, because some view adequacy in terms of proportionality (believing that the sex distribution for trial participation should mirror the sex distribution in incidence or prevalence of a specific cancer), some view adequacy in terms of minimum numbers necessary to conduct valid subgroup analyses (which in some cases exceeds the proportionality criterion and in others does not), and some view adequacy as parity (believing that the proportion of women in a study should mirror their representation among the general population).5, 6, 9, 18-20 Herein, we provided analyses comparing the sex distribution among participants in the studies we analyzed with the sex distribution among incident cases of cancers of various types. However, we caution the reader that assessing adequacy of representation is a more complex endeavor than can be captured by such an analysis alone.
The goal of this project was to extend the insights yielded by previous studies, not only by quantifying the representation of women in a broader range of cancer research studies, but also by exploring the associations between their representation and certain identifiable study characteristics. Our finding that federally funded studies have a higher proportion of female subjects suggests that NIH policies may indeed have some impact, albeit modest in magnitude, in encouraging greater parity in study enrollment. This finding complements previous studies that have evaluated the impact of federal policy by examining time trends in federally funded trial enrollment rather than by directly comparing federally funded studies to others.9 In addition to an effect related to the NIH's explicit policies regarding inclusion of women, our finding of a correlation between government funding and female subject inclusion may reflect a potentially more rigorous review process for studies receiving governmental funding, differences in grant size from government and private sources, or different priorities of researchers who seek government funding compared with industry-funded researchers, for example.
Prior studies have suggested several barriers that may discourage women from research participation, including lack of information, fear, and perceived interference with personal responsibilities, including child care.21 Sex differences in perceptions of risks and benefits of participation in research have been documented that may influence willingness to participate.22 Researchers concerned with protecting women of childbearing age from the potential hazards of participation may unwittingly deprive them and all of society of the critical benefits of studies including a diverse population of participants. Our findings suggest that under-representation of women continues to be a problem in modern clinical cancer research, and one that may be most pronounced in studies without governmental funding. Future research continues to be warranted to deepen our understanding of the mechanisms of this complex phenomenon so that appropriate interventions may be developed.
This study has certain limitations that merit discussion. First, it considered only selected, high-impact journals published in 1 year. Furthermore, by focusing on published research, this study did not provide a pure assessment of trial enrollment but rather an overview of the representation of female subjects in published studies. We believe that this provides a valuable perspective, complementary to that provided by studies focusing exclusively on trial enrollment. The study design was also limited to consideration of variables that were reported in the majority of studies, such as sex, and could not assess other important measures of subject diversity, including socioeconomic status and race. Another important point is that the unit of analysis in the current study was in some instances the published article, and in others, the individual research subject. We deliberately provide, wherever possible, both ways of summarizing the data, because each method has its own limitations. When the published article was the unit of analysis, each publication was weighted equally, whereas when the individual participant was the unit of analysis, the results were driven more by larger studies than smaller ones, and it is possible that a few extremely large studies may have been highly influential. Because assessing the adequacy of representation is a complex endeavor, as discussed earlier, we believe that both estimates are informative. Finally, although this study demonstrates associations between certain study characteristics and the sex distribution of participants, the effect sizes observed are modest, and the multivariate model developed herein leaves much of the variability in sex distribution from study to study unexplained.
In conclusion, the findings of the current study yield information regarding the representation of women as subjects of clinical cancer research being published in high-impact medical journals in the present day. Observed associations between government funding and greater subject diversity merit further exploration. Only by understanding the forces affecting the sex distribution of study subjects can we as a society succeed in ensuring that our medical research efforts are inclusive and to the benefit of all.
Conflict of Interest Disclosures
Supported by a grant from the University of Michigan Institute for Research on Women and Gender. The funding agency played no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the article.
- 1Institute of Medicine. Exploring the biological contributions to human health: does sex matter? Washington, DC: National Academy Press; 2001.
- 3National Institutes of Health. NIH Revitalization Act of 1993, PL 103-43. Subtitle B. Sections 131-133. June 10, 1993.
- 16Practical statistics for medical research. London: Chapman and Hall; 1991..