Industry sponsorship and research outcome

  • Protocol
  • Methodology

Authors


Abstract

This is the protocol for a review and there is no abstract. The objectives are as follows:

The objectives are to investigate whether:

  • funding of drug studies by the pharmaceutical industry is associated with outcomes that are favorable to the funder;

  • drug studies funded by the pharmaceutical industry differ in their risk of bias compared with studies with other sources of support;

  • funding of device studies by the device industry is associated with outcomes that are favorable to the funder; and

  • device studies funded by the device industry differ in their risk of bias compared with studies with other sources of support.

Background

Description of the problem or issue

Clinical research sponsored by the pharmaceutical industry affects how doctors practice medicine (PhRMA 2009; Wyatt 1991). An increasing number of clinical trials at all stages in a product's life cycle are funded by the pharmaceutical industry, probably reflecting the fact that this industry now spends more on medical research than the National Institutes of Health in the United States (Anderson 1991; Dorman 1999; Karlberg 2008). Most pharmacoeconomic studies are either done in-house by the drug companies or externally by consultants who are paid for by the company (Anis 2000; Hill 2000).

Numerous studies and previous systematic reviews have documented that pharmaceutical industry sponsorship of drug studies is associated with findings that are favorable towards the sponsor’s product (Bekelman 2003; Lexchin 2003). There are several ways that industry can sponsor a study, including shared sponsorship, single-source sponsorship and provision of free research product only. There are also several potential ways that industry sponsors can influence the results of a study, including how the question is framed, the design of the study, the conduct of the study, the population studied and the selective publication of the results (Bero 1996). Recent releases of internal pharmaceutical industry documents have revealed examples of pharmaceutical industry manipulation of the conduct and publication of studies (Steiman 2006). Although some journals now require that the role of the sponsor in the design, conduct and publication of the study be described, this practice is not widespread.

Description of the methods being investigated

This review examines the association of industry sponsorship of drug or device studies with publication of outcomes that favor the sponsor. Results that are unfavorable to the sponsor (that is, studies that find a drug is less clinically effective or less safe than other drugs used to treat the same condition) can pose considerable financial risks to companies. 

How these methods might work

Pressure to show that the drug causes a favorable outcome may result in biases in design, outcome and reporting of industry-sponsored research (Bekelman 2003; Bero 1996; Sismondo 2008). For example, use of inappropriate comparators can influence the results of a study, as can a post hoc change in stopping rules or other deviations from the protocol.

Why it is important to do this review

This systematic review is the update of an original meta-analysis by two of the authors (Lexchin 2003) and it investigates whether sponsorship by the pharmaceutical industry is associated with the publication of outcomes favorable to the funder. This review is now out of date and a number of newer studies can potentially be included. Recent developments, such as the adoption of trial registration, could lessen the bias associated with industry sponsorship as publication bias can be more readily detected. On the other hand, the release of internal industry documents as a result of settlement agreements resulting from litigation against drug companies has revealed more examples of industry sponsorship being associated with publication of favorable results. In addition, the scope of the review is now expanded to include device studies, as they are subject to the same biases as drug studies and are often sponsored by companies with a financial interest in the outcome.  

Objectives

The objectives are to investigate whether:

  • funding of drug studies by the pharmaceutical industry is associated with outcomes that are favorable to the funder;

  • drug studies funded by the pharmaceutical industry differ in their risk of bias compared with studies with other sources of support;

  • funding of device studies by the device industry is associated with outcomes that are favorable to the funder; and

  • device studies funded by the device industry differ in their risk of bias compared with studies with other sources of support.

Methods

Criteria for considering studies for this review

Types of studies

Cross-sectional studies, cohort studies, systematic reviews or meta-analyses that quantitatively compare primary research of drug studies or medical devices studies funded by the pharmaceutical or device industry with studies that have other sources of funding. Drugs are defined as medications that require approval by a regulatory authority as a prescription drug, recognizing that these approval standards vary worldwide.

We will include studies in all languages.

We will exclude studies of the effects of funding by non-pharmaceutical or non-device industries (e.g. tobacco, food or chemical) and studies without quantitative data. We will exclude studies of clinical trials evaluating the effectiveness of herbal supplements or medical procedures. We will not include analyses of pharmacokinetic studies. 

Only studies published in full will be included; we will exclude letters to the editor and published conference presentations.

We will exclude papers that quantitatively compare the association of funding and results of secondary research studies (i.e. systematic reviews or meta-analyses) of drugs or devices.

Types of data

Drug and device studies will include human research studies comparing drug to placebo, device to sham, drug to drug, drug to device, device to device, or multi-arm comparisons where the effectiveness, efficacy or safety of the drug or device are being evaluated.

Types of methods

Drug or device studies with pharmaceutical or device industry funding versus drug or device studies with other or undisclosed funding. We will extract the definition of industry funding verbatim from the included study (see Data extraction and management) and reported this in the 'Characteristics of included studies' table. For analysis, we will group the definitions into a variety of categories, including 100% pharmaceutical company/device company funded; 100% government funded; 100% non-profit funded; mixed funding (e.g. non-profit, government and industry collaboration); provision of drug or device only; and undisclosed funding.

Types of outcome measures

Primary outcomes

We will include two primary outcomes:

  1. Whether the results were favorable to the sponsor.

  2. Whether the conclusions were favorable to the sponsor. 

Results considered favorable to the sponsors are those that are statistically significant, e.g. P < 0.05 or 95% confidence interval excluding the possibility of no difference, in favor of the sponsor product. We will combine results that are unfavorable to the sponsor product, neutral and not statistically significant. In the past review (Lexchin 2003), there was an extremely limited number of studies that reported results unfavorable to the sponsor, and as such they were combined with neutral. If we find many unfavorable results in the future, we will consider analyzing three groups and not dichotomizing. However, we anticipate that this will be unlikely. 

Conclusions considered favorable to the sponsor are those where the sponsor’s product is preferred over the control treatment.

Secondary outcomes

We will include three secondary outcomes:

  1. The size of the effect estimate reported in industry-funded studies versus those with other sources of funding.

  2. The methodological characteristics aimed at reducing bias in industry-funded studies versus those with other sources of funding.

  3. If available the concordance between study results and conclusions. 

We will include studies that report one of these secondary outcomes, but not the primary outcomes.

Search methods for identification of studies

Electronic searches

We will search the Cochrane Methodology Register, MEDLINE, PREMEDLINE and EMBASE databases. We will search the Web of Science database for papers that cite any of the studies included in our review.

Search strategy

We will use the strategy shown in Appendix 1 for OVID MEDLINE and adapt it for the other databases.

Searching other resources

Other sources of data will include author files and searches of reference lists in included studies and previous systematic reviews.

Data collection and analysis

Selection of studies

Two observers (AL and OB) will screen the titles and abstracts, when available, of all retrieved records for obvious exclusions, and will assess studies based on full text. Potentially eligible studies will be sent to the other authors for final validation of the inclusion criteria. Any disagreements will be resolved by consensus and we will describe reasons for exclusions of potentially eligible studies in the final report in the 'Characteristics of excluded studies' table. We will translate non-English studies when feasible.

Data extraction and management

Two observers (AL and SS) will independently extract data from eligible studies. Any difference in data extraction will be resolved by consensus. If necessary, we will contact investigators for information or data required for the review not published in the original reports. 

We will extract data on the following.

  • Design of study: cohort, cross-sectional study, systematic review or meta-analysis, or other.

  • Study domain - descriptive: disease area or drug class.

  • Study domain - category (drug class, specific disease, medical speciality/type of diseases, mixed).

  • Type of studies (drug, device, drug and device, mixed).

  • Type of comparisons (drug versus drug, drug versus placebo, drug multi-arm, device versus device, device versus placebo, device versus drug, device multi-arm, mixed, other).

  • Sample strategy used to locate research (electronic search only, electronic plus other, sampling of journals, sampling by venue (e.g. conference)).

  • Whether there were language restrictions on the search.

  • Country of corresponding author.

  • Time period covered by studies in review.

  • Funding categories used in paper. Potential categories are:

    • 100% pharmaceutical company/device company funded;

    • 100% government funded;

    • 100% non-profit funded;

    • mixed funding - e.g. non-profit, government and industry collaboration;

    • provision of drug or device only; and

    • undisclosed funding.

  • Funding categories used in analysis.

  • Financial ties.

  • Description of role of the funder (if any). For example, definition of the funder’s role in the design, implementation or reporting in the sample of studies.

  • Number of separate studies funded by industry included in the included study.

  • Number of separate studies not funded by industry included in the included study.

  • Criteria used to assess methodological quality of the research.

  • Primary purpose of the study.

  • Did the review assess comparators used in studies? Active substance, placebo, no therapy?

  • Did the review comment on appropriateness of comparators?

  • Did the review look at whether conclusions were consistent with results?

  • Results: statistical significance, direction of result, magnitude of effect.

  • Authors’ conclusions about association of drug industry funding and outcome.

Assessment of risk of bias in included studies

We will review studies for risk of bias to determine:

  • if explicit criteria were used to select studies for inclusion/exclusion;

  • whether the review states that two or more investigators reviewed studies for selection;

  • whether the search for studies was comprehensive; and

  • whether the study controlled for methodological differences and for other characteristics that reduce bias.

Measures of the effect of the methods

We will carry out a meta-analysis of the studies that reported the association of funding source with favorable study outcome in cases where a pooled risk ratio and its 95% confidence interval can be computed.

Dealing with missing data

We will contact authors of the original papers in an attempt to obtain missing data. If papers include studies with author conflicts of interest, but without industry funding in their category of industry funding, we will contact the authors in order to obtain separate data for studies with industry funding.

Assessment of heterogeneity

We will assess heterogeneity of the reviews statistically using a Chi2 test with a P value of 0.10.

Data synthesis

We plan to construct a pooled risk ratio. We will attempt to use a Mantel-Haenszel statistic to create a pooled risk ratio, however if significant heterogeneity is observed, we will use a DerSimonian-Laird random-effects model. In an effort to reduce heterogeneity, we will include analyses of device studies as a separate subgroup analysis. 

Subgroup analysis and investigation of heterogeneity

The primary analysis will compare the number of favorable results and conclusions in studies with industry funding to those without industry funding; 'industry funding' will include 100% pharmaceutical company/device company funding, mixed funding and provision of drug or device only. 'Non-industry funding' will be defined as 100% government funding, 100% non-profit funding, other and undisclosed funding. We will separately analyze papers of head-to-head studies comparing results and conclusions in company-funded studies with other company-funded studies, based on whether the sponsor produces the test or comparator drug.

A sensitivity analysis (see below) will exclude those with mixed funding sources and those with funding consisting solely of free product from the 'industry funding' category to determine if this has an impact on the initial analysis. As noted under 'Data extraction and management,' we are dependent on how the studies in our review define 'funding' and this will be described in detail for each study. We will also conduct a sensitivity analysis excluding studies with undisclosed funding from the category of 'non-industry funding' to determine if classifying them as having pharmaceutical industry funding would change our initial analysis. In addition, we will discuss the possible bias introduced by non-disclosure of funding in our discussion.  

We will consider the following factors as potential explanations for heterogeneity in a meta-regression or using the test for subgroup differences in RevMan 5.1 (RevMan 2011), if there are sufficient data:    

  1. We hypothesize that the association of industry funding and favorable outcomes may be larger for lower quality studies. We will assess quality of the included studies using the criteria described in Assessment of risk of bias in included studies. We will regard studies with adequate study inclusion, a comprehensive search and those that controlled for bias as having a low risk of bias; others as having a high risk. We will analyze studies with a low risk of bias separately. This analysis will pool adjusted odds ratios using the generic inverse variance method. 

  2. We also hypothesize that the association of industry funding and favorable outcomes may be sensitive to how 'industry funding' and other types of funding have been defined, and we will analyze these in a sensitivity analysis as described above.   

  3. We will analyze studies of devices as a subgroup, as they differ from drug studies in design.

  4. Additionally, as the study domain may contribute to heterogeneity, we will analyze studies on specific drugs or diseases in separate subgroup analyses and studies of mixed types in another analysis.

Sensitivity analysis

See above.

Acknowledgements

We thank Bryan Sandlund for help with developing the protocol and Sarah Chapman (Trial Search Coordinator) at the Methodology Review Group for developing the search strategy.

Appendices

Appendix 1. Search strategy

1. Drug Industry/

2. ((drug$ or pharmaceutical$ or device$ or for-profit) adj (industr$ or company or companies or manufacturer$ or organisation$ or organization$ or agency or agencies)).ti,ab. 

3. private industr$.ti,ab. 

4. (industr$ or nonindustr$ or non-industr$).ti,ab. 

5. 1 or 2 or 3 or 4 

6. Conflict of interest/ 

7. Financial support/ 

8. Research support as topic/ 

9. (funded or funding or sponsor$ or support$ or financ$ or involvement).ti,ab. 

10. "competing interest$".ti,ab. 

11. or/6-10 

12. 5 and 11 

13. Publication bias/ 

14. "Bias (Epidemiology)"/ 

15. bias$.ti,ab. 

16. or/13-15 

17. 12 and 16 

18. Treatment outcome/ 

19. "Outcome Assessment (Health Care)"/ 

20. (outcome$ or findings).ti,ab. 

21. or/18-20 

22. (favor$ or favour$ or positive or significan$ or beneficial or benefit$ or effective or effectual or efficacious).ti,ab. 

23. (insignifican$ or nonsignifican$ or negative or adverse or ineffectiv$ or ineffectual or unfavorabl$ or unfavourabl$).ti,ab. 

24. 22 or 23 

25. 21 and 24 

26. 12 and 25 

27. ((favor$ or favour$ or positive or significan$ or insignifican$ or nonsignifican$ or negative or unfavorabl$ or unfavourabl$) adj result$).ti,ab. 

28. 12 and 27 

29. 17 or 26 or 28

History

Protocol first published: Issue 9, 2011

Contributions of authors

Development of protocol (AL, JL, LB and SS), design of the search strategy (AL); initial screening of articles (AL and OAB); final selection of studies (all authors); data extraction (AL and SS); data analysis and interpretation of results (all authors); wring of manuscript (all authors).

Declarations of interest

Lisa Bero, Joel Lexchin and Sergio Sismondo are authors of some of the studies that will be included.

In 2007, Joel Lexchin was retained by a law firm representing Apotex to provide expert testimony about the effects of promotion on the sales of medications. From 2007 to 2008 he was retained as an expert witness by the Canadian federal government in its defense of a lawsuit challenging the ban on direct-to-consumer advertising of prescription drugs in Canada. In 2010 he was a consultant to a law firm acting for the family of a patient who died from an alleged side effect of a drug made by Allergan. He is also on the management group of Healthy Skepticism Inc.

The authors have no other relevant interests. 

Sources of support

Internal sources

  • The Nordic Cochrane Centre, Copenhagen, Denmark.

    The authors are personally salaried by their institutions during the period of the review.

  • York University, Toronto, Canada.

    The authors are personally salaried by their institutions during the period of the review.

  • Queen's University, Kingston, Canada.

    The authors are personally salaried by their institutions during the period of the review.

  • University of California, San Francisco, USA.

    The authors are personally salaried by their institutions during the period of the review.

  • University Health Network, Toronto, Canada.

    The authors are personally salaried by their institutions during the period of the review.

External sources

  • Canadian Institutes of Health Research, Canada.

    The work of Octavian A. Busuioc is supported by a grant from the Canadian Institutes of Health Research (#106892).

Ancillary