• Open Access

Publishing Genomic Studies: Walking the Fine Line


  • Arthur M. Feldman M.D., Ph.D.

    Corresponding author
    1. Temple University School of Medicine, Office of the Dean, 3500 N. Broad Street, Suite 1150 Philadelphia, PA 18140.
    Search for more papers by this author

AM Feldman (arthur.feldman@tuhs.temple.edu)

A little over a decade ago it cost my laboratory $6.00 to ascertain the presence of a single nucleotide polymorphism (SNP) at specific genetic locus in a sample of blood from an individual subject. We also needed to have the requisite skills in molecular biology. Today, that same molecular analysis can be performed for less than a penny and the technologic capabilities required to perform the experiments are minimal. Many techniques are now automated or available in kits with easy to follow instructions. No longer limited by cost or technologic know-how, our ability to carry out genomic or pharmacogenomics studies has exploded and the number of genomic manuscripts that are submitted each month to journals around the world has increased dramatically. Yet the technologic advances have not solved two important dilemmas that journal editors face. First, when is a study simply too small to have meaning and second, how do we interpret studies when we recognize that the field of bioinformatics and genetic statistics has not caught up with genomics? Two recent studies illustrate the conundrum that journals face.

In the December 28, 2011 issue of JAMA, Michael Holmes et al. published a systematic review and meta-analysis of studies that had assessed whether genetic variants in the cytochrome P450 subclass 2C19 (CYP2C19) would influence the pharmacologic effectiveness of the antithrombotic drug clopidogrel.1 Clopidogrel is an antiplatelet agent that is used by approximately 40 million patients worldwide to prevent atherothrombotic events; in particular, those that occur after percutaneous coronary revascularization.2 Clopidogrel therapy significantly reduced the rates of cardiovascular events when compared with placebo although it was also associated with an increase in major bleeding.

Clopidogrel is a prodrug that must be metabolized to its active moiety by CYP2C19. Genetic variants on at least one CYP2C19 allele reduce the function of the enzyme thereby reducing plasma exposure to the active metabolite of clopidogrel as much as 32%.3 Carriers of a reduced-function CYP2C19 allele were found to have diminished platelet inhibition and a higher rate of major adverse cardiovascular events including stent thrombosis, than did noncarriers when assessing outcomes in a cohort of 1,477 subjects enrolled in the Trial to Assess Improvement in Therapeutic Outcomes by Optimizing Platelet Inhibition with Prasugrel-thrombolysis in Myocardial Infarction trial who were treated with clopidogrel.3 These findings led clinicians to propose that individuals who were hyporesponders to clopidogrel might benefit from different dosing strategies or additional antiplatelet therapy.4 Indeed, on March 12, 2010, The US Food and Drug Administration (FDA) took the unusual step of giving clopidogrel a “boxed warning.” The FDA cautioned that slow metabolism of clopidogrel was associated with higher cardiovascular event rates. They suggested that genetic testing could identify individuals who were slow metabolizers and proposed that physicians implement “alternative treatment strategies” in that group of patients.5 This FDA warning was highly controversial as neither the American Heart Association nor the American College of Cardiology concurred with the recommendations of the FDA.

Holmes et al. retrieved 32 studies for their meta-analysis to assess the role of the CYP2C19 genotype in assessing outcomes in patients receiving clopidogrel. Six studies were randomized trials and the remaining 26 reported treatment-only analysis of individuals exposed to clopidogrel. In treatment-only analysis, subjects with one or more alleles associated with lower enzyme activity had lower levels of the active metabolite of clopidogrel, less platelet inhibition, and a lower risk of bleeding. They also had a higher risk of a cardiovascular event. However, when Holmes and his group restricted their analysis to studies with 200 or more events, they were not able to discern a relationship between CYP2C19 genotype and an effect of clopidogrel on cardiovascular end points or on the risk of bleeding—suggesting the presence of sample-size bias in the literature. The results of the meta-analysis led at least one individual to describe the FDA warning regarding clopidogrel use as “irrational exuberance.”5 However, the meta-analysis performed by Holmes et al. was not without concerns: the investigators used aggregate rather than participant-level data thereby limiting their power to detect differences, the components of the composite cardiovascular end points differed across studies; a substantial number of studies were small in size; only five studies of clopidogrel treatment response were nested within a randomized trial in which both treatment and control groups were genotyped; some studies enrolled patients with stable coronary artery disease whereas other studies enrolled only patients with acute coronary syndromes; and some studies genotyped multiple genetic variants in the CYP2C19 alleles whereas other studies genotyped only a single variant. Thus, although the Holmes meta-analysis points to the need for larger rather than smaller pharmacogenetic studies, it doesn't clarify the ongoing controversy regarding the role of genotyping in subjects being treated with clopidogrel. It does, interestingly enough, point out many of the flaws inherent in some meta-analyses.

Andrew Johnson et al. published a second seminal report that informs our view on pharmacogenetics and genomics in the March 2011 issue of Hypertension.6 Johnson et al. had previously conducted genome-wide association meta-analysis of systolic blood pressure, diastolic blood pressure, and hypertension in 29,136 individuals from six cohort studies in the Cohorts for Heart and Aging Research in Genomic Epidemiology Consortium (CHARGE). Their goal was to identify genetic variants associated with hypertension to eventually be able to ascertain whether those genetic variants were also associated with differential effects in response to treatment. Each of these six cohort studies, unlike the clopidogrel meta-analysis, provided patient-level data. In their March 2011 study Johnson et al. took the next logical step. They recognized that genome-wide association (GWAS) studies often miss important information because they actually provide too much data, present data from genes whose function is unknown, and that they cause investigators to pay a penalty for multiple testing. Therefore, they scanned their GWAS data from CHARGE (29,136 subjects) to identify presumptive functional SNPs in the genes that encoded 30 proteins that serve as drug targets for the treatment of hypertension including the targets of β-adrenergic receptor blockers, angiotensin converting enzyme inhibitors, α-adrenergic receptor blockers, angiotensin-receptor blockers, calcium-channel blockers, diuretics, and vasodilators and analyzed SNPs in those gene regions for association with blood pressure and hypertension. They looked for association of these SNPs with one or more blood pressure traits in the CHARGE population. Then they attempted to replicate the top meta-analysis SNPs for these genes in the Global BPgen Consortioum (n= 34,433) and the Women's Genome Health Study (n= 23,019).

All together, 85,5888 individuals contributed information to this meta-analysis study. They found that a functional variant in the β1-adrenergic receptor gene (ADRB1) in which a Glycine replaces an Arginine at position 389 (Arg389Gly) was associated with a lower mean systolic blood pressure, diastolic blood pressure, and prevalence of hypertension. They also found that a variation in the angiotensinogen gene (AGT) was associated with systolic blood pressure as well as diastolic blood pressure and hypertension. Thus two loci, ADRB1 and AGT, contain SNPs that reached a genome-wide significance threshold in meta-analysis for the first time. These SNPs will now be important targets for subsequent pharmacogenomics studies to identify whether there are drug–gene interactions that influence the efficacy of select antihypertensive agents. Importantly, this investigation in an enormous population of patients failed to confirm earlier studies suggesting a role for SNPs in other pharmacologic target proteins in the development of hypertension including the βl-adrenergic receptor and the angiotensin converting enzyme genes. On the other hand, this elegant study set a very high bar for genomic studies; over 85,000 subjects contributed data. However, the study gives one pause: how many investigators have access to DNA from 85,000 and the wherewithal to effectively analyze such a large dataset.

So how does a journal walk the fine line between irrational exuberance and undue caution? A purist would require that to be published, a genomics study must include data from a study that: has a large enough data set to preclude the bias that comes from performing multiple comparisons in small populations, provides confirming data from a second study in a population of subjects with the same phenotype, or includes meta-analysis of data sets from multiple large cohort studies with data being abstracted at the subject level. Indeed, the study published by Johnson et al. could be a template for all genomic studies. But if journals set the bar too high, both the journal and investigators might suffer. For example, we would eliminate the ability of investigators to publish their findings if they did not have access to the enormous data sets evaluated by Johnson et al. or access to the requisite bioinformatics and genetic statistical and information technology support for performing analysis on these huge data sets. Furthermore, stringent guidelines might cause investigators to avoid genetic studies in diseases that affect only a small population of patients and might also markedly limit the ability of investigators to identify new and interesting SNPs that have functional significances. When the bar is set so high or is too inflexible it might also become difficult for reviewers and editors to find a compromise solution. However, I would propose the following. First, we should try as far as possible to attract for publication as full papers those studies in which investigators have confirmed their study in a similarly sized and phenotypically similar population. Studies that use only a single population should be considered as “preliminary communications”—yet these studies might not reach the level of “acceptable for publication” if they are underpowered to meet their primary endpoints. Studies in a single population will be considered as a full publication if the authors identify the biologic relevance of a unique SNP or variant as well as defining the predictive power of the SNP. Finally, we will continue to assiduously review each paper on its own merits—judging papers on the quality of the data as well as on its importance and relevance to the field. As technology and the field of bioinformatics continue to advance, we will need to continually reassess these criteria with the goal of always publishing data that is meaningful and useful for the reader. CTS