SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

Eur J Clin Invest 2011; 41 (9): 1004–1009

Summary Points

  •  The rapid and continuing progress in gene discovery for complex diseases is fuelling interest in the potential application of genetic risk models for clinical and public health practice.
  •  The number of studies assessing the predictive ability is steadily increasing, but the quality and completeness of reporting vary.
  •  A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by prior reporting guidelines.
  •  These recommendations aim to enhance the transparency of study reporting and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct or analysis.
  •  A detailed Explanation and Elaboration document is published as an accompanying article [1].

Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

The recent successes of genome-wide association studies and the promises of whole-genome sequencing fuel interest in the translation of this new wave of basic genetic knowledge to healthcare practice. Knowledge about genetic risk factors may be used to target diagnostic, preventive and therapeutic interventions for complex disorders based on a person’s genetic risk or to complement existing risk models based on classical nongenetic factors such as the Framingham risk score for cardiovascular disease. Implementation of genetic risk prediction in health care requires a series of studies that encompass all phases of translational research [2,3], starting with a comprehensive evaluation of genetic risk prediction.

With increasing numbers of discovered genetic markers that can be used in future genetic risk prediction studies, it is crucial to enhance the quality of the reporting of these studies, as valid interpretation could be compromised by the lack of reporting of key information. Information that is often missing includes details in the description of how the study was designed and conducted (e.g. how genetic variants were selected and coded, how risk models or genetic risk scores were constructed and how risk categories were chosen) or how the results should be interpreted. An appropriate assessment of the study’s strengths and weaknesses is not possible without this information. There is ample evidence that prediction research often suffers from poor design and bias, and these may also have an impact on the results of the studies and on the models of disease outcomes based on these studies [4–6]. Although most prognostic studies published to date claim significant results [7,8], very few translate to clinically useful applications. Just as for observational epidemiological studies [9], poor reporting complicates the use of the specific study for research, clinical or public health purposes and hampers the synthesis of evidence across studies.

Reporting guidelines have been published for various research designs [10], and these contain many items that are also relevant to genetic risk prediction studies. In particular, the guidelines for genetic association studies (STREGA) have relevant items on the assessment of genetic variants, and the guidelines for observational studies (STROBE) have relevant items about the reporting of study design. The guidelines for diagnostic studies (STARD) and those for tumour marker prognostic studies (REMARK) include relevant items about test evaluation; the REMARK guidelines also have relevant items about risk prediction [6, 11–13]. However, none of these guidelines are fully suited to genetic risk prediction studies, an emerging field of investigation with specific methodological issues that need to be addressed, such as the handling of large numbers of genetic variants (from 10s to 10 000s) and flexibility in handling such large numbers in analyses. We organized a 2-day workshop with an international group of risk prediction researchers, epidemiologists, geneticists, methodologists, statisticians and journal editors to develop recommendations for the reporting of Genetic RIsk Prediction Studies (GRIPS).

Genetic risk prediction studies

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

Genetic risk prediction studies typically develop or validate models that predict the risk of disease, but they are also being investigated for use in predicting prognostic outcome, treatment response or treatment-related harms. Risk prediction models are statistical algorithms, which may be simple genetic risk scores (e.g. risk allele counts), may be based on regression analyses (e.g. weighted risk scores or predicted risks) or may be based on more complex analytic approaches such as support vector machine learning or classification trees. The risk models may be based on genetic variants only or include both genetic and nongenetic risk factors [14].

Aims and use of the GRIPS statement

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

The 25 items of the GRIPS statement are intended to maximize the transparency, quality and completeness of reporting on research methodology and findings in a particular study. It is important to emphasize that these recommendations are guidelines only for how to report research and do not prescribe how to perform genetic risk prediction studies. The guidelines do not support or oppose the choice of any particular study design or method, e.g. the guidelines recommend that the study population should be described, but do not specify which population is preferred in a particular study.

The intended audience for the reporting guidelines is broad and includes epidemiologists, geneticists, statisticians, clinician scientists and laboratory-based investigators who undertake genetic risk prediction studies, as well as journal editors and reviewers who have to appraise the design, conduct and analysis of such studies. In addition, it includes ‘users’ of such studies who wish to understand the basic premise, design and limitations of genetic prediction studies to interpret the results for their potential application in health care. These guidelines are also intended to ensure that essential data from future genetic risk prediction studies are presented in standardized form, which will facilitate information synthesis as part of systematic reviews and meta-analyses.

Items presented in the checklist are relevant for a wide array of risk prediction studies, because GRIPS focuses on the main aspects of the design and analysis of risk prediction studies. GRIPS does not address randomized trials that may be performed to test risk models, nor does it specifically address decision analyses, cost-effectiveness analyses, assessment of healthcare needs or assessment of barriers to healthcare implementation [15]. Once the performance of a risk model has been established, these next steps toward implementation require further evaluation [11,16]. For the reporting of these studies, which go beyond the assessment of genetic risk models as such, additional requirements apply. However, proper documentation of genetic predictive research according to GRIPS might facilitate the translation of research findings into clinical and public health practice.

Development of the GRIPS statement

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

The GRIPS statement was developed by a multidisciplinary panel of 25 risk prediction researchers, epidemiologists, geneticists, methodologists, statisticians and journal editors, seven of whom were also part of the STREGA initiative [12]. They attended a 2-day meeting in Atlanta, Georgia (US), in December 2009 that was sponsored by the US Centers for Disease Control and Prevention on behalf of the Human Genome Epidemiology Network (HuGENet) [17]. Participants discussed a draft version of the guidelines that was prepared and distributed before the meeting. This draft version was developed on the basis of existing reporting guidelines, namely STREGA [12], REMARK [6], and STARD [13]. These were selected out of all available guidelines (see http://www.equator-network.org) because of their focus on observational study designs and genetic factors (STREGA), prediction models (REMARK) and test evaluation (REMARK and STARD). During the meeting, methodological issues pertinent to risk prediction studies were addressed in presentations. Workshop participants were asked to change, combine or delete proposed items and add additional items if necessary. Participants had extensive postmeeting electronic correspondence. To harmonize our recommendations for genetic risk prediction studies with previous guidelines, we chose the same wording for the items wherever possible. Finally, we tried to create consistency with previous guidelines for the evaluation of risk prediction studies of cardiovascular diseases and cancer [3,18]. The final version of the checklist is presented in Table 1.

Table 1.   Reporting recommendations for evaluations of risk prediction models that include genetic variants
Title and abstract
  1. *Marked items should be reported for every population in the study.

 1(a) Identify the article as a study of risk prediction using genetic factors. (b) Use recommended keywords in the abstract: genetic or genomic, risk, prediction
Introduction
 Background and rationale2Explain the scientific background and rationale for the prediction study
 Objectives3Specify the study objectives and state the specific model(s) that is/are investigated. State if the study concerns the development of the model(s), a validation effort or both
Methods
 Study design and setting4*Specify the key elements of the study design and describe the setting, locations and relevant dates, including periods of recruitment, follow-up and data collection
 Participants5*Describe eligibility criteria for participants, and sources and methods of selection of participants
 Variables: definition6*Clearly define all participant characteristics, risk factors and outcomes. Clearly define genetic variants using a widely used nomenclature system
 Variables: assessment7*(a) Describe sources of data and details of methods of assessment (measurement) for each variable. (b) Give a detailed description of genotyping and other laboratory methods
 Variables: coding8(a) Describe how genetic variants were handled in the analyses. (b) Explain how other quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why
 Analysis: risk model construction9Specify the procedure and data used for the derivation of the risk model. Specify which candidate variables were initially examined or considered for inclusion in models. Include details of any variable selection procedures and other model-building issues. Specify the horizon of risk prediction (e.g. 5-year risk)
 Analysis: validation10Specify the procedure and data used for the validation of the risk model
 Analysis: missing data11Specify how missing data were handled
 Analysis: statistical methods12Specify all measures used for the evaluation of the risk model including, but not limited to, measures of model fit and predictive ability
 Analysis: other13Describe all subgroups, interactions and exploratory analyses that were examined
Results
 Participants14*Report the numbers of individuals at each stage of the study. Give reasons for nonparticipation at each stage. Report the number of participants not genotyped, and reasons why they were not genotyped
 Descriptives: population15*Report demographic and clinical characteristics of the study population, including risk factors used in the risk modelling
 Descriptives: model estimates16Report unadjusted associations between the variables in the risk model(s) and the outcome. Report adjusted estimates and their precision from the full risk model(s) for each variable
 Risk distributions17*Report distributions of predicted risks and/or risk scores
 Assessment18Report measures of model fit and predictive ability, and any other performance measures, if pertinent
 Validation19Report any validation of the risk model(s)
 Other analyses20Present results of any subgroup, interaction or exploratory analyses, whenever pertinent
Discussion
 Limitations21Discuss limitations and assumptions of the study, particularly those concerning study design, selection of participants, measurements and analyses, and discuss their impact on the results of the study
 Interpretation22Give an overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies and other relevant evidence
 Generalizability23Discuss the generalizability and, if pertinent, the healthcare relevance of the study results
Other
 Supplementary information24State whether databases for the analyzed data, risk models and/or protocols are or will become publicly available, and if so, how they can be accessed
 Funding25Give the source of funding and the role of the funders for the present study. State whether there are any conflicts of interest

The GRIPS explanation and elaboration article

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

Accompanying this GRIPS statement, an Explanation and Elaboration document has been written [1], modelled after those developed for other reporting guidelines [19–22]. The Explanation and Elaboration document illustrates each item with at least one published example that we consider transparent in reporting, explains the rationale for its inclusion in the checklist and presents the details of the items that need to be addressed to ensure transparent reporting. The Explanation and Elaboration document was produced after the meeting. The document was prepared by a small subgroup and shared with all workshop participants for additional revisions and final approval.

Concluding remarks and future directions

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

High-quality reporting reveals the strengths and weaknesses of empirical studies, facilitates the interpretation of the scientific and healthcare relevance of the results – especially within the framework of systematic reviews and meta-analyses – and helps build a solid evidence base for moving genomic discoveries into applications in healthcare practice. The GRIPS guidelines were developed to improve the transparency, quality and completeness of the reporting of genetic risk prediction studies. As outlined in the introduction, GRIPS does not prescribe how studies should be designed, conducted or analysed, and therefore, the guidelines should not be used to assess the quality of empirical studies [23]. The guidelines should be used only to check whether all essential items are adequately reported.

Finally, the methodology for designing and assessing genetic risk prediction models is still developing. For example, newer measures of reclassification were first introduced in 2007 [24], and several alternative reclassification measures have been proposed [25]. Which measures to apply and when to use measures of reclassification are still subject to ongoing evaluation and discussion [26]. Furthermore, alternative strategies for constructing risk models other than simple regression analyses are being explored, and these may add increased complexity to the reporting. In formulating the items of the GRIPS statement, these methodological advances were anticipated. It is for this reason that the GRIPS statement recommends how a study should be reported and not how a study should be conducted or analysed. Therefore, methodological and analytical developments will not immediately impact the validity and relevance of the items, but the GRIPS statement will be updated when this is warranted by essential new developments in the construction and evaluation of genetic risk models.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

The members of the GRIPS group are A. Cecile J. W. Janssens, John P. A. Ioannidis, Sara Bedrosian, Paolo Boffetta, Siobhan M. Dolan, Nicole Dowling, Isabel Fortier, Andrew N. Freedman, Jeremy M. Grimshaw, Jeffrey Gulcher, Marta Gwinn, Mark A. Hlatky, Holly Janes, Peter Kraft, Stephanie Melillo, Christopher J. O’Donnell, Michael J. Pencina, David Ransohoff, Sheri D. Schully, Daniela Seminara, Deborah M. Winn, Caroline F. Wright, Cornelia M. van Duijn, Julian Little and Muin J. Khoury.

Funding

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

Workshop was sponsored by the Centers for Disease Control and Prevention on behalf of the Human Genome Epidemiology Network (HuGENet). The findings and conclusions in this report are those of the authors and do not necessarily reflect the views of the Department of Health and Human Services. A. Cecile J.W. Janssens is financially supported by grants from the Erasmus University Medical Center Rotterdam, the Center for Medical Systems Biology in the framework of the Netherlands Genomics Initiative (NGI) and the VIDI grant of the Netherlands Organisation for Scientific Research (NWO). John P.A. Ioannidis: Tufts CTSI is supported by the National Institutes of Health/National Center for Research Resources (UL1 RR025752). Opinions in this paper are those of the authors and do not necessarily represent the official position or policies of the Tufts CTSI. Julian Little holds a Canada Research Chair in Human Genome Epidemiology. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Provenance

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

Not commissioned; externally peer reviewed. In order to encourage dissemination of the GRIPS Statement, this article will also be published by Annals of Internal Medicine, BMJ, Circulation: Cardiovascular Genetics, European Journal of Clinical Investigation, European Journal of Epidemiology, European Journal of Human Genetics, Genetics in Medicine, Genome Medicine and Journal of Clinical Epidemiology.

Address

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References

Department of Epidemiology, Erasmus University Medical Center, Rotterdam, the Netherlands (A. C. J. W. Janssens, C. M. van Duijn); Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece (J. P. A. Ioannidis); Biomedical Research Institute, Foundation for Research and Technology, Ioannina, Greece (J. P. A. Ioannidis); Department of Medicine, Tufts University School of Medicine, Boston, MA, USA (J. P. A. Ioannidis); Center for Genetic Epidemiology and Modeling and Tufts CTSI, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA, USA (J. P. A. Ioannidis); Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, CA, USA (J. P. A. Ioannidis); Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, ON, Canada (J. Little); Office of Public Health Genomics, Centers for Disease Control and Prevention, Atlanta, GA, USA (M. J. Khoury).

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Genetic risk prediction studies
  5. Aims and use of the GRIPS statement
  6. Development of the GRIPS statement
  7. The GRIPS explanation and elaboration article
  8. Concluding remarks and future directions
  9. Acknowledgements
  10. Funding
  11. Provenance
  12. Address
  13. References