SEARCH

SEARCH BY CITATION

Introduction

  1. Top of page
  2. Introduction
  3. Funding
  4. Acknowledgment
  5. References

One of most important mandates of the National Institutes of Health's (NIH) Clinical and Translational Science Awards (CTSA) is to accelerate the spread of scientific discoveries beyond academic health centers so that the public benefits from health-related research.[1, 2] The University of Michigan's Michigan Institute for Clinical and Health Research (MICHR) exists to carry out the CSTA's mandate. The overarching mission of the MICHR Community Engagement and Research Core (CERC) is to foster equitable and bidirectional research partnerships with those who will use scientific discoveries to benefit health. The CERC increases community decision-making and action for health promotion, disease prevention, and treatment by involving the community in all phases of clinical research planning, design, implementation, evaluation and dissemination.[3, 4]

Clinical researchers go through many years of training in order to be able to design and conduct rigorous clinical studies. However, the ultimate value of such studies is determined by their dissemination and application in communities to prevent and treat illness. Communities often include people with many varying cultures. Many of which are not part of the dominant culture such as underserved and economically disadvantaged groups, ethnic and racial minority groups, and marginalized groups (e.g., lesbian, gay, and transgendered groups). Such communities may have valuable, but often-muted insights about causes of health problems and viable solutions to those problems. To have significant impact the results of clinical research must pass muster at both the scientific and community level. It is hoped that over time that the interaction of the academic/science culture with community cultures will result in a shared perspective across cultures that will enhance the translation of health-related research into improved health.

MICHR's Scientific Review Committee (SRC) reviews multiple pilot applications as part of the overall Pilot Grant Program (PGP), including the Community University Research Partnership (CURES) pilot applications. When SRC reviews are available, they are shared with the CECC but only the CECC's preliminary scores have been submitted. CECC review criteria are distinct from the SRC, which structures review based on the standard NIH criteria. Instead, CECC reviewers assess applications for the quality of the community-academic partnership, level of community participation, equitable distribution of budget between community and academic investigators, and relevance of the research topic to the community, as well as a project's potential for securing external funds for a larger study. While proposals with poor SRC scores are typically not considered for funding, final award decisions are based on final CECC review scores.

Although many CECC members were comfortable reviewing the pilot study applications a subset of our community partners felt uneasy reviewing university/community partner research projects. They felt that they did not possess the expertise or experience to judge the merits of scientific research grant applications. On occasion these members felt uncomfortable assigning a proposal a score that was significantly at odds with the score given by the SRC. This was unfortunate because the community perspective typically adds essential insights that will ultimately drive the dissemination and sustainability of a research study's benefit to human health.

These guidelines were developed in partnership with our CECC during the process of reviewing community-engaged research proposals. They are intended to help community partners bring to bear their unique expertise and experience in the evaluation of proposed research studies that include community partners as co-principal investigators. These guidelines offer an approach for ensuring that the community's voice is heard when pilot project proposals for community-engaged research are developed and reviewed. A discussion of these guidelines could be used to orient new members of a community council. They could also be adapted to better serve specific needs of various community engagement councils.

CECC review process

After thinking about and discussing the discomfort expressed by some of the community members on our CECC about reviewing scientific research proposals, we realized the discomfort was caused (or at least exacerbated) by the fact that the academic partners, had not provided specific guidelines and/or support to CECC community members regarding their role in reviewing community-based pilot study applications. After considering the issue we realized that when evaluating pilot studies the expertise possessed by our community partners relates primarily to the feasibility of conducting the study in their communities rather than the scientific merit of the proposal. Bringing this expertise to bear on pilot study reviews is essential if these pilot studies are to succeed in contributing to the health of communities locally and nationally.

This led us to develop a draft set of community guidelines (Table 1) organized under the headings used in the guidelines NIH provides to members of its study sections. The guidelines were rewritten to match the unique experience and expertise community partners bring to the review process. The guidelines were reviewed and revised based on input from the entire CECC, as well as by three groups of community partners, that is, from the CECC's three focal communities and then again by the entire CECC. The guidelines in Table 1 have incorporated to the extent possible the suggestions and advice of the community members who reviewed them. The answers to the questions in Table 1 were designed to empower our community partners to make more informed judgments when reviewing- and scoring-engaged community research grant applications.

Table 1. Community research review guidelines
1. Significance: does this study address a problem or barrier to progress that is viewed as important by the majority of people in the community, especially the people that the study is designed to include. Will this project have lasting impact in the community? Will it be sustainable in the community when the study ends? If so what will be the long-term impact on the community? How does the project improve conditions for community members at both the individual and community level? If successful will the results be viewed as making an important contribution to the health and well-being of community members? Does the study address health disparities experienced by vulnerable groups, for example, members of medically underserved racial and ethnic groups, children, the elderly, and indigent community members. Will the new knowledge obtained from this study enhance the economic, environmental, social, and cultural conditions in the community? Will the study result in improved “best practices” by community health care providers?
2. Community-based co-principal investigators: are the co-principal investigators (CPI) collaborators, and other researchers well suited to the project? Do the CPI's have complementary and integrated expertise? Do respective community stakeholders trust the community-based co-principal investigator to behave professionally and ethically in the conduct of this project? Is the community-based co-principal investigator known, respected and trusted in the community? Does the community co-principal investigator have a record of meaningful accomplishments in the community? Do the co-principal investigators have a history of working together? Does the community-based co-principal investigator have expertise and experience necessary to carry out his/her role in the proposed study?
3. Partner organizations: are both the investigator's organizations (e.g. university or hospital) and the community organization trusted and respected in this community? Does the co-principal investigator's university have a history of making positive contributions to the community? Does the community organization have a history of providing high quality and needed services? Does the community organization have a history of previous successful collaborations with universities or hospitals? Is the community-based co-principal investigator's organization respected and trusted in the community? Will the resources provided to conduct the study be distributed equitably between the community and the university?
4. Originality: does the study appear to propose a new and creative approach to community-based research? How is it new or different? Are you familiar with similar studies conducted in this or other communities previously? How does the study improve and build upon previous research? If successful will this study lead to improved health or health care in the community?
5. Approach: given what you know about this community and community members will the study be perceived as important? From a community perspective does the project seem realistic? Is the budget realistic considering the work that is proposed? Will the study be able to recruit participants? Does the study offer benefits to those who participate that justify the risks or difficulties involved in being in the study? Does the project benefit the community directly, for example, hiring and training local people to work in the study, for example, data collector, community liaison, project manager? If you were eligible for this study would you participate in it? Have the CPIs identified potential community barriers to the successful completion of the study and do they have plans for overcoming them? Are you satisfied that the study contain plans for the protection of human subjects from research risks? Does the study contain a viable plan for recruiting participants? Does the application contain a convincing statement of past and planned community engagement?
6. Environment: will the study take place in a particular location that is perceived to be safe and accessible? Is transportation or reimbursement for transportation provided for the participants in the study? Is public transportation readily available? Is the location one with which research participants will be familiar and trust? Will local weather conditions help or hinder the study? Is the location of the study considered safe?

CURES pilot study applications are submitted through the pilot grant program's online application submission system and are reviewed by MICHR's Scientific Review Committee (SRC). They are also sent to the CERC for review by the CECC. CERC program staff members work with CECC members to identify one lead reviewer and one secondary reviewer for each proposal who are responsible for presenting a summary score and critique at the pilot study review meeting. Prior to the review meeting all CECC members submit preliminary scores and comments on key strengths and weaknesses for each proposal. CECC uses the NIH's 9-point scoring scale to assess proposals for the criteria listed in Table 2.[5]

Table 2. NIH scoring system[5]
ImpactScoreDescriptorAdditional guidance on strengths/weaknesses
High1ExceptionalExceptionally strong with essentially no weaknesses
 2OutstandingExtremely strong with negligible weaknesses
 3ExcellentVery strong with only some minor weaknesses
Medium4Very GoodStrong but with numerous weaknesses
 5GoodStrong but with at least one moderate weakness
 6SatisfactorySome strengths but also some moderate weaknesses
Low7FairSome strengths but with at least one major weakness
 8MarginalA few strengths and a few major weaknesses
 9PoorVery few strengths and numerous major weaknesses

All CECC members receive compiled scores prior to the pilot study review meeting. They also receive the scores and critiques from the SRC prior to (depending on the timing of the two review processes) the CECC review meeting. The final CECC scores are averaged together (including the range) and the SRC scores are also averaged together (including the range) for each submission.

After preliminary scoring, the CECC uses a review meeting format similar to NIH study sections, where the primary and secondary reviewers present their critiques. If an agreement between SRC and CECC scores is reached then the application is either awarded or not awarded. If there is a disagreement between SRC and CECC scores, the CECC takes into consideration if there are any moderate to significant scientific weaknesses in the application. However, CECC scores are used to make the final funding decision. CECC and SRC scores and comments are shared back with all applicants. Applicants may then consult with CERC staff on addressing reviewer critiques for potential resubmission.

These guidelines can be used to increase the expertise and effectiveness of community reviewers, who have a unique (i.e., community) perspective. They allow community partners not trained in the scientific method to have a viable and valued voice in the academic paradigm of scientific review. Informed community input is essential if engaged community research is to be successful.

A final note, an appendix explaining common research designs and a glossary of terms used by the scientific community is included with this article. These materials have been extensively revised to improve their clarity and accessibility.

Funding

  1. Top of page
  2. Introduction
  3. Funding
  4. Acknowledgment
  5. References

Research reported in this publication was supported, in part, by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number 2UL1TR000433–06. This research was also supported in part, by The Michigan Center for Diabetes Translation Research (MCDTR) from the National Institute of Diabetes and Digestive and Kidney Diseases P30DK092926 and by the Prevention Research Center of Michigan Cooperative Agreement Number 1-U48-DP-001901 from the Centers for Disease Control and Prevention. The content of this article solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or The Centers for Disease Control and Prevention.

Acknowledgment

  1. Top of page
  2. Introduction
  3. Funding
  4. Acknowledgment
  5. References

We gratefully acknowledge the important contributions of CERC staff members who reviewed, revised, and discussed the guidelines thus contributing significantly to the final product.

References

  1. Top of page
  2. Introduction
  3. Funding
  4. Acknowledgment
  5. References