SEARCH

SEARCH BY CITATION

Keywords:

  • community-engaged research;
  • community–academic partnerships;
  • grant review;
  • pilot grants;
  • institutional funding;
  • Michigan

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References

In 2007, the Michigan Institute for Clinical and Health Research (MICHR) at the University of Michigan received a Clinical and Translational Science Award (CTSA). Within MICHR, the Community Engagement (CE) program supports partnership efforts between researchers, practitioners, and community-based organizations in specific focal communities throughout Michigan. A key component of the CE program is the Community Engagement Coordinating Council, a group that provides input and guidance on program priorities, strategic planning, and reviews pilot funding proposals for community–academic partnerships. This paper will describe a unique MICHR pilot funding mechanism for Community–University Research Partnerships (CURES) with an emphasis on the ways that community partners are involved in the review process, as well as the benefits, challenges, and insights gained over 5 years of pilot review. There is a growing need for community involvement and expertise in review of funding proposals for community-engaged research at both institutional and federal levels. The CURES pilot review process is one example of an institutional effort to engage community partners in university funding decisions and has demonstrated clear benefit toward accomplishing the aims of the CTSA.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References

In 2007, the Michigan Institute for Clinical and Health Research (MICHR) at the University of Michigan (UM) received a Clinical and Translational Science Award (CTSA). A community engagement and research core was in place at the time of funding and has evolved over the years. The MICHR Community Engagement (CE) program supports partnership efforts between researchers, practitioners, and community-based organizations in specific focal communities throughout Michigan, including Ypsilanti, Ann Arbor, Detroit, Flint, Dearborn, and the Upper Peninsula. A key component of the CE program is the Community Engagement Coordinating Council (CECC), a group that provides input and guidance on program priorities, strategic planning, and reviews pilot funding proposals for community–academic research partnerships. The CECC is made up of approximately 4 UM faculty, 5 staff members from various UM departments and schools, and 10–12 community partners who represent community-based organizations across MICHR focal communities. CECC meetings are facilitated by two community cofacilitators that are nominated and elected by community partners on the Council. The community cofacilitators also participate in additional CE program meetings and CTSA consortium activities, such as the annual CTSA Community Engagement Conference. Through the CECC, the CE program strives to integrate the community's expertise and knowledge in all of its work to improve the quality of clinical health research and produce outcomes that measurably benefit the health of local communities.

Community involvement in community-engaged research grant review

The emergence of Community-Engaged Research (CEnR) in CTSAs represents new opportunities to create transformative change in both academic institutions and communities seeking to improve health outcomes. CEnR is an umbrella term used to describe various types of community–academic collaborations that necessitate varying levels of colearning, power sharing, and a commitment to translating research findings into action for improved outcomes.[1-4] CEnR challenges the dominant paradigm of traditional research, where communities have historically been excluded from participation and involvement, and puts researchers and communities on a path toward equitable partnership characterized by shared responsibility and decision making.[5] From 2006 to 2012, the CTSA funding mechanism required CE as a core function to foster collaborative partnerships and enhance public trust in research (NIH RFA-RM-10–020). CTSAs are also required to house pilot programs that provide funding and administrative support for innovative clinical and translational research projects (NIH RFA-TR-12–006).

Many CTSA CE programs utilize pilot and seed grants as one strategy to facilitate the development of new community–academic partnerships, encourage younger faculty to gain skills in CEnR approaches, and generate data that will be used to secure external funding for larger-scale health interventions and studies.[6, 7] Some CE programs have begun to identify challenges (e.g., matching interested community and academic partners, prioritizing community-defined health problems) and successful outcomes (e.g., increased financial and social return on investment, improved community capacity to conduct research) from funding relatively small-scale ($5,000–$50,000) CEnR activities.[8-10] At the federal level, increased funding for CEnR has brought with it a unique set of challenges for funding agencies, reviewers, and applicants, particularly with regard to issues of equity and effectiveness within the grant review process. The NIH peer review system is built within an academic culture that requires scientific training and expertise to evaluate research proposals.[11] Community-based reviewers, however, that have direct experience with and knowledge of community–academic partnerships are also needed to balance academic perspectives and add valuable insights that would otherwise be missing from review discussions. Moreover, the standard NIH review criteria used to evaluate proposals for scientific merit may not be responsive to principles of community engagement and participation (e.g., extent of community involvement in all phases of research, evidence of prior collaboration, attention to partnership development).[12, 13]

CTSA pilot funding practices and review processes often mirror NIH standards and regulatory requirements. Consequently, CE programs within CTSAs that fund CEnR must be equally cognizant of ways that pilot funding might be hindered by a lack of community involvement and committed to identifying strategies that address those gaps. This paper will describe a unique MICHR pilot funding mechanism called the Community–University Research Partnership (CURES) Award with an emphasis on the ways that community partners are involved in the review process, as well as the benefits, challenges, and insights gained over 5 years of CURES pilot review.

CURES award

MICHR's pilot grant program was established to catalyze research that spans the bench, bedside, and the community. Both the pilot grant and CE programs support a funding mechanism called CURES that is designed to strengthen community–university partnerships for health research. CURES supports 1-year pilot or feasibility research studies of up to $25,000 that offer innovative interventions and/or techniques designed to benefit the health of the community where the research is being conducted. CURES aims to: (1) build capacity of communities and the UM faculty to engage in research partnerships; (2) combine the knowledge, wisdom, and experience in communities and at UM to make a positive impact on local health; and (3) enhance networks and infrastructure that will promote community research that not only takes place in the community, but engages the community in all phases of research.

Currently, there are two CURES rounds of funding and subsequent requests for proposals (RFP) issued each year. CURES guidelines require research teams to have at least two Investigators, one representing an academic unit at the UM, and one community-based organization that is located within one of MICHR's focal communities. The UM researcher or community partner may serve as the Principal Investigator (PI) as long as the project demonstrates collaboration and joint ownership in all research activities. Applications require a research plan that outlines specific aims, significance, study design and methods, a description of the partnership and collaborative work proposed, dissemination plan, and impact statement. Applications must also clearly show that the research topic is a priority to both the community partner organization and focal population. CURES encourages applications that seek to reduce health disparities and improve health outcomes in populations that face increased barriers to becoming engaged in research and are more likely to experience worse health outcomes. Examples of CURES-funded research topics include: increasing food insecurity through gardening, mental health for teen moms, HIV prevention among young men who have sex with men (MSM), and improving family relationships for incarcerated mothers.

While the pilot grant program administers the RFP and funding for CURES and several different internal pilot mechanisms, the CE program provides more tailored assistance for CURES applicants and awardees through preaward consultations, partnership development, and support with Institutional Review Board research protocols. Both programs have specific groups that review CURES proposals, described further in the next section.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References

CURES review process

CURES proposals are reviewed by two separate groups: the pilot grant program's Scientific Review Committee (SRC) and the CECC. The SRC primarily reviews applications through a scientific lens, whereas the CECC focuses on specific aspects of the community–academic partnership and community partner involvement in the research. Both groups use the same 9-point NIH scoring scale (Table 1) and basic review format, yet bring very different perspectives to the review process. Figure 1 displays the step-by-step review process for each group.

Table 1. NIH scoring scale and descriptors
ImpactScoreDescriptorAdditional guidance on strengths/weaknesses
High1ExceptionalExceptionally strong with essentially no weaknesses
 2OutstandingExtremely strong with negligible weaknesses
 3ExcellentVery strong with only some minor weaknesses
Medium4Very goodStrong but with numerous weaknesses
 5GoodStrong but with at least one moderate weakness
 6SatisfactorySome strengths but also some moderate weaknesses
Low7FairSome strengths but with at least one major weakness
 8MarginalA few strengths and a few major weaknesses
 9PoorVery few strengths and numerous major weaknesses
image

Figure 1. Description of the CECC and SRC review processes for CURES Pilots.

Download figure to PowerPoint

The SRC, comprising senior faculty, representing diverse professional disciplines across UM, utilizes a review process similar to NIH that places emphasis on “expert peer reviewers.” Each proposal is assigned three reviewers; two who have relevant content expertise and a separate reviewer focused on biostatistics. The SRC review process and scoring form are structured around the standard NIH review criteria (“Significance,” “Investigators,” “Innovation,” “Approach,” and “Environment”) and scoring scale; including “Overall Impact” which encompasses the five review criteria. Proposals that receive either the most competitive preliminary impact scores or wildly discrepant preliminary impact scores are forwarded for discussion by one of the study sections, while remaining proposals are triaged. Applications discussed by one of the study sections receive additional impact scores by each member of the panel, the average of which is used to calculate an application's final overall impact score.

The CECC brings a community-focused lens to pilot review that emphasizes a separate set of criteria specific to CEnR (Table 2). CECC reviewers consider whether the proposed partnership involves direct community participation in all stages of research, provides benefits to the community, addresses a research topic that was identified by and is of relevance to the community partner organization, allocates resources equitably between the university and community, and has the potential for sustainability within the community after the study ends. CECC reviewers also consider how well the project is articulated and the likelihood of leading to additional external funding through federal or foundation funding sources. By giving attention to these criteria, the CECC enhances pilot review and ensures community voice and insights to a process that would otherwise rely primarily on standards for scientific merit.

Table 2. CECC review criteria and questions
CriteriaReview question
Community involvementWhat is the extent of community participation in shaping the direction of the research?
Community priorityDoes the application address a health issue or research topic identified by the community?
Community benefitIs there a tangible benefit (i.e., product, program) that will be left with the community?
Project teamIs there an appropriate distribution of academic and community investigators on the project?
BudgetIs the budget distributed fairly between the academic and community partners?
SustainabilityWhat are the plans for sustaining the work of the community–university partnership beyond the 1-year funding period?
External fundingIs there a likelihood of leading to successful external funding through federal or private sources?

CE program staff members consult with pilot teams prior to submission and prescreen CURES applications to ensure that basic criteria and requirements are fulfilled (e.g., eligibility of community focal areas, community, and academic PIs). CE program staff sends requests for individual members on the CECC to serve as lead and secondary reviewers based on relevant knowledge and expertise of the research topic and community area or focal population. Given that the CECC comprises community and academic partners, CE staff work to ensure there is a balance of both when assigning reviews. All CECC members, except CE program staff, provide preliminary scores and comments on key strengths and weaknesses. Averaged scores and compiled comments are provided to the CECC before discussion at the review meeting. If SRC scores and reviews have been completed, they are also shared electronically with the CECC prior to the review meeting. The community cofacilitators on the CECC facilitate the review meetings and ensure timeliness of presentations. Proposals with the most competitive scores are reviewed first. The CECC collectively decides whether or not to triage proposals with the least competitive scores based on meeting time constraints and recommendations of the lead and secondary reviewers. CECC members then rescore each proposal in a format similar to SRC's format previously mentioned.

Funding decisions and support for pilot teams

In earlier rounds, both CECC and SRC review scores were averaged and the lowest (best) scoring applications awarded based on available funding for that round. Over time, the CURES funding decision-making process changed to reflect growing community involvement in pilot review. This change was prompted, in part, from a recognized need to give more weight and priority to community perspectives and the fact that CECC reviews represented a balance of perspectives, rather than only faculty perspectives primarily guided by scientific merit. In recent rounds, the final CECC scores have been averaged and used to determine whether proposals within an acceptable score range (typically no higher than 3) get funded. However, SRC reviews and scores still contribute to final decisions and proposals with at least one moderate scientific weakness typically are not considered by the CECC for funding.1

Regardless of the funding outcome, both CECC and SRC scores and comments for each application are shared with the respective pilot teams. If a pilot is funded, the research team meets with either the CECC or a smaller group of CE faculty and staff prior to implementation to discuss in depth the CECC review critiques, as well as project aims, timelines, and funding expectations. If a pilot is not awarded funding, teams are still encouraged to meet with CE faculty, staff, and CECC reviewers to discuss resubmission of their proposal. Investigators can submit a revised proposal only once and if not awarded upon resubmission, must significantly change the proposal or concept before reapplying.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References

The number of CECC members who participate in pilot review has steadily increased each round. The current number of reviewers ranges from about 10–12 community partners and 4–6 university partners. From March 2007 to February 2013, a total of 50 applications were submitted and reviewed during 12 rounds of CURES pilot grant funding (average of 4–5 proposals per round) and 16 projects were awarded totaling approximately $357,000 (Table 3). To date, CURES pilots have resulted in 13 peer-reviewed publications, 15 presentations at conferences and community-specific settings, and a 180% financial return on investment, including over $730,000 in extramural funding that has been secured by community partner organizations and approximately $330,000 by academic researchers. While the NIH's standard for return on investments for pilots is extramural funding and publications, the CE program also documents nontraditional measurements of success, such as the development of new health policies or practices, increased organizational capacity, CEnR knowledge and skill development, and community health improvements that may have resulted from pilot funding.

Table 3. List of funded CURES projects
RoundYearProject titleAwarded ($)
32008Engaging the Community in Health Care Research: Creating a Community/Academic Partnership$12,192
  The Flint Sleep Project: Understanding Sleep Deficits and Planning Sleep Education$21,654
  Genesee County Healthy Sexuality Project$25,000
  A Community–University Partnership to Develop Technology-Enhanced Patient-Empowerment Programs for People with Chronic Kidney Disease: Capacity Building, Feasibility Assessments, and Study Refinement$24,326
42008MPOWER Jr./YMCA Childhood Obesity Intervention in the Community Environment (My Choice)$25,000
52008The Four Square Society: Understanding the Impact of Urban Gardening in Reducing Food Insecurity in Ypsilanti, Michigan$24,780
62009Integrating Traditional Healing and Behavioral Health Services for Urban American Indians$25,000
72009Feasibility of Conducting a Peer-Led Lifestyle Modification Intervention for African Americans in a Church-Based Setting$24,361
  Building a Technology-Enhanced Social Network Intervention to Promote HIV Testing among Young Men Who Have Sex With Men (MSM): Pilot Study$24,994
82010Parenting While Incarcerated$25,000
92010The Peer-to-Peer Depression Awareness Project: A Collaboration Between the UM Depression Center and the Ann Arbor Public Schools$25,000
102011Determining High and Low Risk Vulnerability Factors in the Prevention of Perinatal Depression through Home Visiting in Low-Income Women: A Pilot Study$24,184
  A Community-Generated, Socioculturally Relevant Intimate Partner Violence Training: A Feasibility Study in South Asian Communities$24,991
112011Flint Conversation Map: Addressing Functional Health Literacy and Decision Making$25,000
122012Buenos Vecinos Survey: Building Capacity to Promote Washtenaw County Latino/a Health$25,000
  Moms 4 Moms: Community-Participatory Parenting Project for African American Teen Moms in Metro Detroit$25,000

The following insights represent lessons learned from community and university partners who have served as reviewers and staff who have assisted pilot teams and coordinated reviews.

Improving CECC pilot review procedures

The CECC review process has evolved over time to become more community-engaged, bidirectional, and participatory. Initially, only CE faculty served as lead reviewers and presented their review and critique of each proposal. CECC members then followed to offer discussion and additional critiques. A new approach more similar to NIH was adopted that included both a lead and secondary reviewer. Instead of the CE faculty presenting each application, all members of the CECC had the opportunity to be assigned as lead reviewers. This resulted in a shared process that achieved a greater balance between community and academic perspectives.

Community input and participation has also been crucial to improving the process and quality of reviews. For instance, community partner feedback related to conflicts of interest (COI) resulted in the creation and implementation of a formal COI disclosure process. In past rounds, when a CECC reviewer had what they thought to be a COI, they typically recused themselves from the review. When potential conflicts were under question, the CECC would discuss as a group to determine whether a COI existed. The CECC collectively decided that it was important to have guidelines that explained how to identify a COI and what to do in the event that a conflict existed. CECC member guidelines now include this information and more specific step-by-step instructions for completing reviews.

The value of community perspectives

Community perspectives are especially important for CURES review, in part, because applications frequently propose work with vulnerable populations that have historically been harmed by or excluded from research (e.g., racial/ethnic minorities, people living with HIV/AIDS).[14, 15] The complex causes of health problems experienced by these groups (e.g., chronic illnesses, racial/ethnic disparities) are best understood by community members living in the physical and social environments where the research is taking place.[15] Community members on the CECC offer important insights about the ethical protections of participants, cultural appropriateness, and the quality of the environment where the research is being conducted. The benefits to scientific quality that stem from this kind of community expertise are clear. For example, a data collection site that is not on a public transportation route could limit low-income group participation; insufficient attention to participant equity could increase attrition within longitudinal samples.

Prioritizing scientific and community standards for CEnR

The level of experience and skill set in evaluating research proposals varies among CECC reviewers and can sometimes draw attention away from community-relevant criteria and more toward the quality of science. In one example, an academic CECC reviewer scored an application as having a fatal flaw (i.e., ineligible for funding) based on a major methodological weakness, but this flaw was not recognized by some of the community partner reviewers on the CECC, thus resulting in discrepant views and scores. In this example, multiple issues were identified: (1) the academic reviewer's deviation from CECC review criteria, (2) tension between judging for quality of science versus community involvement, and (3) limited capacity for understanding and communicating issues of scientific merit between community and academic reviewers.

Typically, the SRC will identify significant scientific weaknesses and score applications accordingly. When available, SRC scores are made available to the CECC ahead of the review meeting, but only after preliminary scores have been submitted. CECC members have expressed concern that SRC reviews could bias CECC reviewers so as to not fully consider the full range of community-relevant review criteria when applications are being scored. These issues speak to the need for more bidirectional communication between the SRC and CECC and thorough understanding of the importance for both sets of reviews. MICHR faculty members have discussed the option of creating a special study section specifically for CURES applications that would combine community and academic expert reviewers onto one panel. Additional options for improving education and communication across these two review groups are being explored.

Clarity of CEnR terms and review criteria

Interpretation of review criteria has played a significant role in shaping scoring decisions. Within the CECC, community and academic partners often associate different meanings with key terms that have created confusion during review discussions and possible discrepancies in scoring outcomes. For example, a review question regarding “community benefit” asks reviewers to judge whether there “is a tangible benefit that will be left with the community.” With the exception of a few brief examples of “tangible benefit” in the scoring form (i.e., “new products or programs, increased knowledge”), reviewers are not given further instructions or definitions to assess applications for this criterion. Without a clear and consistent definition, reviewers may be looking for different types of benefits and may disagree on what qualifies as a benefit to the community. One CECC reviewer illustrated this point, stating “building capacity to do research in a community setting is a benefit to the academic partner; however, it is not always necessarily seen as a benefit to the individual organization participating in the project.” The CECC has expressed the need for creating definitions and building consensus on what constitutes similarly ambiguous review criteria (e.g., “community priority,” “community involvement,” “community partner”). A recent evaluation survey conducted by CE program staff revealed that 88% of CECC members felt pilot review could be strengthened through clarity and improved definitions of review criteria. CE faculty have developed a new resource that provides more detailed review questions for community-based reviewers (see: Training and education for community partners evaluating CEnR proposals) that will be used to address this area for improvement.

Understanding and responding to reviewer expectations

The CE program staff have multiple points of contact with CURES applicants and awardees, from the initial RFP and preaward consultations to issuing funding notices and postaward progress reporting. Throughout the course of application development, staff members offer one-on-one assistance with investigators so that research teams can effectively prepare strong, competitive applications that meet CURES goals. However, staff members have had difficulty advising teams when there is inconsistency between pilot grant program documents and reviewer expectations. For instance, some CECC reviewers have scored applications poorly based on the extent to which community partners are involved in shaping the research, yet the RFP does not specify what constitutes an acceptable level of community involvement. The discrepancy is particularly apparent when investigators are attempting to address reviewer critiques through resubmissions. This was highlighted recently by a CURES applicant who felt her proposal “engaged consumers in all aspects of the project except identification of the issue” and had received a comment from a reviewer that “the proposal does not do much to engage the community.” Applicants, reviewers, and staff all need to have a clear understanding of what is expected by reviewers so this does not become a barrier to receiving funding. One potential solution related to this problem is clarity in the RFP. The CECC recently updated the CURES RFP to include definitions of key terms and detailed descriptions of both the CECC and SRC's review criteria.

Increasing grant review knowledge and skills for community reviewers

CECC review of pilots builds capacity for the community partners to better understand the research and grant review process. Community partners who have reviewed over multiple rounds have reported greater confidence in their ability to evaluate research and score pilots using community-relevant review criteria. Participation in review has also shown to increase understanding of how traditional grant review occurs at the federal level. One CECC reviewer remarked that “I always learn more when someone talks about how NIH or another scientific body scores and determines what should be funded.” Based on comments received by both funded and unfunded CURES projects, applicant knowledge is also increased around CEnR. Review comments provide strategies to strengthen projects and make them more community-engaged. This insight has led to a better understanding of this type of research and assisted applicants seeking to secure additional funding.

Time and compensation for review activities

Pilot review requires a significant amount of time and effort. Some community partners on the CECC have reported difficulty in finding adequate time to do the reviews given existing pressures within their own organizations. There is a learning curve that comes with understanding scientific terminology, research design, and processes for evaluating research proposal and community partners with less experience need additional time and training. In addition to participating in the review meeting, all CECC members are asked to read and critique the content of, on average, four–five proposals each round, complete scoring forms, and submit preliminary scores and comments. Community partners are compensated $1,000 annually as part of an established agreement that details responsibilities for serving on the Council, including participation in at least 6 of 10 meetings and at least one round of CURES review. Despite this, some partners have expressed that compensation should be provided based on the number of applications, as opposed to the number of meetings, to more appropriately recognize the time needed to fully participate in pilot review. Issues related to compensation and member responsibilities are typically addressed at CECC meetings when all members have a chance to share thoughts and provide recommendations.

Training and education for community partners evaluating CEnR proposals

When CECC partners were first included in CURES pilot review, there were no specific materials that outlined the steps involved. As the CECC review process became more structured and refined, a detailed set of instructions were created to explain the process, review criteria, and scoring form. However, there is still a need for formal training and education on how to evaluate CEnR proposals. In the past, CECC members have expressed concern about having limited expertise in research design and understanding which methodologies are most appropriate for CEnR. CE faculty, in partnership with community partners, have been developing a set of guidelines for community-based partners for reviewing community-based research grant applications that contains a glossary of research terms, list of common research designs, and considerations for grant project critiques. These guidelines provide valuable community interpretations of the standard NIH review criteria and illustrate how they can be adapted for CEnR. For example, the guidelines state that the standard NIH criterion for “Significance” is determined not only by whether the study addresses an important problem in the field, but also if that problem is viewed as important by the majority of people in the community, especially the people that the study is designed to include. These materials are currently being incorporated into a larger set of established CECC member guidelines that provide information about other aspects of pilot review (e.g., instructions, COI disclosure procedures).

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References

MICHR's partnership with community members representing community-based organizations has become increasingly stronger as it works to accomplish CTSA aims. The CECC has provided a forum for identification and discussion of community-defined research priorities in the MICHR focal communities that has led to new relationships and collaborative research teams. We have found that developing a process for involving community perspectives in CURES proposal review to be an essential community engagement strategy. The results reported here provide tangible evidence of the critical contributions that engaged knowledgeable community members make to the rigor of clinical research and the grant review process. Ongoing bidirectional learning has been inevitable: researchers have come to recognize that the absence of community input can seriously compromise the effectiveness of a clinical research project and community members have gained skills in providing guidance to research teams as they develop applications for pilot funding.

The results also shed light on some of the challenges that are faced when grant review practices for traditional models of research are adapted for community-engaged approaches. Although scientific and community reviewers bring unique perspectives, both groups need to have a common understanding of CEnR and the necessary skills and experience to evaluate CEnR proposals. Barriers to involving community members, patient advocates, and other stakeholders outside the scientific community in grant review have been documented and often include concern that scientists do not value community contributions,[16] discomfort on the part of community members who are unfamiliar with scientific language and subject matter, and divergent perspectives on review criteria between scientist and public reviewers.[17] To address these barriers, the NIH Council of Public Representatives created a framework and criteria for assessing community engagement in research proposals.[18] Green and colleagues have also developed a comprehensive set of reliability-tested guidelines and rating scales for assessing participatory research proposals that have been adapted and used by members of past CBPR review panels.[19, 20] These and other resources, such as the National Cancer Institute's “Consumer Guide to Peer Review” could be adapted and utilized to build the skills and capacity of community reviewers within CTSAs.[21]

A 2011 survey conducted by the CTSA Community Engagement Key Function Committee Community Partners Integration workgroup found that community representatives had significant levels of formal involvement in nearly all CTSAs (94%), but their work was mostly compartmentalized within CE cores and did not extend to other core programs.[22] Establishing roles for community partners to serve as reviewers on CE pilot grants has helped increase community voice within MICHR, include community perspectives in funding decisions, and improve CEnR research proposals. We are confident that CTSAs with similar staff, funding infrastructure, and levels of community involvement could replicate or build on this approach to strengthen community presence in their CTSA institutions and pilot grant programs.

Since the CECC first began reviewing proposals in 2008, a number of key lessons and pilot program improvements have been incorporated as a result of reviewer and applicant feedback. Building on the insights we have gained, we make the following recommendations:

  1. Involve community partners at every step of grant review process development and implementation.
  2. Provide CEnR training and ongoing guidance for community reviewers. Education is critical.
  3. Establish clear, consistent, and mutually agreed upon definitions for review criteria.
  4. Allocate resources and infrastructure (staff and faculty time, fiscal) to fairly compensate community partners.
  5. Organize bidirectional communication and learning opportunities between scientists and community-based reviewers.

Conclusion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References

Investing in pilot funding for CEnR has been recommended as a key strategy to overcome barriers that academic health centers face in engaging communities.[23] There is a growing need for community involvement and expertise in review of funding proposals for CEnR at both institutional and federal levels. The CURES pilot review process is one example of an institutional effort to engage community partners in university funding decisions that has demonstrated clear benefit toward accomplishing the aims of the CTSA.

Acknowledgments

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References

We gratefully acknowledge the critical contributions of MICHR staff members who generated initial guideline drafts and participated in discussions that resulted in the final product, particularly Beth LaPensee who provided significant editing support. We especially thank the community partners and UM faculty who laid the foundation for this work, and the research teams that have applied for CURES funding and contributed toward program accomplishments over the years. The project described was supported by the National Center for Research Resources, Grant UL1RR024986, and is now at the National Center for Advancing Translational Sciences, Grant UL1TR000433. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

  1. 1

    The pilot grant program uses a different process for making funding decisions for all other pilot funding mechanisms. The pilot grant program awards based on the overall impact score or percentile rank of the applications in comparison with other applications. Number of awards is dictated by current available funds (typically the highest ranking 25% or 30%). Where ties exist, leadership relies on relevance to program goals, including but not limited to; scientific merit, potential for extramural support, whether or not an applicant has been previously funded by the Pilot Grant Program, and academic rank (with emerging or junior investigators receiving priority).

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusion
  8. Acknowledgments
  9. References
  • 1
    CTSA Community Engagement Key Function Committee and CTSA Community Engagement Workshop Planning Committee. Researchers and their communities: The challenge of meaningful community engagement. Duke University. 2009. Available at: https://ctsacorus.org/resources/273/download/Monograph-ResearchersTheir_Communities-CTSA-2007-08_3.pdf. Accessed May 13, 2013.
  • 2
    Israel B, Schulz A, Parker E, Becker A. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998; 19(1): 173202.
  • 3
    Sadler LS, Larson J, Bouregy S, Lapaglia D, Bridger L, McCaslin C, Rockwell S. Community–university partnerships in community-based research. Prog Community Health Partnersh. 2012; 6(4): 463469.
  • 4
    Sadler LS, Newlin KH, Johnson Spruill I, Jenkins C. Beyond the medical model: interdisciplinary programs of community engaged health research. Clin Transl Sci. 2011; 4(4): 285297.
  • 5
    Ross LF, Loup A, Nelson RM, Botkin JR, Kost R, Smith GR Jr, Gehlert S. Human subjects protections in community-engaged research: a research ethics framework. J Empir Res Hum Res Ethics. 2010; 5(1): 517.
  • 6
    CTSA Principal Investigators, Shamoon H, Center D, Davis P, Tuchman M, Ginsberg H, Califf R, Stephens D, Mellman T, Verbalis J, Nadler L, et al. Preparedness of the CTSA's structural and scientific assets to support the mission of the national center for advancing translational sciences (NCATS). Clin Transl Sci. 2012; 5(2): 121129.
  • 7
    Frechtling J, Raue K, Michie J, Miyaoka A, Spiegelman M. The CTSA national evaluation final report. Rockville, MD: Webstat, 2012. Available at: https://www.ctsacentral.org/sites/default/files/files/CTSANationalEval_FinalReport_20120416.pdf Accessed May 7, 2013.
  • 8
    Tendulkar SA, Chu J, Opp J, Geller A, Digirolamo A, Gandelman E, Grullon M, Patil P, King S, Hacker K. A funding initiative for community-based participatory research: lessons from the Harvard catalyst seed grants. Prog Community Health Partnersh. 2011; 5(1): 3544.
  • 9
    Winckler E, Brown J, LeBailly S, McGee R, Bayldon B, Huber G, Kaleba E, Lowry KW, Martens J, Mason M, et al. A novel program trains community-academic teams to build research and partnership capacity. Clin Transl Sci. 2013; 6(3): 214221.
  • 10
    Main D, Felzien M, Magid DJ, Calonge BN, O'Brien RA, Kempe A, Nearing K. A community translational research pilot grants program to facilitate community–academic partnerships: lessons from Colorado's clinical translational science awards. Prog Community Health Partnersh. 2012; 6(3): 381387.
  • 11
    Seifer S, Shore N, Holmes S. Developing and sustaining community-university partnerships for health research: infrastructure requirements. Community-Campus Partnerships for Health. 2003. Available at: http://ccph.memberclicks.net/assets/Documents/FocusAreas/infrastructurerequirements.pdf. Accessed May 15, 2013.
  • 12
    Seifer S, Greene-Moton E. Realizing the promise of community-based participatory research: community partners get organized! Prog Community Health Partnersh. 2007; 1(4): 291294.
  • 13
    Palermo A, Park A, Seifer S, Ybarra V. Response to NOD-07–074 – the NIH peer review process. Community Partner Summit Policy Workgroup. 2007. Available at: http://ccph.memberclicks.net/assets/Documents/FocusAreas/responsenihpeerreviewprocesssept7.pdf Accessed May 16, 2013.
  • 14
    Killien M, Bigby JA, Champion V, Fernandez-Repollet E, Jackson RD, Kagawa-Singer M, Kidd K, Naughton MJ, Prout M. Involving minority and underrepresented women in clinical trials: the national centers of excellence in women's health. J Womens Health Gend Based Med. 2000; 9(10): 10611070.
  • 15
    Bhopal R. Ethical issues in health research on ethnic minority populations: Focusing on inclusion and exclusion. Res Ethics Rev. 2008; 4(1): 1519.
  • 16
    Monahan A, Stewart D. The role of lay panelists on grant review panels. Chronic Dis Can. 2003; 24(2–3): 7074.
  • 17
    Andejeski Y, Breslau ES, Hart E, Lythcott N, Alexander L, Rich I, Bisceglio I, Smith HS, Visco FM; U.S. Army Medical Research and Materiel Command Fiscal Year 1995 Breast Cancer Research Program Integration Panel. Benefits and drawbacks of including consumer reviewers in the scientific merit review of breast cancer research. J Womens Health Gend Based Med. 2002; 11(2): 119136.
  • 18
    Ahmed S, Palermo A. Community engagement in research: frameworks for education and peer review. Am J Public Health. 2010; 100(8): 13801387.
  • 19
    Green LW, George MA, Daniel M, Frankish CJ, Herbert CP, Bowie, WR, O'Neill M. Guidelines for participatory research in health promotion. In: Minkler M, Wallerstein N, eds. Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass Publishers; 2003, 419428.
  • 20
    Minkler M, Blackwell A, Thompson M, Tamir H. Community-based participatory research: implications for public health funding. Am J Public Health. 2003; 93(8): 12101213.
  • 21
    National Cancer Institute. The NCI consumer guides for peer review Website. Available at: http://deainfo.nci.nih.gov/consumer.htm. Accessed May 21, 2013.
  • 22
    Wilkins CH, Spofford M, Williams N, McKeever C, Allen S, Brown J, Opp J, Richmond A, Strelnick AH. Community representatives’ involvement in clinical and translational science awardee activities. Clin Transl Sci. 2013; 6(4): 292296.
  • 23
    Michener L, Cook J, Ahmed SM, Yonas MA, Coyne-Beasley T, Aguilar-Gaxiola S. Aligning the goals of community-engaged research: why and how academic health centers can successfully engage with communities to improve health. Acad Med. 2012; 87(3): 285291.