A taxonomy of impacts on clinical and translational research from community stakeholder engagement

Abstract Background Community engagement is increasingly recognized as a valuable tool in clinical and translational research; however, the impact of engagement is not fully understood. No standard nomenclature yet exists to clearly define how research changes when community stakeholders are engaged across the research spectrum. This severely limits our ability to assess the value of community engagement in research. To address this gap, we developed a taxonomy for characterizing and classifying changes in research due to community engagement. Methods Using an iterative process, we (a) identified areas of potential impact associated with community engagement from author experience, (b) categorized these in taxonomic bins based on research stages, (c) conducted semi‐structured interviews with researchers and community stakeholders, (d) validated the codebook in a sample dataset and (e) refined the taxonomy based on the validation. Community stakeholders were involved in every step of the process including as members of the primary study team. Results The final taxonomy catalogues changes into eleven domains corresponding to research phases. Each domain includes 2‐4 dimensions depicting concepts within the domain's scope and, within each dimension, 2‐10 elements labelling activities through which community engagement could change research. Conclusions Community engagement has great potential to enhance clinical and translational research. This taxonomy provides a common vocabulary and framework for understanding the impact of community engagement and suggests metrics for assessing the value of community engagement in research.


| BACKG ROU N D
Patient and community stakeholders are being involved in re-shaping priorities for health research, setting the research agenda, establishing a presence on proposal review committees, and translating research results into easily understood findings for the public audience. [1][2][3] Viewed retrospectively, community stakeholders' contributions have added community needs to research priorities, 4 produced culturally tailored and targeted recruitment strategies 5 and patient-oriented study material, 6 enhanced approaches to research design and implementation, 7 and improved translation and dissemination of research findings. Community Engagement (CE) Studios, 8 focus groups, community listening sessions, 9 advisory/ oversight councils, 10 and grant review committees are examples of strategies employed to involve community stakeholders in clinical and translational research. [11][12][13] Expanding the research process to include patients, caregivers, patient advocates or members of the general public involves bringing researchers together with those who are not primarily affiliated with academic research institutions.
Community stakeholder engagement, then, is multi-disciplinary and complex, yet it lends a lived-experience perspective so that health research itself better reflects what is most important to the population it studies and serves. 14 Lagging behind the growth of new stakeholder engagement approaches is the development of tools for evaluating, comparing and evolving those approaches, and there is an urgent need to develop these tools to demonstrate the impact of community stakeholder engagement in research. 11,15,16 Interactions between researchers and community stakeholders are not consistently captured in a standard or ordered framework, nor is the value of community stakeholders' activities to the research enterprise being measured. [17][18][19][20] With valuation standards and metrics, the meaningful engagement of patients and other community stakeholders could be studied scientifically and adopted with more confidence in clinical research, which is still largely done to patients as participants rather than with them as stakeholders in a bidirectional interaction. [21][22][23] It is imperative to capture community stakeholder input consistently and develop measures for the value of the community stakeholder contributions to research.
There are examples in the literature demonstrating the effectiveness of taxonomies for improving metrics and scientific reporting, suggesting a taxonomy would be an effective first step in establishing a standard vocabulary and developing value measures. [24][25][26][27][28] Other stakeholder engagement efforts are illustrative of the benefits of improving vocabulary around this topic. These include the following: stakeholder engagement frameworks and guidance not focused on community stakeholders specifically, 18,29,30 a scaleable approach to patient engagement for patient-centred outcomes research (PCOR), 31 successful patient engagement for health-care experiences and outcomes, 32,33 and community engagement measures focused on partnership strength. 19,34 Specifically evaluating community stakeholders' contributions to research, however, needs a framework specifically focused on characterizing and measuring community representative activities through the process of conceiving, conducing, analysing and reporting clinical and translational research.
Given the complexity of community stakeholder engagement in clinical research, a taxonomy would provide a common language and framework for community stakeholder engagement that will facilitate needed standards for reporting and measures for metrics development. 35 Over time, reporting and evaluating stakeholder engagement systematically will accelerate advancements in and adoption of community stakeholder engagement across research broadly.
We developed a Community Stakeholder Impacts on Research Taxonomy to address this need.

| Definitions
In this work, the term "community stakeholder" includes patients, caregivers, patient advocates and members of the general public, but not payers, policy makers or health-care product producers. A "community representative" is a person whose primary affiliation is with a non-academic, non-research, community-based organization and/or who represents a defined community. 36

| Overview
We (a) identified areas of potential impact and outcomes associated with community stakeholder participation in clinical and translational research based on author experience, (b) categorized these in taxonomic bins based on the research cycle, (c) conducted semi-structured interviews with researchers and community stakeholders to evaluate the resultant taxonomy, (d) validated the taxonomy in a sample dataset and (e) refined the taxonomy based on the validation. For qualitative analyses, all coding was completed using Dedoose software, an online suite for collaborative qualitative research analysis.
Our research team included leaders from two community organizations (Vaughan and Richmond) and faculty/staff from three institutions with expertise in community engagement, scale development, qualitative analysis and translational research. The experience of the team spanned facilitating CE Studios, conducting community outreach efforts, recruiting for programme participation, implementing public health interventions and evaluation, and advocating for social and economic justice. The study design, recruitment plans and semi-structured interview questions were approved by Vanderbilt University Medical Center's IRB.

| Identification of potential community stakeholder impacts
To generate initial content for the taxonomy, we scanned the literature reporting research in which patient, community and provider stakeholders have been involved. The content generation was guided by our team's expertise in engagement, the PCORI Patient and Family Engagement Rubric, 37,38 and a recent comprehensive review of impact. 39 Searches in PubMed and Google Scholar included these keywords: Community-Engaged Research (CEnR), patient and stakeholder engagement in research, participatory research, patient-centered outcomes research, impact of community/patient/ family/caregiver engagement in research, and evaluation of community/patient engagement in research. 16,[39][40][41]

| Categorization of impacts into initial taxonomy
Two experienced faculty on our team independently reviewed the identified publications and annotated the content related to changes in research from stakeholder engagement activities. Codes were generated using an inductive approach and subsequently grouped based on thematic analysis. Through iterative rounds of review and discussion, the full team (faculty/staff and community stakeholders) developed an initial taxonomy with top-level domains, representing areas where research changes might occur (ie specific research stage or overarching thematic area) and elements, defining the scope of activity in each domain. The elements represent activities that can be assessed or measured. We developed a codebook for qualitative analysis, making domains the parent codes and elements the subcodes.

| Evaluation of initial taxonomy and external content collection
We conducted 12 semi-structured interviews -six with academic researchers and six with community stakeholders -to evaluate the initial taxonomy and gather external content. One week prior to the interview, interviewees were provided with the initial taxonomy (Table 1A). Interviewees answered questions on taxonomy structure (domain nomenclature, domain arrangement and element categorizations), utility and relevance (Table 1B). Interviewees were questioned about each domain and its elements and about their overall impressions. Upon completion of the interview, both academic and community participants were compensated $50 for their time. We used a "think aloud" method to probe deeper into responses given by the interviewees to provide a richer thought process with examples. 42 The semi-structured interviews were recorded, transcribed verbatim and de-identified by two research team analysts who also acted as coders. The two coders independently reviewed the transcripts and coded participants' responses to each domain and element of the taxonomy as indicating "keep", "remove", "add", or "needs improvement" about that particular part of the taxonomy.
Discrepancies in codes were resolved through team adjudication.
Wording changes for clarity, element clustering into taxonomic dimensions, and element recategorizations were discussed among the research team to improve and refine the taxonomy in accordance with the interview results. Coder Group B (n = 3) had two transcripts on the same topic, one each from a CE and T2 Studio both held on eConsent. (Table 2) Afterwards, we interviewed each coder, asking: How did the tool work for them? What challenges did they experience? How can the taxonomy be improved? What were your overall likes and dislikes in the utility of the system? The final taxonomy content and structure resulted from discussion in the research team. Elements describing measurable activities were binned into subdomains, or taxonomic dimensions, describing categories of research activity in each stage.

| External review results
Semi-structured interview results from the coding of researcher and community stakeholder interview transcripts were suggestions of what in the initial taxonomy to keep, remove, add or improve. From these data and subsequent research team discussion, Translation,

| Final taxonomy
The final taxonomy of Community Stakeholder Impacts on Research has eleven domains (codes) describing stages of clinical and translational research, 36 dimensions naming research activity concepts into which subcodes were binned, and 71 elements (subcodes) describing specific community stakeholder activities that can be assessed or measured (Table 4). Links observed while piloting the Community Stakeholder Impacts on Research Taxonomy support a cyclical and iterative model of the research process with opportunities for stakeholders to engage at all phases of research and inform next steps ( Figure 1). The taxonomy systematically characterizes and categorizes community stakeholder activities that can impact the research process and also suggests possibilities for standard measures to assess that impact (Table 4 and Figure 1). Examples of possible measures are listed in the rightmost column of the taxonomy (Table 4).

| Pre-research and Infrastructure Domains
These research phases frame the overall study and the potential out- planning process, such as proposal development and priority-setting.

| Analysis and Dissemination Domains
In these phases of research, the cultural relevance and appropriate language brought to interpretation and presentation affects uptake of the health message and the diversity of the research par-

| Process improvement Domain
In addition to providing new elements for the Ethics domain, the community interviewees identified Process Improvement as a domain in which they felt they had contributed guidance and oversight (see quote). what we just talked about …" … "I think it really will reduce patient stress and reduce all of the risks the patients have. It doesn't make sense to be stressed out while you're in the hospital, which happens a lot.
I went to the emergency room one time, and I was worse off when I left than when I went because it was stressful. So, it seems like you are cleaning up all that. And to be asked how to do that was really a good thing … just the fact that someone was looking to change it and fix it."

| Communication Domain
Communication, a stakeholder impact domain that crosses all phases of research, was identified during the taxonomy pilot. Prior reports support community stakeholder engagement as an approach to increase the translation, dissemination and uptake of research findings. 45

| Limitations
The listing of measurable elements can and will grow as we were not able to capture every existing encounter between researchers and stakeholders. This taxonomy was pilot tested on transcripts from real-world studios; however, this does not capture all contexts in which stakeholders are engaged. This limitation is reflected in the high number of free codes found during the validation (Table 3).
Since the method we used is reproducible and the taxonomy flexible, new concepts can be built in as different engagement contexts are evaluated using the taxonomy.
The taxonomy development process revealed cross-over concepts. Some conceptual elements uncovered in our study belong in more than one dimension and even more than one domain. We believe, however, that this mirrors the research process itself, which is iterative and not always linear. The cross-over elements also reflect the complexity of investigator-community stakeholder interactions.
Many research activities repeat, iterate and occur in multiple process domains. For example, stakeholders can share input on creating materials (such as recruitment materials), survey design and summary of results, activities which can occur in the Research Design, Implementation, and/or Dissemination domains. This multiplex hierarchical structure is common in medical terminology and similar to that seen in medical subject headings (MeSH). The taxonomy's illustration of, and standard structure for, areas of value from stakeholder input is its primary contribution.

| CON CLUS IONS
Community engagement has great potential to enhance clinical and translational research. The Community Stakeholder Impact on Research Taxonomy provides a common vocabulary and framework for understanding the impact of community engagement and suggests metrics for assessing the value of community engagement in clinical and translational research. The taxonomy organizes the complexity of engagement into a framework that can be used to consistently report engagement activities and measure their impact.
Measuring stakeholder impact as engagement strategies are envisioned and tried will drive increased stakeholder involvement and channel it towards the most effective strategies, a needed advance for this field. We anticipate types of engagement will grow as engagement science grows. We see value in the taxonomy's flexibility, and in the reproducibility of the method used to devise it, to capture that growth in a structured way.

| Ethics approval and consent to participate
The reported approach and interview questions were approved by Vanderbilt University's IRB (#140955). The study was deemed exempt, and the consent procedures were approved. For the randomization to CE or T2 Studios part of the study, investigators were administered a survey and survey completion served as implied consent and was recorded with the survey responses in REDCap.
The following is the consent statement at the beginning of each For the structured interviews, consent was granted verbally and recorded along with the interview. The following is the consent statement at the beginning of each structured interview: In order to improve the use and understanding of the instrument, I will be asking you a series of ques-

CO N FLI C T O F I NTE R E S T
The authors declare that they have no competing interests.

DATA ACCE S I B I LIT Y
Data sharing is not applicable to this article as no new data were created or analysed in this study.