Identifying priority areas to support primary care engagement in breast cancer survivorship care: A Delphi study

Existing approaches in cancer survivorship care delivery have proven to be insufficient to engage primary care. This study aimed to identify stakeholder‐informed priorities to improve primary care engagement in breast cancer survivorship care.


| INTRODUCTION
Breast cancer is the most prevalent cancer among women, affecting over 4 million women in the U.S. and represents the largest population of longer-term cancer survivors. 1hile the overall five-year and 15-year relative survival rate for breast cancers are 91% and 80%, respectively, 1 women with a history of breast cancer face considerable medical, physical, and psychosocial late effects that persist beyond treatment. 24][5] A recent study across 33 states from 2016 to 2020 reported that primary care was involved in care delivery for 65.4% of breast cancer survivors. 6Primary care clinicians are also willing to participate in survivorship care. 6Moreover, evidence suggests that primary care-engaged survivorship care models are cost-effective and as efficacious in identifying recurrence as oncology-led models. 7However, their adoption continues to flounder in the U.S. Identified barriers to primary care delivery of survivorship care include uncertainty about roles, lack of training, and inaccessibility of documentation detailing the needs of patients with a history of cancer. 8he idea that primary care needs to be engaged is not new.It was a central focus of the Institute of Medicine's (now the National Academy of Medicine) seminal survivorship report 17 years ago, outlining recommendations that described the need to engage primary care and strategies to support primary care in survivorship care. 9,10n 2016, the American Cancer Society and American Society of Clinical Oncology published recommendations to guide primary care in the follow-up of longerterm cancer survivors. 11In the U.S., the engagement of primary care in survivorship delivery may be hampered by the weak and under-resourced primary care system infrastructure. 9,12The systemic shortcomings of U.S. primary care have warranted the publication of two separate National Academy of Medicine reports (1996 and 2021), the latter of the two acknowledging that little to no progress has emerged since the first. 12After the seminal survivorship report, survivorship care plans to inform primary care and clinical guidelines outlining care needs were the major national strategies to support primary care engagement. 13Barriers to care plan implementation include lack of resources and systems to support their development and distribution. 13,14spite the evidence supporting primary care engagement in the care for patients with a history of breast cancer, approaches for improving survivorship care have tended to be onco-centric.As a result, these efforts have done little to translate survivorship care delivery into the realities of U.S. primary care.For example, a comparative case study describing 12 primary care practices with advanced workforces (e.g., sampled from a national registry of workforce innovators) found that none of these clinics had implemented survivorship clinical services.One of the barriers identified from these 12 practices was the lack of actionable information to guide patient care. 15Hence, the literature has encouraged participatory research that emphasizes the need to understand the U.S. primary care context, capacity, and receptivity to address the translational lag of primary care engaged survivorship care. 10,16Given the onco-centric approaches identified in the extant literature, this study intentionally centers on the context and priorities of primary care and builds from insights gleaned from expert interviews of Phase I of a larger project focused on addressing the barriers of translating survivorship care into U.S. primary care clinics.The larger project aims to develop an intervention to be implemented and evaluated in a hybrid type 1 effectiveness-implementation cluster randomized design in 26 primary care practices. 17Phase 1 interviews explored breast cancer survivorship care experts' current working models of care and implementation experiences.This study phase convened a Delphi panel of primary care-engaged survivorship experts to identify primary care-centered priorities to enhance breast cancer care and reflect on their value according to their collective wisdom.The next phase of this project aims to tailor these priority items identified by primary-engaged survivorship care experts to clinic contexts in order to more carefully consider strategies for implementation.

| MATERIALS AND METHODS
9][20] Drawing on the collective knowledge of experts has been argued to lead to more accurate results than relying on individual judgment. 21,22Delphi breast cancer, communication, Delphi panel, primary care, survivorship care panels enable panelists to assess the value of available knowledge, whether that knowledge stems from prior scientific findings and/or from the collective expertise of panelists. 223][24] As significant uncertainty persists regarding how to transfer knowledge from cancer care to primary care and because challenges may stem from the oncocentric efforts to date, consensus-building approaches leveraging the clinical judgment of a diverse group of primary-care engaged survivorship experts to evaluate current knowledge related to clinical questions can help establish the priority areas that may best enhance primary care involvement in the care of patients with a history of breast cancer.

| Expert panel participants
As is common in Delphi panels, 18,19 we purposively sampled cancer survivorship experts.We considered individuals who were currently participating in and/or developing primary care engaged cancer survivorship programs in the U.S. to possess expert knowledge about (1) the value of primary care's engagement in survivorship care and (2) strategies required for primary care to participate effectively in the care of patients with a history of breast cancer.We identified expert panelists using three approaches: (1) recommendations from the larger project's scientific advisory committee members and consultants who include experts in survivorship care; (2) snowball sampling, that is, recommendations from individuals we identified as survivorship experts working at the cutting edge of survivorship care and whom we interviewed during Phase I of the study; and (3) internet searches of leading cancer survivorship programs in the U.S. Because Delphi panels with multiple stakeholder groups have been found to generate more accurate results, [19][20]25 our sampling approach intentionally sought diverse representation among panelists: primary care clinicians; oncology care clinicians; patients, survivors, and patient advocates; researchers (i.e., those testing models of care for survivors); and administrators.

| Panel design
This Delphi panel included four rounds.Data was collected between 8/8/2022 and 9/30/2022.In Round 0, we asked panelists to describe priority areas when they were asked to "identify what you think is needed for primary care to deliver quality survivorship care for patients with a history of breast cancer."We intentionally assessed primary care involvement broadly rather than using a specific survivorship care delivery model as previous research indicates limited adoption of any of the models described in the literature. 15,26In Round 0, panelists were able to describe their recommendations without word limits in an open text box format. 27From Round 0, 77 recommendations were generated.Additionally, prior to the Delphi panel, an initial list of 46 priority areas was developed from 40 in-depth interviews 28 with diverse stakeholders with interest in survivorship (n = 32) as well as primary care clinicians (n = 8).Each of the 77 panelistgenerated items was cross-checked against the initial list of 46 items developed from Phase I interviews.To finalize the list of priority items for Round 1, the research team consolidated items if redundant.Priority items that conceptually overlapped or those that included multiple priority items were refined and clarified in the final item description (see Figure 1).This process resulted in 31 unique priority items presented in the subsequent rounds.The research team organized the 31 priority items into six thematic categories: (1) primary care practice training (n = 7); (2) resource development and distribution (n = 6); (3) oncology-primary care connections (n = 5); technological solutions (n = 5); patient education (n = 4); and system-level strategies (n = 4).However, priority items were presented to panelists as independent items to not influence their evaluations.
Given the volume of items, the initial goal of Round 1 was to reduce the number of items for the subsequent evaluation rounds.While not a traditional step in the Delphi process, we added this step due to concerns about participant burden and attrition. 29In Round 1, we asked panelists to prioritize the "top 10 recommended strategies most crucial for supporting primary care engagement in the care for patients with a history of breast cancer."Any unranked items from Round 1 were eliminated from the subsequent evaluation rounds.We designed the study to include two evaluation rounds to avoid forced consensus due to participant burnout. 29,30In Round 2, we asked panelists to assess each of the items from the Round 1 consolidated list of items on a 9-point Likert-type scale along two dimensions: importance (how likely the strategy is to improve breast cancer survivorship care and/or adequately resolve a challenge with survivorship care delivery in primary care settings) and feasibility (how likely it is the strategy can be implemented in practice).In Round 3, panelists were asked to review the group results (medians, IQRs) and reevaluate the importance and feasibility of each item in an effort to reach consensus.Panelists were paid $50 for each evaluation round (i.e., Rounds 2 and 3), which they completed up to $100.The Rutgers University IRB approved the study protocol (Pro 2021000838), and electronic informed consent was obtained from all participants.

| Data analysis
To determine the final group evaluation for each priority item, 31 we applied the RAND/UCLA Appropriateness Method (RAM) Manual because it provides a validated approach to consensus for nine-point Likert scales. 32A priority item was considered important or feasible if: (a) less than a third of all responses were in the lower tertile of response categories (response options 1-3) and in the upper tertile (response options 7-9) (i.e., responses were not bimodally distributed) and (b) the group median fell into the upper tertile (categories 7-9).If the distribution was not bimodal and the group median fell into the lower tertile (response options 1-3), the group decision was considered not important or not feasible.If the median fell into the middle tertile (response options 4-6), the decision was considered "uncertain."We analyzed evaluation data for each priority item separately. 32

| RESULTS
We contacted 40 survivorship experts via email and received 24 (60.0%)replies.One panelist dropped due to unavailability, leaving 23 participants, which is an adequate size for a Delphi panel. 24,33All 23 experts participated in Rounds 0 and 1. Participation rates dropped in Round 2 (n = 15, 65.2%) and rose in Round 3 (n = 20, 87.0%), which F I G U R E 1 An illustration of generating a Round 1 Delphi item from expert interviews and Round 0 priority recommendations.

Step 1a: Team mined expert interviews to identify priority areas
Example of relevant interview extract: "I think just number one is being aware that there are -there is such a thing as late and long-term effects, right?So, I guess that falls into the bucket of educating myself and knowing that this is a real thing.Education needs to take the form of survivorship 101 education and talks and grand rounds and things like that to update providers about survivorship."

Step 1b: Team catalogued Delphi panelists' Round 0 priority items and noted items that conceptually overlapped with interviews
Example of Round 0 item from Delphi panelist that overlapped with interview extract: "Targeted educational material for PCPs on breast cancer survivorship including surveillance, potential late effects from treatment." Step 2: Team cross-checked interview extracts and Round 0 items to develop finalized items.This process entailed blending similar items, teasing out distinct items, and refining language for conciseness and clarity.
From this example, the team observed two distinct training items -one focused on late and long-term effects and one focused on surveillance -from cross-checking 1a and 1b that became final items for Round 1. Item 14. Train primary care clinicians on surveillance for patients with a history of breast cancer.
Item 16: Train primary care clinicians on common breast cancer treatments and their late and long term effects.
exceeds the expected 45%-50% participation rate. 33,34anelists who participated in only Round 3 (n = 5) were analyzed separately from those who participated in both rounds.We decided to allow these panelists to participate in Round 3 because they included three of the eight participating oncologists, a perspective we wanted to capture.Participant demographics are described in Table 1.
b Three participants provided no demographic information.
Panelists reviewed the 31 priority items in Round 1, and after ranking their top ten items, only two items remained unranked and were dropped from subsequent rounds: • Work with oncology to educate patients about the need to continue to see their primary care physician (strategy #30 in Table 2).• Implement lunch-and-learn sessions with staff and patients/caregivers about cancer survivorship (#31).
Table 2 shows panelists' assessments of the importance and feasibility of the remaining 29 items that were evaluated in Rounds 2 and 3.

| Consensus
Although panelists did not reach consensus for many items in the initial round of evaluations, by the end of Round 3, consensus was reached for most items.They did not reach consensus on feasibility for one item after completing the final evaluation round: Provide primary care clinicians blocked time in schedules dedicated for cancer survivors (strategy #28 in Table 2).Panelists also did not agree on the importance of three items: (1) Develop/research existing point of care resources for survivorship care (#22); (2) Implement a project ECHO (Extension for Community Healthcare Outcomes) on breast cancer survivorship for primary care clinicians (#24; project ECHO consists of a virtual learning community of primary care physicians, specialists, and staff who would join a series of 60-min Zoom sessions discussing real-life patient scenarios); and (3) Implement patient support groups/group visits for breast cancer survivors in primary care (#29).

| Most important, uncertain feasibility
Panelists agreed that three priority items were most important.These were the only items to be scored a 9 on the 1-9 scale following Round 3: • Define clear roles and responsibilities for both oncology and primary care for patients with a history of breast cancer (#1).• Train oncologists about how primary care can address survivorship care (#2).• Create clear pathways for communicating and sharing information with oncology (#3).
Notably, all three items prioritize the theme of "oncology-primary care connections," highlighting the need to clarify the roles and responsibilities of primary care and oncology and strengthen their communication and relationships.While panelists indicated improving primary care-oncology connections was especially important, they were uncertain about the feasibility of these items (scored 5 or 6).

| Important and feasible strategies
Panelists agreed that four priority items were both important and feasible (scored 7 or 8 on both criteria): • Educate patients on survivorship (#4).
• Enable screening reminders, monitoring alerts, and automated messaging within electronic health records (EHRs) (#5).• Identify available resources for primary care clinicians to recommend to patients/families (#6).• Develop and/or distribute accessible clinician reference guides detailing common breast cancer drugs and effects (#11).
These items highlight three of the five identified thematic groupings: "patient education," "technological solutions," and "resource development and distribution."

| Importance
Overall, panelists who participated in both rounds evaluated the importance of most items favorably, with only one item -Implement patient support groups/group visits for breast cancer survivors in primary care (#29)scored as not important, although panelists did not reach agreement on this (as noted above).For items where consensus was eventually reached, panelists evaluated 18 items as important and eight items as uncertain.After the final evaluation round, all five items to build connections between oncology and primary care (#s 1, 2, 3, 9, and 17) were evaluated as important.
Four of the seven "primary care practice trainings" (#s 10, 13, 14, and 16) were assessed as important and emphasized knowledge of breast cancer treatment and clinical guidelines (i.e., treatment effects, surveillance, screenings).Two of the five "technological solutions"-creating automated EHR reminders (#5) and standardizing cancer history information (#7)-were identified as important.Two of the four "patient education" items, aiming to improve patients' survivorship knowledge (#4) and their   Note: Numbers in parentheses: Median value, range 1-9 of panelists who participated in Round 3 only (n = 5) but did not provide independent assessments in Round 2; they were analyzed separately.* = panelists did not reach consensus based on the RAND/UCLA Appropriateness Method described in the methods section.ECHO = Extension for Community Healthcare Outcomes, which consists of a virtual learning community of primary care physicians, specialists, and staff who would join a series of 60-min Zoom sessions discussing real-life patient scenarios.Abbreviation: PCP, primary care provider.skills communicating with primary care clinicians (#8), were evaluated as important.Four of the six "resource development/distribution" items were deemed important and included resource lists for patients and families (#6), the development of clinician reference guides (#11) and risk-stratification guidelines (#15), and the inclusion of patient-reported outcomes on intake forms (#12).Only one of the four "system-level strategies" to provide coordinators to facilitate survivorship care (#18) was identified as important.Round 3 only panelists (n = 5) generally agreed with the larger panel's assessments of importance with a few exceptions.Three items assessed as important by the larger panel were evaluated by Round 3 only panelists as uncertain: • Identify available resources to recommend patients/families (#6).• Work with oncology to develop primary care-friendly plans (#9).• Work with oncology to engage primary during acute treatment (#17).
Two items, which the larger panel agreed were uncertain, were evaluated as important by Round 3 only panelists: Train primary care clinicians on how to use pointof-care resources to find information (#26) and develop a patient registry for cancer survivors in the EHR (#27).

| Feasibility
The larger panel agreed that the feasibility of most (21/29) items was uncertain.Panelists assessed only five items as feasible after Round 3. Four of these items were also evaluated as important (discussed above).One additional item was evaluated as feasible and of uncertain importance: Design primary care clinician-friendly templates for follow-up plans in EHR (#19).
Panelists also agreed that two priority items were not feasible: Provide managers/coordinators to facilitate survivorship care (#18) and Implement patient support groups/ group visits for breast cancer survivors (#29).Panelists did not positively evaluate the feasibility of any items that were thematically categorized as "primary care practice trainings" or "system-level solutions."Round 3 only panelists shared similar views to the larger panel regarding feasibility with four exceptions.Round 3 only panelists evaluated all four of the exceptions as more feasible than the larger panel: • Educate patients on survivorship care strategies and how to communicate care needs with primary care clinicians (#8).

| DISCUSSION
Overall, panelists reached consensus about the importance and feasibility of most priority items after Round 3 with a few exceptions.Panelists identified 18 of the 29 items as important, indicating the value of varied priority areas with which to consider tailoring to context and implementation.However, they were less optimistic about feasibility: following Round 3, they assessed 21 of the 29 items as uncertain.This may have stemmed from the fact that prompts were framed for panelists to elicit priority areas broadly rather than in a specific context.However, it may also reflect the inertia of translating primary care-engaged survivorship models into practice in the U.S. despite advancements of implementing these models in other countries. 35Previous critiques of strategies that have failed have re-directed survivorship experts to pay closer attention to context and processes rather than favoring static approaches (e.g., guideline development, survivorship care plans). 11,36This was the impetus for this study-to identify priority areas that can be tailored to context that support breast cancer survivorship care in primary care settings.Our findings regarding the widespread uncertainty about the feasibility of most priority items (including those most important) indicate that stakeholder engagement and capacity building may be needed to render important priority items actionable.
To that end, our findings reveal two important takeaways.First, primary care clinicians and oncologists have been found to hold divergent views when it comes to communication and their respective responsibilities for survivorship care, 37 illustrating the gulf that currently exists.This is not a problem unique to cancer, as primary care generalists and specialty care physicians tend to operate from different clinical orientations. 38To date, efforts to address the cancer survivorship care gap with primary care are characterized as "onco-centric," favoring a specialty care orientation that aims to: (1) identify disease for management; (2) interpret what needs to happen through specialized knowledge; and (3) develop and carry out a care plan. 38In contrast, the generalist orientation (1) recognizes problems and assesses capacity; (2) prioritizes attention and action to foster health and connection; and (3) personalizes care based on patient priorities. 38Panelists recommended a set of priorities that would shift both generalist and specialist perspectives by aiming to clarify roles for both groups, improve communication, and foster productive relationships between these groups.Among the 31 priority items evaluated, panelists agreed that these items were the most important (scoring 9 on importance).In contrast to many of the recommended priority areas, these items require cross-team collaboration and unbillable time, and thus, unsurprisingly, panelists were uncertain about their feasibility.These priority areas call for cross-discipline capacity building and shared understandings, which have been absent from previous cancer survivorship implementation efforts.Notably, these items ranked as most important do not emphasize static approaches, such as developing additional tools or disseminating guidelines.Additionally, they emphasize the need for knowledge translation that honors both specialist and generalist orientations rather than emphasize deficits in primary care.
Regarding the second important takeaway, four priority areas were evaluated as both important and feasible.Delphi items that are positively assessed along all criteria (in this case as both important and feasible) are often deemed "preferred." 39However, when considering how to effectively improve survivorship care, we approach these items with caution.These items recommend offloading responsibility to patients, who are already caught in the middle, having to coordinate the exchange of information between oncology and primary care; automating information and tasks; and providing ready-made resources that clinicians can access as reference tools that require less extensive investment than other educational recommendations (e.g., training clinicians on survivorship care guidelines).Although empowering patients and promoting patients' self-management are important goals, by intentionally shifting the responsibility to coordinate care in a fragmented delivery system to patients, these priority items may exacerbate disparities in cancer survivorship care, especially for patients with greater care complexity (due to social need or chronic conditions). 40Although these priorities may require relatively limited investment and collaboration, rendering them more "actionable," they may achieve limited long-term success given that they will simply saturate survivorship care with yet more tools that have not been well received.
Although panelists questioned the feasibility of how primary care-oncology connections might be fostered, investing in these priorities may have lasting impact.As experts who are directly involved in developing survivorship programs and delivering care to survivors, these panelists are well-positioned to consider the processes and structures needed to help tailor and implement tools and guidelines that have yet to be effectively adopted.Having identified panelists' priorities, consideration of how to tailor and implement them is a crucial next step.Additionally, mapping these priority areas onto the Expert Recommendations for Implementing Change (ERIC) framework will help operationalize them in practice. 41ffectively implementing these changes to improve shared understandings across primary care and oncology may be the foundation of many needed changes.This capacity building could have a ripple effect that would make the tailoring and implementation of other important priority areas more feasible.For example, with better communication and understanding of roles, standardizing cancer history information in the EHR and engaging primary care during acute treatment can be more easily managed.Perhaps most importantly, facilitating stronger relationships between primary care and oncology can encourage paradigm shifts for both clinician groups, further supporting the coordination and collaboration between primary care and oncology that is currently lacking but necessary for long-term sustainable change.
This study is not without limitations.Sampling experts currently working in and developing primary careengaged survivorship programs provides perspectives that may not resonate with on-the-ground primary care clinicians.As such, we do not know if their views generalize to primary care more broadly.Nevertheless, given their deep knowledge of cancer care and ongoing efforts to bridge to oncology, their expertise brings valuable understandings about what is required to effectively fill gaps primary care is currently facing when providing care to patients with a history of breast cancer.The sample size of the study was also quite small, limiting our understanding of the representativeness of these experts' perspectives.However, given that this specialized population is not large to begin with and that they reached consensus on many issues provide meaningful insights.The sample size is also consistent with other Delphi studies. 24,32Additionally, although we asked panelists to consider breast cancer survivorship care, what we learned from them spans the survivorship continuum and is likely relevant to other cancer care contexts.

T A B L E 2 1 Onc
Panelist evaluations of importance and feasibility.Define clear roles and responsibilities for both oncology and primary care for patients with a history of breast cancer clinicians on how to use point of care resources to find information on breast cancer survivorship oncology to educate patients about the need to continue to see their PCP Patient Education Unranked in Round 1 31 Implement lunch-and-learn sessions with staff and patients/caregivers about cancer survivorship System-level strategies Unranked in Round 1