Patient engagement in the process of planning and designing outpatient care improvements at the Veterans Administration Health‐care System: findings from an online expert panel

Abstract Context There is a strong interest in the Veterans Administration (VA) Health‐care System in promoting patient engagement to improve patient care. Methods We solicited expert opinion using an online expert panel system with a modified Delphi structure called ExpertLens™. Experts reviewed, rated and discussed eight scenarios, representing four patient engagement roles in designing and improving VA outpatient care (consultant, implementation advisor, equal stakeholder and lead stakeholder) and two VA levels (local and regional). Rating criteria included desirability, feasibility, patient ability, physician/staff acceptance and impact on patient‐centredness and care quality. Data were analysed using the RAND/UCLA Appropriateness Method for determining consensus. Findings Experts rated consulting with patients at the local level as the most desirable and feasible patient engagement approach. Engagement at the local level was considered more desirable than engagement at the regional level. Being an equal stakeholder at the local level received the highest ratings on the patient‐centredness and health‐care quality criteria. Conclusions Our findings illustrate expert opinion about different approaches to patient engagement and highlight the benefits and challenges posed by each. Although experts rated local consultations with patients on an as‐needed basis as most desirable and feasible, they rated being an equal stakeholder at the local level as having the highest potential impact on patient‐centredness and care quality. This result highlights a perceived discrepancy between what is most desirable and what is potentially most effective, but suggests that routine local engagement of patients as equal stakeholders may be a desirable first step for promoting high‐quality, patient‐centred care.


Introduction
The term 'patient-centred care' typically refers to medical care focused on the patient and his/her health-care needs. 1 Achieving patientcentred care calls for active patient engagement, often defined as patients working in close partnership with care professionals. 2 To achieve patient-centred care, health-care systems, including the Veterans Health Administration (VA), encourage patients to engage in making decisions about their own care. In addition, these systems often aim to engage patients in improving quality and safety of health-care services 3 and in shaping health-care policy. [4][5][6][7] For example, patients have been engaged through surveys, 3 sharing their perspectives during focus groups 8 and joining patient advisory boards. 9 Yet few investigations have assessed the desirability, feasibility or impacts of engaging patients more deeply in routine organizational improvement and policy activities.
Research suggests that patient and stakeholder engagement may increase health-care research and policy relevance to community needs, 10 shape priorities for improving or redesigning care delivery, 11,12 improve care quality and patient safety, 13,14 reduce disparities in care access and outcomes 15 and improve opportunities for achieving community and policy impacts. 16 A recent systematic scoping review of empirical evidence on outcomes of public involvement in health-care policy, however, did not find conclusive evidence of impacts on care, while acknowledging that engagement may have intrinsic value. 17 Although we are not aware of a systematic review specifically directed at evaluating patient and public engagement in care planning and design, there is a strong interest in this topic and its relationships to care quality and patient safety at local (e.g. clinic or hospital) and/or regional (e.g. health-care system) levels. 6, 18 Moreover, existing literature on patient engagement in care planning and design suggests that patient input could inform quality improvement initiatives, but not always in a standardized way; 18 that there is often a gap between intentions to involve patients and the actual engagement of patients; 19 and that patient engagement may be most effective in improving information that is made available to patients, facilitating access to care services and improving overall care environment. 20 In this paper, we use a conceptual model of patient engagement 2 to explore experts' opinions about engaging patients in planning and designing outpatient care improvements at the VA. Using an online expert panel methodology, we asked a diverse and purposefully selected national group of experts (n = 48) with experience in patient engagement to rate desirability, feasibility, stakeholder acceptance and potential outcomes of eight scenarios describing four patient engagement roles (consultant, implementation advisor, equal stakeholder and lead stakeholder) and two health-care system levels (local and regional).

Project context
The VA is the largest integrated health-care delivery system in the United States with nearly 1400 sites, organized into 21 Veterans Integrated Service Networks, or VISNsregional systems of care. Serving 8.3 million Veterans, VA offers a full spectrum of inpatient, outpatient and long-term care services 21 and provides a unique opportunity to explore patient engagement at local and regional levels. While advancing the delivery of patient-centred primary care has been a VA goal over more than a decade, patient engagement is a more recent core organizational principle among VA's strategic goals. Empowering Veterans to improve their well-being, for example, is highlighted as a core goal in the Blueprint for Excellence, a vision statement that provides guidance for improving the care delivery process. 22 Every year, VA fields the Survey of Health Experiences of Patients (SHEP) to evaluate patient health-care experiences, with topics ranging from health-care access to provider interpersonal skills. 3 The VA has numerous ongoing initiatives that enlist patients and family members in planning and designing care at local hospital and VISN levels. For example, similar to some Federally Qualified Health Centers (FQHCs) 23 and integrated not-for-profit healthcare providers, 24 each VA facility has Patient Advisory Councils (PACs) that meet regularly and suggest ways of improving services. Finally, besides engaging individual Veterans and their family members, the VA operates a Patient Advocacy Programme in all facilities. This programme aims to improve the health-care system to better meet patient expectations and works with governmental agencies at the local, state and federal level and non-governmental groups (e.g. Veterans service organizations) to obtain services and benefits for eligible Veterans. 25 Although VA engages patients in a number of ways, there is no consensus on the degree to which patients should be involved in improving the VA health-care system, nor which healthcare settings or organizational levels are best suited for patient engagement in health system planning. Moreover, despite the availability of patient viewpoints, there is little evidence that Veteran patients or families are systematically involved in routine regional and local decision making around on-going outpatient care priorities, improvements or policies.

Theoretical framework
Our project is informed by Carman et al.'s patient and family engagement framework, 2 which suggests that patients play three roles. At the lower end of the engagement continuum, patient input is collected, but patients have no decision-making authority. Their role is to provide information about their preferences and opinions. Patients can become more involved by serving in an advisory role, for example, by joining patient advisory boards. At the upper end of the engagement continuum, patients engage equally in the decision making by serving in the same roles as other stakeholders on decisionmaking bodies. This framework also posits that engagement can occur at three levels across the health-care system: direct care, organizational decision making at the local level and policy decision making at the regional or national level.
After consulting with VA patient representatives and patient engagement specialists, we modified this framework to better fit the VA context (see Table 1). Specifically, instead of 'patient involvement', we conceptualize patients as implementation advisors who directly affect the implementation of care delivery changes or quality improvement initiatives. We added a 'patient leadership' role to account for situations prioritizing patients' input over other stakeholders' perspectives. 26 Because of the project's focus on care planning and design, we focused only on organizational decision making (e.g. care design and practice improvement) at the local (outpatient medical facility) and regional (VISN) VA levels. Although engaging patients at the direct care level is highly important, 27 we did not include it because, unlike engagement in design decision making at local and regional levels, a vast literature about engagement in direct care already exists. Furthermore, including it would have increased the burden on panel participants.

Methods
We designed an exploratory project to identify areas of agreement and disagreement among experts on four roles and two levels of patient engagement. Each role-level combination represents one patient engagement scenario. We note that by 'patient engagement', we refer to the mode of collecting input from patients as well as the way that input is used.
This project addressed the following questions:

Participants
We used our professional networks and a snowball sampling approach 28 to recruit a diverse, purposeful sample of US-based experts on patient engagement. We reached out to individuals who have practical experience in involving patients in outpatient health-care quality and care design decisions both within and outside of VA, who studied patient engagement or who served as patient representatives themselves. First, we contacted the directors of all VA health services and implementation research centres, as well as consultants and patient engagement researchers outside of the VA with whom we had previously collaborated. We also invited Veterans who volunteered with our local VA facility's PAC. We invited these individuals to participate and/or nominate someone to participate in the expert panel. To ensure adherence to our human subjects review, requiring that only patients with recognized expertise be invited, non-employee patients were recruited only if they were members of an official VA body, such as the PAC. Interested individuals were asked to register for the three-round panel by providing their demographic information. Of 59 registered experts, 48 (81%), who all reported familiarity with the VA context, participated in at least one rating round of this online panel (see below for round descriptions). Rating results described in the paper are based on analysis of the input collected from those experts who provided their final ratings (n = 28; 58% of 48 participating experts). We found no statistically significant difference between experts participating in Rounds One and Three except for on gender: men were less likely to participate in Round Three. Research suggests that the online format allows for engaging experts while avoiding the expense and inconvenience of travel to a centralized location and coordinating busy schedules. 29 Based on research demonstrating that anonymity may increase participants' readiness to be more honest and to evaluate the perceived value of each other's arguments, participation in our panel was completely anonymous. 30 We used ExpertLens TMan online previously evaluated system that combines two rounds of questions with a round of statistical feedback and asynchronous, anonymous discussion. 29 We chose ExpertLens because it allows for iterative engagement of large, diverse and geographically distributed groups of experts; combining quantitative and qualitative data; and exploring group agreement and disagreement. 30,31 ExpertLens has been used successfully in studies on different topics, including developing national suicide prevention research goals, 32,33 identifying definitional features of continuous quality improvement in health care, 29,34 developing quality and performance indicators/measures for patients with arthritis [35][36][37] and exploring relevance of ethical principles of communityengaged research in translational science. 38 In Round One, experts reviewed and rated eight patient engagement scenarios (see Appendix 1). We chose to use scenarios to generate fruitful discussions among participants, reduce the possibility of providing socially desirable answers and engage participants around potentially difficult topics. 39,40 Based on our conceptual framework, the first four scenarios described patient engagement at local VA outpatient care facilities; the last four described patient engagement at the VISN (regional) level. Patient engagement at each level was illustrated with four variations of the same example that highlighted differences in patients' roles, which ranged from consultation to patient leadership. The role of consultant was operationalized as soliciting patient input on care planning and design decisions at VA on an as-needed basis (e.g. through surveys, focus groups, advisory council meetings). The role of implementation advisor was described as engagement in the implementation of changes in care delivery based on patients' input and care preferences. The equal stakeholder role referred to situations where the input of patients is valued equally to the input of other stakeholders, whereas the role of lead stakeholder described situations where patients' input was prioritized.
Participants used 9-point Likert scales to rate each scenario on six criteria, namely feasibility, patient input, physician/staff acceptance, patient-centredness, impact on health-care quality and overall desirability (see Table 2). Overall desirability was a summary criterion capturing the expert's considerations about pursuing implementation of the scenario's approach, including consideration of other five criteria.
We developed the rating criteria based on the results of a recent study that identified different barriers to widespread patient engagement in practice improvement. 5 We tailored these descriptions for each scenario to draw participants' attention to differences between scenarios, while keeping the same labels for each criterion. Opentext boxes after each question allowed participants to explain their ratings and note the factors that most affected their answers.
In Round Two, experts reviewed bar charts showing each participant his/her own response (see Fig. 1) in relation to the distribution of Round One responses. They also interacted with other participants using an anonymous discussion board. Experienced discussion moderators encouraged dialogue by asking participants to explain their perspective on patient engagement, identify barriers and offer suggestions on how these barriers could be overcome.
Finally, in Round Three, participants revised Round One responses in consideration of Round Two feedback and discussion.
RAND Corporation's Human Subjects Protection Committee and the VA Greater Los Angeles Healthcare System's IRB determined this project to be non-research/quality improvement.

Data analysis
To determine the final panel decision, we applied the two-step consensus determination technique described in the RAND/UCLA Appropriateness Method User's Manual 41,42 to Round Three data. In the first step, we determined the existence of disagreement among participants by calculating the value of the interpercentile range (IPR), or the range of responses between the 70th and the 30th percentiles. We then calcu-lated the value of the interpercentile range adjusted for symmetry (IPRAS), a measure of dispersion for asymmetric distributions, and compared the values of IPR and IPRAS (see Table S1). If IPR>IPRAS, we concluded that disagreement existed among experts, indicating an uncertain group decision due to disagreement. If IPR≤IPRAS, we concluded that there was no disagreement and moved on to step two determining the group decision. Group decisions could be positive, negative or uncertain Feasibility -How feasible is it to implement changes in care delivery processes at VA outpatient clinics or hospitals based on patients' input and care preferences? [1: very unfeasible; 9: very feasible] Patient ability -How likely is it that patients will have the interest and skills necessary for providing input on how changes in care delivery processes should be implemented at VA outpatient clinics or hospitals? [1: very unlikely; 9: very likely] Physician/staff acceptability -How likely is it that VA physicians/staff will accept that the patients' perspectives affect the implementation of changes in care delivery processes at VA outpatient clinics or hospitals? [1: very unlikely; 9: very likely] Patient-centredness -How likely is it that using patients' input to implement changes in care delivery processes at VA outpatient clinics or hospitals will improve patient-centredness of VA care? [1: very unlikely; 9: very likely] Health-care quality -How likely is it that using patients' input to implement changes in care delivery processes at VA outpatient clinics or hospitals will improve care quality? [1: very unlikely; 9: very likely] Overall desirability -Considering all of the issues discussed above, how desirable is it to use patients' input to implement changes in care delivery processes at VA outpatient clinics or hospitals? [1: very undesirable; 9: very desirable] without disagreement; they were determined based on median scores. A median score of 7-9 indicated a positive decision (e.g. a scenario was considered desirable, feasible); a median of 1-3 indicated a negative decision (e.g. a scenario was undesirable, not feasible); and a median of 4-6 indicated an uncertain decision without disagreement.
To answer our project questions, we determined group decisions for each question and rank-ordered patient engagement scenarios on each rating criterion based on the group median responses. We then pooled rankordered data across scenarios at the local and regional levels, regardless of patient role, to determine whether patient engagement at the VISN level is more or less desirable than engagement at the local level. Similarly, we pooled rank-ordered data across patient roles to explore which role(s) participants consider most desirable, regardless of engagement level.
Finally, we analysed all qualitative data thematically to identify reasons for high or low scenario ratings. We grouped all explanations of numeric responses based on the numeric rating they referred to and linked all discussion board posts to a patient engagement scenario and rating criterion they described. We then coded all text inductively to identify the most prominent themes and those that span across patient engagement scenarios within the same level of engagement or the same patient role. Coded text was reviewed by two team members to ensure coding consistency; disagreements were discussed until consensus was achieved. 34

Scenario ratings
Rating data (see Table 4) revealed no significant disagreement among participants; however, none of the patient engagement scenarios received positive median scores (7-9) on all criteria. Two scenarios (S4 a and S7) were rated as uncertain (median scores 4-6) on all criteria, and one scenario (S8) received only uncertain or negative ratings (median 1-3).
Experts agreed that consulting with patients at the outpatient facility level (S1) is most desirable (median = 9). This scenario also had the highest median values on three other rating criteria, two of which met our standards for a positive group decision. Panellists agreed that soliciting patients' input on care planning and design decisions on an as-needed basis at VA outpatient clinics or hospitals, which is how S1 was described to participants, was highly feasible (median = 8) and that patients would have interest and skills necessary for providing their input (median = 7). Although this scenario had the highest median rating on the physician/staff acceptance criterion, the median score of six indicated uncertainty among the experts about physicians' acceptance. Similarly, panellists were uncertain about the impact of local consultaa To facilitate the presentation of findings, we used S1 to refer to Scenario 1, S2 to refer to Scenario 2, etc. See Appendix 1 for scenario descriptions. tions on health-care quality (median = 6). Experts, however, agreed that these consultations are likely to improve patient-centredness of VA care (median = 7). The second most highly rated scenario on the overall desirability criterion described patients as local implementation advisors (S2) (median = 8). This approach had ratings similar to local-level patient consultation (S1), with two exceptions: its median rating on feasibility was slightly lower (7), and its median rating on patient-centredness was slightly higher (7.5).
Engaging patients as equal stakeholders at the local level (S3) was also considered desirable (median = 7) and received the highest median rating on the patient-centredness (median = 8) and health-care quality criteria (median = 7). However, experts were uncertain about its feasibility (median = 6), the interest and ability of patients to provide input (median = 6) and physician/staff acceptance of patients as equal partners (median = 5). Patient leadership at the local level (S4) received uncertain ratings on all criteria, with median ratings ranging from 4 to 6.
Similar to S1 (local-level consultations), panellists' ratings indicated that regional  consultations (S5) were desirable (median = 7) and likely to improve patient-centredness (median = 7). While experts agreed that it was feasible for patients to be consultants at the regional level (median = 7), they were uncertain about the impact on health-care quality (median = 6), as well as patient willingness (median = 6) and physician/staff acceptance (median = 5.5). Experts also agreed that engaging patients as implementation advisors at the regional level (S6) was desirable (median = 7) and likely to have a positive impact on the level of patientcentredness (median = 7) and health-care quality (median = 7), but this scenario received uncertain ratings on all other criteria.
S7 describing patients as equal stakeholders at the regional level received uncertain ratings on all criteria, with median ratings ranging from 5 to 6. Similar to S4 and S7, S8 (patient leadership at the regional level) received uncertain ratings on all criteria with one notable exception: it received a negative rating on physician/staff acceptance (median = 3). Table 5 contains the desirability rankings for each scenario in relation to all others. Numbers presented in this table are average rankings across all participants for scenarios, with row totals aggregated across patient roles and column totals aggregated across health-care system levels. A lower mean rank indicates a more desirable response. Rankings indicate that local-level patient engagement was more desirable than regionallevel patient engagement (mean rank of 3.85 across all patient roles at the local level vs. 5.03 at the regional level). They also show that lessengaged patient roles were rated as more desirable. Nonetheless, although the role of a consultant received the most favourable rank (mean rank of 3.2), participants believed that the roles of implementation advisor and equal partner (mean ranks of 3.59 and 4.36, respectively) would have the highest impact on health-care quality (data not shown).

Thematic analysis of participants' comments
In general, participants' comments showed that they had very positive opinions about patient engagement in the design and planning of VA outpatient care. They felt that soliciting patients' input is 'essential to the survival of the system'. As one expert put it, 'if you want our Veterans to stay, they MUST [sic] participate. They MUST [sic] feel it is their system. And that way they will be loyal and continue to be a partner in the process of improvement planning'. Several participants noted that 'patients are the customers', 'they are the ones we want to please' and they are the ones who can identify 'where glitches in the delivery of care exist'. 'Without patient input', noted another expert, 'care planning becomes one-sided'.
Although generally positive about patient engagement, discussion comments provided insight into experts' ratings regarding specific patient engagement approaches. Experts voiced concerns about the feasibility and physician/staff support of even the least intensive approaches to patient engagement (e.g. S1), given the existing organizational structure and culture of VA. As one expert summarized his/her perspective to patients as consultants, 'there is a potential for feasibility, but currently there are too many barriers for this to happen. The situation is also While some experts felt that it was impossible for patients to be equal partners, others believed that Veteran patients' input should be valued equally to that of other stakeholders. Some indicated that facility or VISN leadership may not be oriented towards patient engagement and argued for a paradigm shift in the organizational culture of VA to consider patients as equal partners in outpatient care planning and design decision making. Similarly, panellists were concerned about physician/staff acceptance of patients as equal partners, citing 'the level of cynicism directed towards patients'. Some experts argued that patients' input could be as influential as that of other stakeholders only if top leadership insisted on it and if providers and staff were exposed to a collaborative approach to making decisions.
Likewise, experts expressed doubts about patients having a more influential voice than other stakeholders in outpatient care planning and design decisions. Indeed, this role received either uncertain or negative ratings on all criteria at the local and regional levels (see Table 4). Experts reasoned that physicians and staff may not be ready to accept patients as more powerful than other stakeholders. While some experts had reservations about patients' knowledge regarding the logistics of implementing care design decisions or in-depth understanding of how VA operates, others expressed concern that VA staff and physicians may feel unappreciated, offended and forced to participate 'in a manner they do not believe in'. Several questioned the representativeness of Veteran patients chosen to voice patients' perspectives and the ability of representatives to remain unbiased.
Discussion comments also provided further insight into the desirability and potential impact of engagement at the local vs. regional levels. As one expert explained, 'patients may more readily provide input on ways to improve practices at the local level because they have a concrete point of reference that has immediate relevance to them; by contrast, asking for input at the VISN level may feel abstract and less compelling'. The rating data also revealed that three of the five desirable patient engagement scenarios described engagement at the local level. While participants felt that local-level engagement is desirable, some commented that engagement at the regional level is likely to have a greater impact because 'more systems are involved [at the VISN level] and [there are] greater implications for action or inaction'. Other experts defended the importance of local engagement, suggesting that engaging patients in local clinics is a building block for engagement at the regional level. 'Local input is always important and can be channeled to VISN level', said one expert. 'Utilizing the local VSOs [Veteran Service Organizations] is always a win', stated another. 'This is truly LOCAL [sic] input affecting the LOCAL VHA [sic]. Do this in each VISN supervised hospital to gather consensus info for VISN level action. Win/win'. One strategy of bridging the local-regional divide suggested by experts was to 'gather [patient] input at the source of care, but topics should include those that will impact VISN policy decisions'.

Discussion
Our project was designed to explore expert opinion about the desirability, feasibility, stakeholder acceptance and potential outcomes of different roles and levels of engaging VA patients in outpatient care design decision making. Our findings show that experts agreed that soliciting patient input as consultants on care planning and design decisions on an as-needed basis, including through surveys, focus groups and advisory councils, is currently the most desirable approach to promoting patient influence on VA outpatient care. They also agreed that this approach is feasible, that patients have the interests and skills to contribute as consultants at the local level and that engaging patients this way will have a positive impact on patientcentredness of VA care.
Our finding about the overall perceived desirability of engaging patients as consultants is consistent with the findings of a recent study on public involvement in health-care decision making, suggesting that the public generally favours the role of a consultant who provides input, but is not responsible for making the ultimate decision or its consequences. 43 We hypothesize, based partly on participants' discussion comments and existing literature, 18 that the overall desirability of this patient engagement approach may be due to its 'low-intensity', relative familiarity to both patients and providers, and Veterans' willingness to share their care experiences in this manner.
While deemed desirable, the role of a consultant at the local level received uncertain ratings on physician/staff acceptance and health-care quality. Research conducted in other settings suggests that despite declaring their interest in patient satisfaction surveys, health-care staff often do not sufficiently discuss survey results, which limits their use in practice and their potential impact on initiating organizational change. 44 Therefore, it is important to ensure that healthcare professionals recognize the value of patients' experiential knowledge. 20 Experts also agreed that equal partnership between patients and providers at the local level is most likely to affect the level of patientcentredness and quality of VA care. Although local equal partnerships did not receive the highest rating on the overall desirability criterion, the positive ratings this approach received on patient-centredness and care quality suggest that building equal partnerships with patients might be an aspirational goal for VA and other healthcare systems in their attempts to improve patient care. This finding is consistent with a recent recommendation for designing patient engagement approaches that value and give equal weight to contributions of every stakeholder. 18 Our results also illustrate a contradiction between perceived effectiveness and feasibility of more active patient engagement. Although participating experts generally felt that higher levels of engagement may be more effective, more engaged patient roles were considered to be less feasible. Experts' uncertainty about the feasibility, patient willingness/interest and physician/ staff acceptance of patients as equal stakeholders suggests that health-care systems interested in patient engagement may need to build patients' capacity and educate physician/staff on how to work in partnership with patients. Recent experimental research on community representation also suggests that it is important to develop on-going, collaborative and constructive relationships among patients and providers that can change providers' attitudes towards patient engagement. 45 In general, patient engagement at the local level received higher desirability and feasibility ratings than patient engagement at the regional level. This finding seems consistent with the result of a UK-based study showing higher perceived value for engaging patients in practical, operational issues that are likely to generate results in the short term, compared to engaging them in larger-scale strategic decision making. 20 Indeed, qualitative data in this project revealed that patients may perceive they have more interest and skills to contribute at the local level. Patients may be more comfortable with the local level because they perceive that their input is not tokenistic, a known concern about patient engagement. 46 A patient's input might also be easier to apply effectively if it is specific to the place he or she receives care. 47 Finally, patients may be able to directly observe the impact of their engagement in a facility they visit, and thus experience greater rewards from participating. 48 Experts' ratings of patients as local consultants on all criteria were very similar to their ratings of patients as local implementation advisors. However, the latter was rated slightly less feasible and slightly more patient-centred than the former. The endorsement of more than one patient engagement approach suggests that patients and health-care systems can choose an approach that they are most comfortable with 18 or for which they are most ready.

Limitations
Although innovative and timely, this exploratory project has important limitations. First, as is typical for expert panels, 49 our participants were not a representative sample of all relevant stakeholders. Therefore, results of this project may be biased towards a more positive view of patient engagement. Second, our final sample consisted of Round Three ratings provided by only 28 experts. Although potentially biased (e.g. it may include only those who liked the online format), this sample is much larger than a sample of nine participants used in traditional expert panels. 41

Lessons learned
Based on our interpretation of expert opinion collected in this panel, we suggest five lessons learned about patient engagement. 1. Experts agreed that engaging patients as consultants and implementation advisors is relatively feasible, particularly at the local level, and highly desirable. Therefore, VA leaders may want to develop approaches for routinely soliciting patient input on care planning and design decisions. 2. Engaging patients at the local level may be a crucial step towards broader engagement at the regional level because patients are better able to imagine the possible impact and see the actual outcomes of their engagement in their local facility.

Scenario 4. Local level: patient leadership
Patients' input in care planning and design decisions in VA clinics or hospitals is more influential than the input of other stakeholders Example: During a previously described local Quality Council meeting convened to prioritize ways to improve care quality in a VA hospital, providers felt strongly about focusing on provider continuity, whereas patient representatives insisted on improving the parking situation. Because parking was the top patient priority identified in the patient satisfaction survey (described in Scenario 2), the Quality Council decided to increase the number of patient parking spaces instead of working to improve continuity.

Scenario 5. Regional level: consultation
Patients' input on care planning and design decisions in Veterans Integrated Service Networks (VISNs) is solicited on an as-needed basis Example: A VISN located in a predominantly rural area surveyed its patients to determine their satisfaction and solicit care improvement suggestions. VISN leadership was briefed on survey findings.
Scenario 6. Regional level: implementation advisor Patients' care preferences affect the way changes in care delivery processes are implemented at the VISN level Example: Results of the VISN-wide patient satisfaction survey identified the following three patient care improvement priorities that were deemed to be equally important by participants: Complementary and alternative medicine therapies; Telehealth services; Improvements in the operation of a regional call centre.
To improve patients' ability to reach clinical services, VISN leadership made a decision to institute performance goals related to timeliness of response by the call centre staff. Veteran representatives were consulted to determine performance targets. Based on their feedback, a 2min performance goal was implemented. Scenario 7. Regional level: equal partnership est in using the following to improve anxiety and depression symptoms: Hyperbaric oxygen therapy (a treatment in which a patient is placed in a chamber and breathes oxygen at higher than atmospheric pressure) Reiki (a form of treatment based on the belief that there is a universal healing energy, the Reiki, which is channelled through the practitioner to the patient to support the patient's body's healing abilities) Herbal supplements (plant-derived compounds used for medicinal purposes) VISN leadership formed an Advisory Council for Mental Health Services, which included VA providers, administrators and patient representatives, to offer advice on possible introduction of complementary and alternative medicine (CAM) treatment modalities. Although Veteran patient representatives favoured hyperbaric oxygen therapy, providers had strong objections based on the lack of empirical evidence for the use of hyperbaric oxygen therapy to treat anxiety and depression. The Advisory Council, however, agreed to make a recommendation to hire a naturopathic physician and promote the use of herbal supplements.

Scenario 8. Regional level: patient leadership
Patients' input on care planning and design decisions at the VISN level is more influential than the input of other stakeholders Example: In discussing different complementary and alternative medicine (CAM) options for treating anxiety and depression, VA providers on the Advisory Council for Mental Health Services expressed their strong reservations about the effectiveness of different non-traditional modalities, including Reiki and relaxation therapies. They cited inconclusive evidence of the effectiveness of these treatment modalities and raised concerns about the impact they may have on care quality. Nonetheless, VISN leadership initiated a pilot programme at one outpatient clinic to test Reiki and relaxation therapies based on the overwhelmingly positive patient interest in these treatment options identified by the patient satisfaction survey (described in Scenario 6).