Development of the Core Outcome Set for Psychiatric Pharmacists

The American Association of Psychiatric Pharmacists (AAPP) sought to develop a consensus‐based set of outcomes and measures to standardize psychiatric pharmacy practice and research. The measures in this Core Outcome Set for Psychiatric Pharmacists (COS‐PP) are designed to study the impact of psychiatric pharmacists caring for individuals living with psychiatric disorders.

consensus, health care, health care/standards, outcome assessment, pharmaceutical services, pharmacists/standards, quality indicators

| INTRODUCTION
In recent years, there has been a movement to better define and capture the impact clinicians have on patient outcomes.Organizations are seeking to define standards of care utilizing measurement-based strategies and other performance indicators. 1,2In psychiatry, there is still limited guidance, 3,4 especially for psychiatric pharmacists.
To examine the currently available data on the impact of psychiatric pharmacists, the American Association of Psychiatric Pharmacists (AAPP) conducted a literature review collecting patient care outcomes related to the quadruple health care aim of patient care, health care costs, patient experience, and clinician well-being. 5This literature review identified the major limitations of previously published studies regarding psychiatric pharmacist practice as variability in practice inconsistent outcome measures, and small sample populations lacking statistical significance. 5,6These limitations restrict pharmacists' ability to replicate promising outcomes.One way to consistently describe and quantify the contributions of psychiatric pharmacists is to leverage a core outcome.For this reason, AAPP created the Core Outcome Set Task Force (COSTF) to apply the Core Outcome Set (COS) development process to psychiatric pharmacy and standardize measurements of care across practice settings, with the goal of creating a clinically relevant practice blueprint promoting consistent documentation of value-added interventions.Subsequently this data could support expansion of services if found to be impactful to stakeholders such as employers, payers, and patients.The purpose of this paper is to summarize the work of the COSTF and document the Core Outcome Set for Psychiatric Pharmacists (COS-PP).Outcome Set-STAndards for Reporting (COS-STAR). 7,8The target population for the COS was defined as pharmacists caring for individuals of any age living with psychiatric disorders.The target intervention was the integration of a Board-Certified Psychiatric Pharmacist (BCPP) into the treatment team and was not dependent on the care setting.Potential outcomes and measures were derived from a literature review and a survey of BCPP stakeholders.A series of virtual summit meetings were subsequently held to finalize the COS and outcome measurement instruments (OMIs) through expert consensus.
The study protocol, including scripting of solicitation communications and summit meeting instructions, was approved by the Institutional Review Board (IRB) of the University of Cincinnati.

| Identify existing outcomes and measurement instruments
An initial list of outcomes was generated from interim results of a systematic literature review conducted by an AAPP workgroup; the complete methods are reported elsewhere. 9In brief, the review included The primary investigator grouped all potential outcomes, OMIs, and disease states from the information-gathering steps and excluded researcher-specific and non-validated instruments.Nineteen disease states were identified in the previously published literature examining psychiatric pharmacy practice and included: alcohol use disorder, anxiety disorders, attention-deficit/hyperactivity disorder, bipolar affective disorder, chronic pain, delirium, depression, developmental disorders, epilepsy, insomnia, neurocognitive disorders, nicotine use disorder, opioid use disorder, Parkinson's disease, posttraumatic stress disorder, schizophrenia spectrum disorders, substance use disorders, tardive dyskinesia, and medical or infectious disease.

| Elicit views about potential outcomes
Commentaries on potential outcomes were collected through an online survey hosted by REDCap.The survey was distributed to 631 eligible participants from August 1 to 31, 2022, via the AAPP automated email system.Survey participants were eligible if they maintained an active BCPP certification and had membership with AAPP for at least five consecutive years.The survey was distributed multiple times via email.
Respondents were asked to rate each potential outcome on a Likert Scale according to five factors, adapted from McBane and colleagues 10 : (1) comprehensive, (2) attributable to pharmacist intervention, (3) feasible, (4) scientifically sound, and ( 5) usable (see Table 1).These criteria were chosen to enhance the quality and clinical relevance of the final outcomes.Aggregate scores, expressed as means, from the 19 completed surveys were provided to the summit participants for their consideration of the final COS.There was an opportunity to provide free text response and propose additional outcomes.Respondents were not compensated for their participation.

| Consensus process to finalize core outcomes and measurement instruments
Summit participants were recruited in the same email as the survey with an invitation that included a consent to participate.Summit participation was restricted to BCPPs, and candidates were blinded for review by the COSTF.The COSTF aimed to cultivate a participant group representative of the multitude of BCPP practice settings, with inclusion of participants from varied regions of the United States and representation of different genders, races, and ethnicities.The same participant group was used throughout the summit meetings and participants were included even if they missed one or more sessions.
Virtual summit meetings deliberated and voted on psychiatric pharmacy outcomes and OMIs.Participants were provided with a packet outlining potential outcomes and OMIs along with aggregated data from the online survey.Outcomes were addressed first, and a COS finalized, followed by determination of OMIs.Consensus was attained when the number of participants voting "Yes" reached a specified threshold.The threshold was calculated as 60% of the number of participants in the first 15 min of each summit session, and it was not adjusted when people were unable to vote for any reason.Not all potential outcomes were discussed in the COS Summit.Participants could advance an outcome for discussion and voting, including outcomes not previously identified.The COS was considered final when no additional outcomes were proposed for review.Likewise, not all OMIs were discussed in the summit.A list of potential measures was displayed for each outcome and participants could propose additions and removals.
When no additional changes were advanced for consideration, the group voted on the final list according to the consensus rules.The number of summit sessions was determined by the progress toward finalizing the COS and OMIs; once the participants considered the results complete, the summit meetings were terminated.

| Summit participants
Twenty one eligible participants volunteered and 18 participated in the summit.Participant demographics are displayed in Table 2. Participant practice settings are displayed in Figure 1; participants could select all practice settings in which they spend 20% or more of their time working.

| COS and OMI consensus
Consensus for the final COS and OMIs was reached through a series of four virtual summit meetings in October 2022, totaling 10.5 h.The majority threshold for session 1 was eight participants (60% of 12 total participants), eight participants for session 2 (60% of 13 total participants), six for session 3 (60% of 10 total participants), and five for session 4 (60% of 9 total participants).The final COS was comprised of 21 outcomes and 84 OMIs, including variations for different disease states and drug classes.Seven outcomes in the final COS had no recommended OMI.The COSTF edited the COS and OMIs for clarity and consistency (see Table 3).The outcomes are ordered for convenience and list position is not indicative of importance.

| DISCUSSION
To the authors' knowledge, this paper is the first attempt to standardize core outcomes with associated measures for psychiatric pharmacy.
The extensive literature search ensured that the COSTF captured the breadth of possible outcomes, which were then narrowed down through a consensus process guided by the COSMIN initiative.In summary, 18 BCPPs examined 46 potential outcomes and 289 OMIs to develop a final COS for psychiatric pharmacy consisting of 21 outcomes and 84 OMIs.

| Limitations and future directions
The COS process is not without limitations.This present study was limited by a low response rate to the initial electronic survey to elicit views about important outcomes, but the impact of this limitation was partially mitigated by the extensive literature search completed prior to the survey.There were also a limited number of summit participants and not all participants could attend all summit sessions.Summit participant demographics seem to overrepresent women and the white population; these percentages are reflective of AAPP's membership.In future examination of the COS, ensuring racially diverse   As suggested by other authors, ongoing refinement of these outcomes and OMIs will need to integrate logistics of collection, reporting of metrics, workflow applicability, and electronic health record (EHR) capabilities. 1,12Additionally, the authors strove to emulate the characteristics of quality measurement, generating outcomes that are comprehensive, attributable to pharmacy, feasible, scientifically sound, and usable. 10rther research is needed to determine if the selected outcomes and OMIs meet these goals and are clinically applicable within the profession of psychiatric pharmacy.Continued evaluation of included disease states and medication classes as reflective of current practice is necessary, including consideration for symptomology across disease states, such as suicidality.The authors recognize that not all the listed outcomes and OMIs are feasible for pharmacists to measure or influence.
Furthermore, multiple outcomes have no measures, which indicates that the participants agreed with the importance of the outcome but were unable to reach consensus on a standardized measure.This is especially evident in the area of provider well-being, for which the participants could not identify any measures that had been used extensively related to the mental health team.Future research is needed in these areas to develop and standardize these measures for all disciplines.The hope is that this COS and associated OMIs will evolve as more research is conducted, reactions and observations are collected from a wider range of practicing psychiatric pharmacists, and perspectives of other stakeholders, such as psychiatrists, can be incorporated.
T A B L E 3 (Continued) In 2022, Nundy and colleagues proposed the addition of a fifth health care aim, health equity.The quintuple aim reinforces the importance of quality improvement efforts incorporating disparity reduction.Going forward, integration of outcomes focused on this fifth aim into the COS is imperative to reduce health disparities and minimize unintentional worsening of disparities when addressing other aims. 13COS-PP implementation guide with measure-related calculations, potential scales, updated research, and additional guidance will be maintained at aapp.fyi/cos-pp to operationalize and disseminate this research.

| CONCLUSIONS
The COS-PP and associated OMIs represent the consensus opinion of a panel of experts from a variety of psychiatric pharmacy practice settings.It will help define psychiatric pharmacists' support of the interdisciplinary team in achieving the quadruple aims in health care.Standardization of outcomes and measures across practice settings and institutions allows collaboration and publications that better clarify the contributions of psychiatric pharmacists and allows aggregation and replication of positive results.

2 | METHODS 2 . 1 |
Study overview COS-PP was developed using guidelines set forth by the Core Outcome Measures in Effectiveness Trials Initiative (COMET), promoted by the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) initiative, and formalized in Core digitally available posters presented at AAPP Annual Meetings from 2007 to 2022 and articles on pharmacy practice published between 1965 and 2021.To locate articles indexed in PubMed as of April 27, 2022 pertaining to psychiatric pharmacy outcomes, three pharmacyrelated search terms were used in tandem with 95 psychiatric terms, including disease states and medication classes.Manual review applied six exclusion criteria to ensure that resources had patient-level outcomes, relevance to psychiatry and neurology, and pharmacist interventions.The review provided 193 articles and 40 posters to COSTF for evaluation.COSTF reviewed guidelines, white papers, and journal articles not assessed by the literature review for additional outcomes and measures.

Figure 2
Figure 2 delineates the progress of the COS from information gathering to final consensus.From the 234 resources included in the systematic literature review, the COSTF extracted 554 initial outcome/ measurement instrument combinations.An additional 32 resources were reviewed to fill gaps in knowledge, increasing the number of raw

F I G U R E 1
Practice settings and patient populations of summit participants.F I G U R E 2 Core Outcome Set For Psychiatric Pharmacists (COS-PP) development process.T A B L E 3 Core Outcome Set For Psychiatric Pharmacists (COS-PP).

Outcome 5 .Outcome 8 .
Improved progress toward treatment goals Global measures 5.1 Patients with medication response based on a disease-specific validated rating scale 5.2 Patients in remission based on a disease-specific validated rating scale 5.3 Patients with a clinically significant improvement on a global validated rating scale 5.4 Patient reported symptom-free days Alcohol use disorder (AUD) 5.5Utilize the global measures and consider using the timeline followback (TLFB) method to assess 5.7per month (For men, consuming more than 4 drinks on any day or more than 14 drinks per week; For women, consuming more than 3 drinks on any day or more than 7 drinks per week) 5.9 Days to first drink 5.10 Patients with undetectable blood alcohol levels (BAL) Antipsychotic-induced movement disorder 5.11 Utilize the global measures in conjunction with antipsychotic-induced movement disorder-specific validated rating scales 5.12 Anti-extrapyramidal symptoms (EPS) interventions Anxiety disorders 5.13 Utilize the global measures in conjunction with anxiety-specific validated rating scales Attention-deficit/hyperactivity disorder (ADHD) 5.14 Utilize the global measures in conjunction with ADHD-specific validated rating scales Bipolar affective disorder 5.15 Utilize the global measures in conjunction with bipolar affective disorder-specific validated rating scales 5.16 Number of episodes per 6-month period Chronic pain 5.17 Utilize the global measures in conjunction with chronic pain-specific validated rating scales 5.18 Daily prescribed morphine milligram equivalents (MME) Delirium 5.19 Utilize the global measures in conjunction with delirium-specific validated rating scales 5.20 Days to resolution of delirium Depression 5.21 Utilize the global measures in conjunction with depression-specific validated rating scales the global measures in conjunction with developmental disorder-specific validated rating scales Epilepsy 5.23 Utilize the global measures in conjunction with epilepsy-specific validated rating scales 5.24 Seizures per specified time period Insomnia 5.25 Utilize the global measures in conjunction with insomnia-specific validated rating scales Utilize the global measures in conjunction with neurocognitive disorder-specific validated rating scales 5.33 As needed (PRN) medication administrations for acute agitation per specified time period with reduced non-prescribed opioid use in a specified time period Parkinson's disease 5.41 Utilize the global measures in conjunction with Parkinson disease-specific validated rating scales 5.42 Number of off episodes per day 5.43 Time spent in off episodes per day 5.44 Time in dyskinesia per day Posttraumatic stress disorder (PTSD) 5.45 Utilize the global measures in conjunction with PTSD-specific validated rating scales Schizophrenia spectrum disorders 5.46 Utilize the global measures in conjunction with schizophrenia spectrum-specific validated rating scales Outcome 6. Decreased all-cause mortality Decreased length of stay 8.1 Hospital days per patient Outcome 9. Improved formulary management No recommended measure.Further research is encouraged.perspectives are captured is important.The participants over the four sessions represented the majority of the wide psychiatric pharmacy practice settings and patient populations served that the authors sought to represent when designing this study.The practice settings not represented were long term care and managed care, which together represent less than 6% of the psychiatric pharmacist community. 11 Outcome 16.Improved patient perceptions of care No recommended measure.Further research is encouraged.Outcome 19.Improved collaboration and respect among the health care team members No recommended measure.Further research is encouraged.Outcome 20.Increased care team utilization of pharmacists for drug information, education, and mentorship No recommended measure.Further research is encouraged.Outcome 21.Increased workplace engagement (e.g., leadership, committee service, program development) No recommended measure.Further research is encouraged.
Outcome 17. Improved patient medication access No recommended measure.Further research is encouraged.Aim: Provider well-being Outcome 18. Reduced care team burnout with clinical pharmacy support No recommended measure.Further research is encouraged.