Co‐designing for quality: Creating a user‐driven tool to improve quality in youth mental health services

Abstract Background Although high quality mental health care for children and youth is a goal of many health systems, little is known about the dimensions of quality mental health care from users’ perspectives. We engaged young people, caregivers and service providers to share experiences, which shed light on quality dimensions for youth mental health care. Methods Using experience‐based co‐design, we collected qualitative data from young people aged 16‐24 with a mental disorder (n = 19), identified caregivers (n = 12) and service providers (n = 14) about their experiences with respect to youth mental health services. Experience data were collected using multiple approaches including interviews, a suite of online and smartphone applications (n = 22), and a co‐design event (n = 16) and analysed to extract touch points. These touch points were used to prioritize and co‐design a user‐driven prototype of a questionnaire to provide feedback to service providers. Findings Young people, caregiver and service provider reports of service experiences were used to identify aspects of care quality at eight mental health service contact points: Access to mental health care; Transfer to/from hospital; Intake into hospital; Services provided; Assessment and treatment; Treatment environment; and Caregiver involvement in care. In some cases, low quality care was harmful to users and their caregivers. Young people co‐designed a prototype of a user‐driven feedback questionnaire to improve quality of service experiences that was supported by service providers and caregivers at the co‐design event. Conclusion By using EBCD to capture in‐depth data regarding experiences of young people, their caregivers and service providers, study participants have begun to establish a baseline for acceptable quality of mental health care for young people.

as improving clinical safety of health services and health outcomes. 4,5 However, methodological and practical challenges to incorporating youth perspectives for quality improvement can exist in child and youth mental health services (CYMHS). As with other hard-to-reach groups in society, young people may experience a power imbalance when asked to provide feedback, and prior negative experiences accessing mental health care may make them unwilling to engage in participatory quality improvement efforts. 6,7 Nonetheless, young people have been engaged to create quality indicators for service engagement 8 as well as quality indicators for mental health in the primary care setting. 9 Across CYMHS more broadly, the fluidity with which young people access services across multiple organizations within and outside of health care makes it difficult to define quality dimensions and indicators that pertain to the networks comprising the "system" and operationalize system-wide quality improvement efforts. A further challenge is that the perspectives and experiences of caregivers and service providers in CYMHS systems improvement are typically not included in more than a tokenistic way, if at all. 10 Experience-based co-design (EBCD) has been highlighted as a best-practice approach to engaging users in quality improvement in mental health care. 11 Using an experience-based approach in research can engage young people with mental health issues, reduce stigma and empower them through service co-design 12 to share their specific needs and approaches to seeking and receiving mental health care. 13 The EBCD approach also incorporates the experiences of caregivers and service providers who work together with youth to co-design quality improvements that reflect all three perspectives.
In this paper, we sought to understand dimensions of quality in CYMHS based on qualitative data that was gathered as part of a larger EBCD study (the myCo-Design study) 14 focusing on the coordination of mental health services for young people aged 16-24, across multiple settings in the south-western region of the province of Ontario, Canada. Ontario was selected because it is Canada's most populous province and has set a high priority on improving the quality of mental health services for this age group. 15,16 For example, in 2011, Ontario's Comprehensive Mental Health and Addictions Strategy was released, with the first 3 years focusing on the mental health of children and youth, 17 and more recently, a number of national 18,19 and provincial efforts are underway to improve mental health system coordination and quality of care for this age group. 20 When it comes to service coordination, a number of structural challenges for this age group exist in the Ontario context. Despite efforts to improve local coordination for young people accessing CYMHS, jurisdictional issues continue to complicate and impede transitions between child and adult mental health services, as well as coordination across agencies, regions and systems * .
The objectives of this paper are threefold: (i) to present the process and outcomes of EBCD and showcase how experience data can inform the development of quality indicators for CYMHS across the inherent complexity of the multiple services and settings involved; (ii) to understand which quality factors hold most promise in improving experiences of mental health services; and (iii) to present a prototype of a tool co-designed by youth, caregivers (family member or other lay person identified by the youth) and service providers in Ontario that can provide user feedback to providers with the objective of improving the quality of service experiences in CYMHS. We place particular emphasis on the experiences of young people as service users and how these can be a basis for considering the highs and lows in service experiences as potential indicators of quality. We also focus on a prototype driven by youth and supported by their identified caregivers and service providers, as a tool that can enhance communication between youth and service providers, and ultimately improve the experiences of youth mental health services.

| Overarching research design
Experience-based co-design (EBCD) is an approach informed by the design sciences, with evidence-informed applications in health care. 10 EBCD employs multiple perspectives (typically service users, an identified caregiver-family member or other support person, and a service provider), to diagnose, intervene, implement and evaluate health services with the ultimate goal of co-designing service improvements based on experiences. 21 Participants' experiences are gathered and analysed for "touch points" that represent significant experiences eliciting emotive responses in their health service use trajectory. The strong emotional response suggests the experience has touched upon a core value with respect to health care that may be a signal of health-care quality. 22 In addition to traditional qualitative interviews, participants in the myCo-Design study were asked to use the suite of apps (myExp apps) consisting of a mobile app (young people) and a web-based app (caregivers and service providers) developed to ask users to provide feedback on youth mental health service experiences from their own perspectives in real-time. The myEXP apps capture experiences when prompted following each appointment, or when spontaneously entered throughout the study period † .
Once touch points are identified for each perspective (youth, caregiver and service provider), they are categorized as highs, or positive experiences of care, and lows or negative experiences of care. The touch points are mapped according to various stages in the journey through the mental health-care system. Typically maps are *In an effort to improve local coordination, Ontario has partially regionalized its health   23 for a more detailed explanation). A key stage in the EBCD process is an event during which the three participant types are brought together to co-design solution prototypes for issues at key touch points. Subsequent EBCD phases can include further development, implementation and evaluation of the prototype or solution, within relevant systems/organizations. These various stages of the EBCD process correspond to what has been described as the "Double Diamond" of design, 24 where there is first an expansive process to explore all dimensions of the problem (discover), followed by a narrowing process to decide what to focus on (define), which comprises the first diamond. The second diamond begins from the narrowed focus of the problem to generate a wide range of potential solutions (develop), followed by a narrowing to a small number of solutions that have the potential to work from all perspectives (deliver). In the EBCD process applied here, the app data and interviews generated large numbers of touch points in understanding the problem. The focus group activities narrowed these through experience mapping by perspective. The research team subsequently synthesized these to three overarching touch points corresponding to each participant perspective, which became the foci for the co-design event. At the co-design event, participants from each perspective first individually and collectively brainstormed solutions. We adopted an inverted stoplight technique 25  Each group then collectively drew a visual representation of their solution prototype. Participants from the other groups next circulated and commented on each other's prototypes to enhance them. Notes were taken that reflected these commentaries to be incorporated in the final prototypes that reflected all participant perspectives.

| Data collection
Data from the myCo-Design study were collected between August 2015 and July 2017 and used to identify touch points to understand what is driving high versus low quality mental health care from the perspectives of youth, caregivers and service providers and to capture their suggestions for prototype solutions for quality improvement (see Table 1). Three data sources were used as follows: (i) semi-structured interviews conducted with young people aged 16 to 24 (n = 19), caregivers (n = 12) and service providers (n = 14); (ii) qualitative data from the myExp app questionnaires  Co-designed prototype event was attended by 6 youth, including a youth engager, 5 caregivers including a family engager, and 6 service providers, and was facilitated by researchers with experience working in codesign processes involving youth. In addition to field notes, researchers wrote memos after conducting interviews about their perspectives on the data being gathered.

| Participant recruitment
Youth were recruited through organizations where they were receiving services and each youth was asked to choose a caregiver to participate, along with the referring service provider § . Service providers represented a range of roles including residential care All data were managed using NVivo software (version 11.0).

| Phase 1: Finding and mapping touch points
We used an interpretive phenomenological approach to coding data and identifying touch points, combined with sensitizing concepts from the literature. 29,30 This approach aligns with the diagnostic stage of EBCD wherein experiences are gathered, and researchers are positioned to record these experiences from an ethnographic stance, consistent with methods aimed at empowering vulnerable groups such as youth. 10,29 Using the high and low touch points from the data to develop experience maps for each perspective, we descriptively categorized these touch points by type of context, setting, location and type of associated patient quality incident. We first identified the main points along young peoples' mental healthcare journeys from the three perspectives and then mapped high and low (or positive and negative) touch points during the interface with services (eg, in accessing mental health care, during transfer to and from hospital, during hospital intake, while receiving services, during assessment and treatment, and discharge from hospital. We also noted high and low points that emerged pertaining to caregiver involvement in services and the treatment environment). We refer to these points of interface and other key themes where touch points were identified as "contact points" and take the variation in high and low touch points at these points of interface as indicators of quality variation.

| Phase 2: Validation and prototype development
The co-design process included several rounds of participant engagement, the first of which was a validation exercise of experience maps. As the EBCD process is participatory in nature, we recorded feedback from the validation exercise in real-time in the like-participant type focus groups (young people, caregivers, service providers). We then worked with participants to interpret their feedback and support them in developing an overarching question that summarized the problem for the prototype co-design phase of the event.
During the co-design work, each group focused on an overarching touch point that captured the core challenge in youth mental health services from their own perspective. The other groups commented in turn on these initial prototypes to enhance them to reflect all perspectives. We focus here on the work of the youth as service users, and the comments of service providers and caregivers to enhance the youth prototype ¶ . figures. While all of these are important from a quality improvement perspective, we focus here on the youth perspective as service users, and statements from the other perspectives that support the issues raised by youth. As shown in Figure 1, young people described extremes in many areas. Notably, a common influence on their perceived quality of mental health care for youth across the § Note that for some youth the service provider was also selected as the informal support rather than a caregiver. ¶ Service providers and caregivers developed prototypes that were designed to improve experiences but did not take the form of a quality feedback tool and so are not reported here, but are discussed in forthcoming publications.

| Touch points and the continuum of quality mental health care
service contact points was the degree to which they felt validated and heard in their interactions with providers.

| Access to services
In terms of accessing services, one youth described a high point when the provider explored unique options such as research studies to provide youth with access to the needed services. Service providers also described instances where they have discretion in facilitating access, by bending rules to be able to continue to see youth beyond the typical limits of service provision. For example, one provider described discharging a youth and then accepting the youth as a new referral.
In contrast, another youth who was running out of medication

| Intake into hospital
For many youth, having providers who listen, understand and follow up was described as a positive touch point, particularly during hospital intake, when they are typically in crisis. However, many youth in our study described arriving at hospital because they were feeling unsafe and yet feeling that staff were dismissive and devalued their experiences during the intake process:

| Services provided
In terms of youths' experiences of ongoing service provision, there were many instances where youth described providers who went above and beyond to provide high quality services. For example, one youth who was admitted to an inpatient setting was very appreciative that a nurse who had worked with the youth before went out of her way to be assigned to the youth so as to provide a "familiar face." However, another youth described being put in the wrong programme in the community, which resulted in a worsening of symptoms.
I was there with the intention of getting help for my depression

| Assessment and treatment
Service providers commented that a marked change has occurred over the years in terms of how much time can be spent working with youth, with a resulting reduction in quality. Service providers explained that there is too much reliance on medication during assessment and treatment, because there is not enough time and staff to offer longer term, ongoing care instead. This may contribute to a disconnect between the care youth felt they needed when reaching out for mental health care, and the care they actually received.

| Treatment environment
Youth also pointed to the importance of environmental elements in However, other environments were described as being highly sterile at best or at worst "traumatizing" and feeling "like a jail." One youth described feeling violated as a hospital took away all of their belongings and tore apart their clothing to remove anything that could be used for self-harm. The youth described then being placed in a room that "someone totally destroyed … like punches and kicks everywhere, and there's plaster all over the walls." (Youth)

| D ISCUSS I ON
Our findings suggest that the EBCD process was helpful in engaging participants' voices and perspectives, about their experiences of mental health-care quality. Youths' experiences pointed to the importance of having access to services, feeling heard, validated and responded to, and having service providers who could adapt to their individual needs. The co-design event allowed youth to validate the relative importance of communication and collaboration to receiving high quality care, and to work together with other youth, as well as caregivers and service providers, to co-design ways to provide feedback to service providers about the quality of their interactions.
Overwhelmingly, youth and caregivers identified the importance of service provider responsiveness to youth as key to delivering high quality care within a complex and fragmented system. Despite being highly motivated to improve mental health services for young people, service providers struggled with communication and informationsharing across organizations arising from system fragmentation. This structural challenge reflects not only the Canadian and Ontario context as discussed above, but has also been identified as an issue for CYMHS in many settings. [31][32][33] Improvements to communication and information sharing are recognized as being critical to improving efficiency of systems, coordinated care, and better health and mental health outcomes. 34,35 The range of high and low touch points shown in In contrast, service providers pointed to organizational and system factors as affecting their ability to provide quality services, such as having to discharge patients to community services that they know are not available, or having to bend rules to keep youth beyond a discharge date that is required by organizational or system level policies. It was disconcerting how often youth and family participants pointed to providers who "bent the rules" in order to meet their needs as indicators of service quality. In a truly quality service, rules would support quality and not need to be bent to deliver quality.

| Implications for policy and practice
When it comes to improving quality, the experiences reported here suggest that there is a need to devise ways to overcome structural barriers to improve service coordination at the organizational and system levels. 36 It is clear that the efforts of some providers to go above and beyond and advocate for high quality services for youth is recognized and appreciated by young people and families. The "wicked" policy problem of improving youth mental health care spans many organizations across multiple systems and sectors.
When facing this problem, service providers play an essential role in engaging and enabling youth, caregivers and service providers to work together to facilitate ongoing quality improvement processes. [36][37][38][39] Many youth and families pointed to experiences of high quality care related to efforts made by particular providers. This suggests there is room to improve quality through provider training in the short to medium term, while addressing systemic and organizational barriers over the medium to longer term ** . In addition, considerable opportunity exists to improve quality by making service environments and approaches more youth-friendly and by empowering and communicating with youth in ways that are more accessible to youth, including using text and email.
In order to develop a higher quality system, the various perspectives need to be considered. In the EBCD process, all perspectives are heard and each group is involved in developing prototypes that reflect their various needs. Youth in our study developed a prototype of a short, simple electronic feedback tool targeting domains of importance to youth that could be used to build quality improvement into mental health service delivery on an ongoing basis, particularly with respect to the provider factors they identified. The feedback tool serves as an important point of departure for ongoing quality improvement in CYMHS both within individual services and across networks of providers as it showcases how important provider communication and collaboration are in young people's journeys through mental health care. Service provider and caregiver participants in our co-design process supported and augmented this prototype with feedback on how this could work in practice, and how the data generated could be used within service delivery organizations.
We anticipate that this process can ensure that organizations and providers develop the mutual understanding and respect for each others' perspectives that can facilitate openness in receiving and integrating such constructive criticism-a condition necessary for quality improvement according to the literature. 40 As is the case for any prototype, is expected to need further refinement, but may be a useful starting point that services in Canada and elsewhere can adopt as a way to enhance system quality going forward.

| Limitations
Each local health-care system is unique, and findings from this study may not be immediately transferable to other settings. We do not claim that the experiences of participants of this study represent larger groups of service users. To our knowledge, this is the first EBCD study to explore how experiences can inform youth-driven quality improvement practices in CYMHS; however, we did find **Prototypes to address these structural barriers were also developed and will be discussed in forthcoming work.
F I G U R E 4 Youth prototype analogous findings in many cases to studies examining patient engagement, experiences and young people's mental health. 7,[41][42][43] The choice to develop a feedback tool as a way to improve quality and service experiences from the youth perspective emerged organically from the EBCD process and was not an explicit objective of the study.

| CON CLUS ION
A clear message across all participant types is that that the time has come to listen to and respect youth and family experiences. A high quality mental health system can no longer be seen as one that dismisses young people and families in crisis only to return home unsupported and ill-informed when at serious risk of self-harm and suicide.
The EBCD process can help to foster mutual respect and understanding and break down the attitudinal barriers that inhibit collaboration across users and other key stakeholders in improving service design, experiences and ultimately mental health and well-being outcomes.
The prototype presented here is a starting point for such engagement. By using EBCD to capture in-depth data regarding experiences of young people, their caregivers and service providers, study participants have begun to establish a baseline for acceptable quality of mental health care for young people.

ACK N OWLED G EM ENTS
We would like to acknowledge the myCo-design research team, particularly Christina Roussakis, Samantha Brandow and Alison Mulvale for their support. We would also like to acknowledge and thank the many youth, family members, and service providers who participated in this work.

CO N FLI C T O F I NTE R E S T
None declared.