Acceptability and utility of digital well‐being and mental health support for university students: A pilot study

To assess the acceptability and explore the utility of a novel digital platform designed as a student‐facing well‐being and mental health support.

Digital platforms can facilitate access to care by signposting to available resources based on symptom levels, informing triage and clinical follow-up, and directing students to other existing campus resources (Becker & Torous, 2019;Lattie et al., 2019).This is an important consideration for stakeholders, as university students have indicated a preference for digital mental health programs as a support system for self-monitoring while receiving or after leaving care, rather than as a replacement for traditional care (Levin et al., 2018).Despite the potential for supplemental self-monitoring tools to be of use among university students, there appears to be a lack of evidence on the feasibility and utility of such tools.Further, digital tools aimed at improving university students' mental health are rarely integrated in a stepped care model or explored in a systematic manner (Berger et al., 2022;Ferrari et al., 2022).Therefore, the aim of this pilot study was to assess the acceptability and explore the utility of a novel digital mental health platform designed in collaboration with students and stakeholders as a student facing well-being resource and to enhance the care experience of help-seeking students.

| Overview of research program
The U-Flourish Digital Well-Being platform (powered by i-spero ® ) is an internet-based application originally developed as part of the PRe-DicT study (Browning et al., 2021), which aimed to improve the treatment of depression in primary care settings in the United Kingdom (Billings et al., 2020).The platform schedules collection of selfreported symptom levels using validated measures to gauge treatment response, provides support and guidance through automated messaging, and informs the need for clinical visits based on user entries that can be shared with providers/clinics.For more detailed information about i-spero ® please see https://www.i-spero.co.uk/.For this study, the i-spero ® platform was adapted for use both as a student self-guided well-being support (well-being pathway) and as part of routine care (enhanced care pathway) at Student Wellness Services (SWS) Queen's University in Canada.For a description of the adaption process see the Supplementary Description file.Offering two pathways was consistent with the notion of a stepped-care framework, providing students seeking care with opportunities for enhanced selfmonitoring in collaboration with their care providers, and giving students the opportunity for self-guided monitoring of monitor their wellbeing and signposting to resources based on their symptom levels.
Beyond direct provider access in the care pathway, there were limited differences between the two pathways.The main difference was language and content (e.g., 'care plans' for those in the care pathway and 'well-being plans' for those in the well-being pathway), while core functionality (e.g., monitoring schedule, measures used, display dashboard) was the same.Inclusion criteria for the care pathway included that the participant was a student (undergraduate or graduate) registered at Queen's University and seeking mental health support through SWS (any provider type).Inclusion criteria for the well-being pathway included that the participant was a student registered at Queen's University (undergraduate or graduate).Participants who did not seek mental health care at Queen's SWS were excluded from the care pathway but remained eligible for the well-being pathway.
The two pathways launched in the fall term (September) of 2021 after a student engagement campaign featuring social media and website posts and digital posters shown in public university spaces and SWS waiting rooms (Supplementary Figure 1).Consenting students self-registered for i-spero ® and completed brief demographic and baseline mental health and well-being measures.The platform then prompted students to complete weekly symptom measures and an Experience Survey at Week 8 (Supplementary Figure 2).Participants who completed 8 weeks of data collection received an electronic $5 gift card and were entered into a draw to win 1 of 5 iPads.Helpseeking students not interested in taking part in the research were offered free access to i-spero ® for the duration of the academic year as part of their routine care.This research was reviewed for ethical compliance and approved by the Queen's University Health Sciences Research Ethics Board (HSREB) (PSIY-692-20).

| Study sample
This pilot study includes data from 120 students in the i-spero ® wellbeing pathway and 121 students in the enhanced care pathway.As described in Table 1, most students in both pathways were between 19 and 22 years of age, had parents with a postsecondary education, and identified as female, heterosexual, and white.

| Demographics
Age in years, gender identity, sexual orientation, and student status were self-reported.Ethnicity was based on Statistics Canada categories ( 2016).The highest level of education achieved by either parent was an indicator of socioeconomic status.PANKOW ET AL.

| Baseline mental health
Students reported lifetime mental disorders of any of the following: mood, anxiety, psychotic, eating, neurodevelopment, sleep, substance use disorder, learning difficulty and/or ADHD, or other disorder.Lifetime and current mental health treatment were reported.Lifetime history of suicide ideation and attempts, and self-harm without suicidal intent was reported using questions from the Columbia Suicide Severity Rating Scale (Posner et al., 2011).

| Mental health indicators
Symptoms of depression and anxiety were measured using the PHQ-9 (Kroenke et al., 2001) and GAD-7 (Spitzer et al., 2006), respectively.On both scales a score of ≥10 is considered a screen positive for clinically significant symptoms.Well-being was measured using the WEMWBS-7 (Stewart-Brown et al., 2009).A score of ≤19 was set as the threshold for low well-being (Ng Fat et al., 2017).

| Student feedback
Students were asked to complete an Experience Survey that assessed the acceptability and perceived usefulness of using the platforms with fixed and open-text responses at Week 8 (Supplementary Figure 2).
The experience survey was based on the Client Satisfaction Questionnaire (Larsen et al., 1979) and tailored for purpose in this study through working groups with undergraduate student members of the research team.
T A B L E 1 Description of students using the enhanced care and well-being platforms.
Well Secondary indices of utility were explored by examining changes in GAD-7 and PHQ-9 scores between baseline and 8 weeks, and at their last completed measure before discontinuation.Both well-being and care pathway users provided ranked responses as to whether using i-spero ® had impacted their studies, saved them from having to seek more intensive face-to-face support, and helped their mental health and well-being improve, again with open-text questions to expand on their response.Utility was further explored for care pathway users through perceived improvement in care (i.e., felt more involved in making choices about your care plan, helped understand progression in treatment, improved quality of your care, and made care feel more responsive to your needs).

| Analysis
An integrated mixed methods analysis was used as outlined by Zhang and Creswell (2013).Quantitative and qualitative data were analysed separately, and the findings integrated in the Discussion.
SAS Version 9.4 (SAS Institute, Cary NC) was used for the statistical analyses.Descriptive statistics were used to summarize user demographics, lifetime and current mental health at baseline, and experience survey responses for students in each pathway.The proportion of students that completed the PHQ-9 and GAD-7 were calculated for each week, along with corresponding 95% confidence intervals using a Wilson score-based approach (Agresti & Coull, 1998).
Differences in symptom and well-being scores at baseline and follow-up (i.e., at 8 weeks or before discontinuation) were examined using a paired t-test.Chi-square tests were used to examine if students who stopped using the platform after baseline differed from those who completed Week 1 based on age, gender, student status, baseline GAD-7 and PHQ-9, and lifetime history of mental disorders.
The main quantitative analyses were conducted and presented separately for the two pathways to investigate if there were differences in the main experiences of the pathways (i.e., acceptability and utility).
The pathways were combined for the exploratory analysis and when examining factors associated with attrition and completion to maximize the power after finding similar patterns in the two pathways separately.We felt this was justified for the exploratory analysis examining changes in depression, anxiety, and well-being because the functional goal of both pathways was to improve student mental health and well-being.
We used a qualitative descriptive approach to guide the analysis (Sandelowski, 2000).Qualitative analysis were conducted together because the questions were identical beyond those of experiences with integration, and combining the analysis provided better information power (Malterud et al., 2016) with the response rate to open text questions.Briefly, the first author (KP) reviewed all open-text responses and inductively generated a code book based on the first 10 participants' responses, which was discussed and refined with a second author (GD).Four senior undergraduate students were trained together on the code book using de-identified data from five participants, and independently coded de-identified data from 10 other participants.Coders had an inter-rater reliability of 84.8% after training.
Following training, data was divided among the four student coders so that each participant's de-identified data was coded by two students.The first author and coders met to reach a consensus.The first author then reviewed all the coded data to generate themes by combining related codes, which were reviewed by all authors.An overview of the most salient themes and representative quotes are presented in Table 5.

| Baseline mental health
At baseline, 58% of well-being users and 83% of enhanced care users reported a lifetime mental disorder (Table 2).The most frequently reported disorders (well-being, care) were anxiety disorders (42%, 68%), mood disorders (38%, 55%), and learning difficulties or ADHD (15%, 18%).In the well-being pathway, 57% of users reported lifetime mental health treatment and 41% were currently receiving treatment, compared to 85% and 75% in the enhanced care pathway, respectively.
In the well-being and care pathways, 48% and 55% met the threshold for low well-being, 45% and 64% screened positive for anxiety and 63% and 76% screened positive for depression at baseline.
Sixty-eight percent of well-being and 80% of enhanced care users screened positive for at least one of anxiety or depression.Further, over 31% of students in the well-being pathway and 45% of students in enhanced care reported lifetime self-harm and 18% and 21% reported a lifetime history of suicide attempt(s) (Table 2).

| Acceptability
There was a substantial drop in completion rates of the GAD-7 and PHQ-9 from Week 0 (baseline) to the end of Week 1 in both pathways (Figure 1).Specifically, about 50% of students in the well-being and 40% of students in the enhanced care pathway did not complete measures after baseline.From Week 1 to Week 8 there was a pattern consistent with slow attrition across both pathways.Students who did not complete measures after baseline had lower baseline symptom scores (Mean [SD]) on the   The proportion of students logging in to i-spero ® each week was similar to weekly GAD-7 and PHQ-9 completion rates (Figure 2).
T A B L E 2 Self-reported mental health status at time of sign-up.Note: (1)*Any diagnosed mental disorder includes mood, anxiety, psychotic, eating, neurodevelopmental, sleep, substance use, learning difficulty and/or ADHD, or self-reported 'Other' disorder, (2) Depressive symptoms were measured using the PHQ-9 and anxiety symptoms were measured using the GAD-7, with higher scores indicating greater symptoms; Mental well-being was measured with the SF-WEMWBS-7, with higher scores indicating greater well-being.
Specifically, there was a significant drop off over Week 1, followed by minor attrition over the next 7 weeks.
Students in both pathways (n = 60) generally agreed that i-spero ® was easy to use and understand, and about 50% expressed feeling satisfied overall with the platform (Table 3).The majority felt neutral or positive about enjoying using the platform and about if the platform included the right amount of information.A minority (20%) felt that using i-spero ® had motivated them, but more students agreed than disagreed that they would recommend i-spero ® to other people they know.

| Utility
Most students reported that i-spero ® had a positive impact on their emotional self-awareness, and 40% of students in enhanced care agreed that i-spero ® helped them know when to reach out for help (Table 4).Users of the enhanced care platform were divided on whether they felt the platform made them feel more supported and led to improvement in their care (Table 4).Most students felt that i-spero ® had no impact on their studies, and only 10% reported that using the platform saved them from seeking more intensive faceto-face services.
In an exploratory analysis that combined i-spero ® pathway data for students who completed the baseline and 8-week measures (n = 47), there was a statistically non-significant decrease in average depression and anxiety symptom scores over the 8 weeks  1).Further, in students who completed a follow-up, but discontinued before Week 8 (n = 86 and 77), average PHQ-9 and GAD-7 scores decreased from baseline to their last recorded observation (13.9 [6.6] to 11.9 [7.2], p = .002and 10.8 [5.9] to 10.0 [5.9], p = .06,respectively).
F I G U R E 1 Proportion of students that completed both the GAD-7 and the PHQ-9 from baseline (Week 0) to Week 8, by pathway.Shaded regions are 95% confidence intervals calculated using the Wilson score approach.
F I G U R E 2 Proportion of students that logged into i-spero ® from baseline (Week 0) to Week 8, by pathway.Shaded regions are 95% confidence intervals calculated using the Wilson score approach.
T A B L E 3 Student feedback on the acceptability of the well-being and enhanced care i-spero ® platforms after 8 weeks.

| Qualitative results
Results from the qualitative analysis of student responses from both pathways are presented in three main themes: (1) perceived mental health benefits of using i-spero ® , (2) refining and enhancing i-spero ® to improve engagement and utility, and (3) integration of digital selfmonitoring in mental health care (Table 5).That is, students reported i-spero ® was a useful tool for improving their self-awareness and self-regulation related to their mental health by providing an opportunity for them to check in on themselves.Further, students felt i-spero ® provided an overall positive user experience, with the most common suggestion to improve engagement being to better tailor i-spero ® to meet student needs by connecting to more accessible and making the platform more engaging and intuitive.
Finally, students using i-spero ® in enhanced care experienced a lack of integration of their data in treatment decisions, which was frustrating and perceived as a missed opportunity to more responsive and individualized care.

| DISCUSSION
The aim of this pilot study was to assess the acceptability and explore the utility of a novel digital mental health platform designed to support university student well-being and enhance the mental health care experience of help-seeking students.The results provide evidence that digital mental health platforms are generally acceptable to students, although engagement and persistence of use remains a challenge.Further, preliminary evidence supports that this digital resource could be useful in promoting mental health literacy (e.g.improving emotional self-awareness) and improving timely help-seeking.Better integration into routine clinical practice has the potential to advance proactive individualized mental health care.
Student engagement with the platform in both the well-being and enhanced care pathways dropped off significantly after Week 1.This observation suggests that some students may have been curious or decided that the resource was not for them after initial exploration.
After Week 1, there was a slow decline in engagement up until Week

Perceived mental health benefits
Improved self-awareness and regulation Students said i-spero had an overall positive influence on their mental health by helping them become more aware of their needs, thoughts, symptoms, and behaviours related to their mental health and wellbeing.
'It makes an individual more cognizant of their level of symptoms and progression across treatment-this awareness was the most beneficial aspect for me'.'I think that it can help some students to be more aware of their emotions and mood cycles.For people like me who sometimes struggle with emotional awareness, this helps'.'It allows for the individual to self-monitor, and notice when they do in fact need additional help'.

50
No perceived benefits Some students did not regarded the impact of i-spero on their mental health as neutral.
Many of these students also reported issues related to the category of Integration.
'I cannot think of a single impact I-spero has had either positive or negative'.'I don't believe it generally had a large impact'.

31
Feedback on app Positive user experience Students expressed generally positive sentiments regarding i-spero.Many did not provide specific examples, while others commented on the ease of use or i-spero exceeding their expectations contributing to a positive experience.
'I thought that it would be too simple but it's relatively thorough'.'I expected it to be longer and more of a commitment.It was quite easy to use'.

29
Met or did not meet expectations Students indicated using i-spero was in-line with or below their expectations.Many students who reported that i-spero failed to meet their expectation also reported issues related to the category of Integration.
'It met my expectations in some ways (such as the self-monitoring questionnaires)'.'It did not differ from my expectations'.'I expected my care team to take my results seriously when I could see a depressive episode coming and continuing to escalate.It made all the difference in when I reached out but I didn't expect to get turned away or shrugs in response to finding solutions to meet my needs and getting help'.

11
Options to tailor to student needs Students thought future iterations of i-spero could benefit from tailoring the app to student needs.Several students suggested including more measures and resources relevant to student mental health and wellbeing.
'It differed a lot from my expectations.I thought there would be a lot more activities and stuff integrated into i-spero'.'I expected some sort of feedback or help, or perhaps more local and remote resources [linked through i-spero]'.'There are a lot of surveys that don't necessarily relate to my care'.

15
Feedback on care Integration Students discussed issues with i-spero not being used responsively in care by providers to respond to student mental health changes.Other students highlighted issues with i-spero in care, like not being able to reach a provider when their symptom levels increased.
' (e.g., Freeman et al., 2017) and speaks to the importance of supported engagement and continued improvement based on a fuller understanding of student need and the student user experience.In additon, a better understanding how best to imbed digital resources into existing mental health care practice seems an important direction for future research.
Students in this pilot study were mostly female, heterosexual, and white.This suggests more work needs to be done to engage students from more diverse backgrounds and minoritized groups.While the observed high levels of clinically significant anxiety and depressive symptoms was expected for students in the care pathway, the high screen positive rates in the self-guided well-being platform speaks to the unmet need for support in non-help-seeking students.For example, screen positive rates for anxiety, depression, and lifetime mental health history were higher for students opting into either pathway compared to rates reported from a large representative student sample from the same university (King et al., 2021).This finding also highlights that symptomatic students seem interested in engaging with digital mental health resources like i-spero ® .Considered within a stepped-care framework, this is an important finding as it suggests that students who are highly symptomatic but not accessing traditional care may be open to alternative forms of mental health management that can facilitate transition to services if they become more symptomatic or improvement stagnates.Students who were symptomatic at baseline were not more likely to drop-out over 8-weeks, which suggests i-spero ® was received similarly by students with higher and lower symptom levels.Finally, the drop in adherence after Week 1, together with students reporting insufficient time to complete scheduled measures, supports that briefer versions of validated symptom measures would be more acceptable and may improve persistence.
Qualitative analysis supported quantitative findings in that students felt i-spero ® was a useful well-being and mental health support, possibly through improving emotional self-awareness and mental health literacy; the latter related to a better understanding of symptom levels, progress, and knowledge of when and where to seek help.
On average, preliminary evidence suggests that students who used i-spero ® for 8 weeks experienced a decrease in symptoms of anxiety and depression, although these effects did not reach statistical significance.Further, discontinuation prior to 8 weeks was associated with decreased symptom levels.This is promising given that these symptom scores typically increase over the academic year (Barker et al., 2018;King et al., 2021), although it is difficult to draw conclusions about the extent to which these symptomatic improvements are related to the use of i-spero ® .In a future study we will be able to examine the impact of i-spero ® on symptom levels by comparing student users to a matched sample drawn from the same University student population who complete the same symptom measures as part of the U-Flourish survey (Goodday et al., 2019).
The qualitative results further support the notion that digital interventions related to university student mental health may be best delivered alongside traditional care, rather than as a replacement intervention for students who seek care (e.g., Levin et al., 2018).Our results also highlight potential pitfalls should students have a poor view of the integration of digital tools into stepped care pathways (e.g., frustration with lack of integration into clinical management decisions), an area that had not been explored previously (Ferrari et al., 2022).Barriers to implementing digital mental health tools are not uncommon (Bucci et al., 2019).In our case, an unforeseen barrier was the difficulty engaging providers in using the student data entered into i-spero ® despite having adopted multiple implementation strategies (e.g., partnering with workplace leadership, providing paid all be facilitated using digital platforms such as i-spero.In short, digital tools seem acceptable and useful to students and could help alleviate the burden on student mental health care providers and maximize opportunities for universities to increase support capacity, leverage existing resources more efficiently and effectively.
This study is among the first to our knowledge to assess the acceptability and utility of a digital well-being and mental health self-monitoring platform tailored for university students.Strengths include the use of validated self-report measures and the involvement of students, practitioners, and developers in the collaborative adaptation and implementation of the platform.Further, our use of the two pathways allowed us to explore the utility of stepped care options within digital tools and provide recommendations for future iterations.
Limitations include high early drop-outs, a possible selection bias, and lack of integration into clinical mental health care.These may limit the generalizability of the findings and may have limited critical feedback on the platform.Specifically, if students who had poor experiences were more likely to dropout and therefore less likely to provide feedback, then our findings may be skewed towards the perceptions of users with positive experiences.However, there are a multitude of possible reasons for discontinuation (e.g.mental health improved and/or no longer felt they needed it, lack of time or interest) and as symptom levels in those who dropped out after Week 1 had improved prior to discontinuation, this suggests positive improvement prior to discontinuation.Future research will incorporate interviews with a diverse range of student users to explore reasons why students may choose to continue or discontinue use of i-spero ® , which will provide actionable feedback to improve aspects of the resource related to adherence.

| CONCLUSION
Overall, this study demonstrates the potential for digital resources to form part of organized university student well-being and mental health support.Findings also highlight the importance of student and stakeholder (ie SWS leadership and providers) engagement and implementation to the success of these resources.Digital mental health solutions may be useful in improving mental health literacy and appropriate help-seeking, thereby reducing the burden on already strained campus wellness services, and in the promotion of well-being.

(
13.1 [SD = 6.3] to 12.0 [7.5], p = .23and 10.5 [5.3] to 9.8 [6.2], p = .36,respectively).Average well-being scores remained stable (20.0 [4.9] to 20.2 [6.3], p = .77)(Supplementary Table group and drop-in training hours for providers).More systematic work is needed to understand what implementation strategies in a university student wellness setting might be effective, guided by frameworks such as the i-PAHRIS framework(Harvey & Kitson, 2016).More research is needed to understand the role and realize the potential of digital tools like i-spero ® in stepped care models of student mental health support.Universities UK's 'Stepchange: Mentally Healthy Universities' framework recommends support that is both responsive to students' changing need and incorporates evidenceinformed interventions (UK U, 2020).The i-spero ® tool may meet these requirements given its ability to empower students in their wellbeing and mental health support journey and provides continuity of clinically meaningful individualized data as students transition between levels of support from self-guided to more formal clinical services.Moving forward, our work will act on this feedback by using briefer screening measures and branching logic (e.g., GAD-2 branching to GAD-7 for screen-positives) and a shorter baseline questionnaire to reduce the time commitment unless a student is symptomatic.Further, grouping resources within intuitive 'well-being plans' that address common student concerns (e.g., 'Managing your stress and anxiety'; 'Boosting your mood'; 'Feeling connected'), and direct students to the appropriate campus and community resources based on their individual data (e.g., academic support for those indicating academic stress, self-guided well-being and social connectedness resources for those indicating low well-being, mental health services for those reporting symptomatic GAD-7 and PHQ-9 scores), and aligned with a stepped care framework(Duffy et al., 2019), could improve utility.Furthermore, using digital tools in collaborative care (dashboards shared between student and provider/clinic) could improve the student care experience and facilitate recovery through the titration of clinical visits to individual need, based on progress in care and levels of symptoms.Seamless transitioning to and from services (i.e.community to university) and stepping up and down through the indicated levels of care (i.e.counselling to family physician) could T A B L E 4 Student feedback on the utility of the well-being and enhanced care i-spero ® platforms after 8 weeks.Qualitative results.
Note: Students using the well-being platform completed a subset of the items assessing utility.T A B L E 5 Most students who used i-spero ® to Week 8 expressed overall satisfaction with the platform and felt that the university should continue to offer this digital resource, although many students in the care pathway reported frustration that their i-spero ® data was not integrated into or considered in care decisions.The observed pattern of expressed acceptability but reduced persistence has been reported in other studies of unguided digital interventions in young people