The study was undertaken while Ms Dellosa was a student at the University of Melbourne; Kristina is now working at the Youth Mood Clinic, Orygen Youth Health, Locked Bag 10, Parkville, Victoria 3052, Australia.
Development and pilot testing of an online monitoring tool of depression symptoms and side effects for young people being treated for depression
Article first published online: 25 FEB 2014
© 2014 Wiley Publishing Asia Pty Ltd
Early Intervention in Psychiatry
Volume 9, Issue 1, pages 66–69, February 2015
How to Cite
Hetrick, S. E., Dellosa, M. K., Simmons, M. B. and Phillips, L. (2015), Development and pilot testing of an online monitoring tool of depression symptoms and side effects for young people being treated for depression. Early Intervention in Psychiatry, 9: 66–69. doi: 10.1111/eip.12127
- Issue published online: 19 JAN 2015
- Article first published online: 25 FEB 2014
- Manuscript Accepted: 20 DEC 2013
- Manuscript Received: 8 OCT 2013
- Colonial Foundation
- evidence-based practice;
- suicide-related behaviour;
To develop and examine the feasibility of an online monitoring tool of depressive symptoms, suicidality and side effects.
The online tool was developed based on guideline recommendations, and employed already validated and widely used measures. Quantitative data about its use, and qualitative information on its functionality and usefulness were collected from surveys, a focus group and individual interviews.
Fifteen young people completed the tool between 1 and 12 times, and reported it was easy to use. Clinicians suggested it was too long and could be completed in the waiting room to lessen impact on session time. Overall, clients and clinicians who used the tool found it useful.
Results show that an online monitoring tool is potentially useful as a systematic means for monitoring symptoms, but further research is needed including how to embed the tool within clinical practice.
Guidelines assist clinicians to provide evidence-based and effective interventions for young people with depression.[1-5] They consistently recommend regular monitoring of symptoms and, for those prescribed an antidepressant, of adverse effects and suicide risk given this risk is increased on antidepressants.[6-8] However, research has shown these recommendations are not uniformly implemented.[9, 10] Based on a study of barriers to monitoring, a self-report tool was designed to assist clinicians to undertake adequate monitoring.
The aim of the current study was to examine the feasibility of this tool.
The Youth Mood Clinic (YMC) is part of Orygen Youth Health (OYH), a public youth mental health service for those aged 15–24 living in the northwestern metropolitan area of Melbourne, Australia. Eighty-five percent of YMC clients have major depressive disorder; 80% receive mediation given the severity of disorder, established by the Patient Health Questionnaire (PHQ-9) and clinical assessment. Psychiatrists regularly review clients, and case managers (clinical psychologists, social workers, occupational therapists) deliver weekly cognitive behavioural therapy and case management.
New or already established YMC clients and all clinicians in the YMC were invited to participate.
Design and development of the monitoring tool
The tool was based on guideline recommendations for the regular monitoring of symptoms and side effects, particularly suicidality.
Phase 1: development
The PHQ-9 was chosen to monitor depression symptoms given its use in the YMC. It has sound psychometric properties and is used in those aged as young as 12. While mostly used for screening, it is a useful outcome measure of depression.[13-15]
The Suicidal Ideation Questionnaire–Junior (SIQ-JR;) was chosen to monitor suicidal ideation on the basis of a systematic review in which it was the most common tool. It has sound psychometric properties and is used in those aged as young as 11.[16, 18]
A checklist of side effects was constructed using the Antidepressant Side-Effect Checklist (ASEC) and a synthesis of reported side effects from a systematic review of antidepressants in adolescents.
The tool was designed to be available online; at the YMC it was available via an iPAD.
Phase 2: refinement
The platform team at OYH, comprised of past and present clients, participated in a focus group. Feedback included: (i) a preference for radio buttons rather than drop-down options; and (ii) a preference to exclude images.
Feedback from YMC clinicians included using language consistent with the service (e.g. ‘case manager’ rather than ‘clinician’) and limiting free text boxes given concerns these would be used to communicate distress outside of clinic hours with the expectation of an immediate response.
A researcher contacted the YMC administrator regularly to obtain information about new referrals. Clients were contacted by telephone, information about the study provided and verbal consent obtained. If the client was under 18 years of age, verbal consent was also obtained from a parent or guardian. Signed written consent was obtained at their next appointment. Written parental consent was obtained during this appointment or via post. Written consent was obtained from clinicians during the consultation session with YMC.
Participants were asked to complete the tool once a week for up to 3 months. Clients could fill in the tool at any location with Internet access at any time with the exception of suicidal ideation items. These were completed at the beginning of their regular treatment session with their clinician on an iPad. Once completed, the client handed the iPad to their clinician, with the iPad programmed to display SIQ-JR responses with an indication regarding the meaning of the scores (i.e. SIQ-JR score of >31 or a score of >4 on items 2, 3, 4, 7, 8, 9, 13 or 18). Clinicians could then respond immediately according to the YMC protocols for suicidal risk.
Clinicians had no access to raw data of clients’ PHQ-9 and SIQ-JR scores but received a chart of these scores after 4 weeks and could choose to share this with their client. They received an email regarding side effects that were endorsed, which they could report to the treating psychiatrist for immediate action.
Clients and clinicians also filled in a survey about the functionality and usefulness of the tool. A focus group with YMC clinicians and telephone interviews with clients were conducted to further investigate these issues.
This study received ethics approval from the Melbourne Health Research and Ethics Committee (reference number 2010.240).
Frequency data and descriptive statistics were used to report on the use of the tool. Analysis of qualitative data was driven by the research aims regarding functionality and usefulness, and was undertaken in accordance with Braun and Clarke's methods.
All seven YMC case managers and 34 of their clients consented to participate. Doctors were not involved as they see clients less frequently and were difficult to liaise with given case load and time constraints.
Of the 34 clients who provided consent, six were unable to commence because a signed parental consent form was not returned. A further 13 did not commence use of the tool. The tool was filled in by 15 clients (mean age 18.74, standard deviation (SD) = 2.65). The mean PHQ-9 score was 13.3, which indicates a moderate level of severity; 73% were on medication. They filled the tool in between 1 and 12 times (M = 5.33, SD = 4.08). The tool was mostly completed with the clinician (Table 1).
|Waiting room at clinic||5||5.95|
Seven clients and seven clinicians completed surveys, and six interviews with clients plus a focus group with clinicians were undertaken.
The time taken to complete the tool was acceptable to most clients (range 2–10 minutes). One client reported that it took too long to complete.
All but one client reported the online format was ‘good’, stating it was easier than having a ‘conversation with the case manager’, and that it was ‘simple’ to complete. One client found it ‘a bit weird’ because they were not used to it, and two reported it was not capable of ‘capturing what you felt by ticking a box’.
Clinicians all reported liking the tool but were concerned about its length and that too much ‘time [was] taken in session’. Most clinicians suggested implementing the tool in the waiting room.
Most clients stated the tool helped them in some way. Generally, it helped them to: (i) understand more about their symptoms; (ii) understand how symptoms fluctuated in response to life events; and (iii) allowed them to feel more in control of managing symptoms. Two clients stated the tool helped them change their behaviour of ‘not eating and sleeping’. Only one client said that the tool was not helpful because ‘it wasn't telling me anything I didn't (already) know‘.
No clinician showed their client graphs of their PHQ-9 or SIQ-JR scores; however, clients generally said this would have been helpful.
Clinicians commented that the tool helped them understand more about their client's risk. One clinician said that it was useful for informing treatment planning, and another found it useful for engagement ‘particularly with clients who are reluctant to disclose information to clinicians’.
Undertaking systematic monitoring of symptoms and side effects using an online format can be beneficial. Clients and clinicians had mostly positive feedback about the functionality and usefulness of the tool. Time taken to complete the tool was generally acceptable and clients reported it easy to use. Clients identified a particular benefit as having a greater understanding of and control over symptoms, consistent with the self-management literature. It was therefore surprising that clinicians did not share with their clients the PHQ-9 or SIQ-JR scores, particularly given clients thought this would have been useful.
Clinicians reported understanding more about their client's risk via the tool. It was helpful for engaging clients, particularly those who were reluctant to disclose information, suggesting self-monitoring can foster a more collaborative relationship between clinician and clients. Given previous research and feedback from clients, this collaboration may be enhanced by the careful sharing of symptom scores over the course of treatment.
However, the uptake of the tool was relatively poor, indicating that further work is needed to embed the tool within clinical practice. The setting, protocols and clinical management tools already in use need to be considered, as also highlighted in primary care research we have undertaken. Further suggestions from clinicians were consistent with previous research and included reducing the length of the tool and having it completed in the waiting room. These improvements, implementation tailored to individual settings, as well as providing evidence of utility in terms of improving clinical practice via weekly feedback of scores (as demonstrated in previous research) would increase the likelihood that the tool would improve adherence to guideline recommendations about monitoring. Further testing of the tool in this regard is required.
No doctors provided feedback, and surveys and phone interviews were not completed by all clients who used the tool, so generalizability may be limited; investigating how doctors, younger clients and clients being treated in different settings use the tool is important for future studies.
Our thanks to Mr Greg Wadley who did the programming and web design. SH is on a National Health and Medical Research Council Early Career Fellowship. OYHRC is affiliated with Melbourne Health and the University of Melbourne, and receives funding from the Colonial Foundation.
- 3National Institute for Health and Clinical Excellence NICE. Depression in Children and Young People: Identification and Management in Primary, Community and Secondary care. Leicester, UK: The British Psychological Society, 2005.
- 52010) Clinical practice guidelines: depression in adolescents and young adults. Melbourne: beyondblue: the national depression initiative; 2011., , et al. (
- 16Suicidal Ideation Questionnaire (SIQ): Professional Manual. Odessa, FL: Psychological Assessment Resources, 1987..
- 18Assessment of suicidal ideation in inner-city children and young adolescents: reliability and validity of the Suicidal Ideation Questionnaire-JR. School Psych Rev 1999; 28: 17–30., .
- 20Using thematic analysis in psychology. Qual Res Psychol 2006; 3: 77–101., .
- 22Monitoring depressive symptoms and suicidality in young people in primary care settings: focus groups with general practitioners. Aust Fam Physician 2013 (in press)., , , .