Evaluating the parent–adolescent communication toolkit: Usability and preliminary content effectiveness of an online intervention

Abstract Aim This study aimed to assess the Parent–Adolescent Communication Toolkit, an online intervention designed to help improve parent communication with their adolescents. Participant preferences for two module delivery systems (sequential and unrestricted module access) were identified. Design Usability assessment of the PACT intervention was completed using pre‐test and posttest comparisons. Usability data, including participant completion and satisfaction ratings were examined. Methods Parents (N = 18) of adolescents were randomized to a sequential or unrestricted chapter access group. Parent participants completed pre‐test measures, the PACT intervention and posttest measures. Participants provided feedback for the intervention to improve modules and provided usability ratings. Adolescent pre‐ and posttest ratings were evaluated. Results Usability ratings were high and parent feedback was positive. The sequential module access groups rated the intervention content higher and completed more content than the unrestricted chapter access group, indicating support for the sequential access design. Parent mean posttest communication scores were significantly higher (p < .05) than pre‐test scores. No significant differences were detected for adolescent participants. Findings suggest that the Parent–Adolescent Communication Toolkit has potential to improve parent–adolescent communication but further effectiveness assessment is required.

health outcomes and can impact the choices an adolescent makes (Resnick et al., 1997).
A strong relationship between a parent and adolescent can protect adolescents from emotional distress, suicidal thoughts and violence (Resnick et al., 1997).
Adolescence can be a challenging developmental stage to parent (Larson, Richards, Moneta, Holmbeck, & Duckett, 1996). Increased conflict within a parent-adolescent relationship is associated with higher levels of parental stress (Pasley & Gecas, 1984). Parents have rate adolescence as the most difficult stage of parenting (Anderson, 2008). Parents of adolescents report lower levels of emotional functioning, less competence, lower self-esteem and less life satisfaction, compared to parents of younger children (Larson et al., 1996).
Additional factors increase parental stress such as adolescent mental health concerns, parent health status, family poverty or lack of parenting supports (Anderson, 2008). These issues can reduce the use of effective parenting techniques, reduce parent-adolescent conflict resolution, and increase the likelihood of detrimental adolescent outcomes (Smetana, 2011).
Despite increased stress within parent-adolescent relationships, there are few accessible supports available to meet the needs of these parents. Effective parent interventions to improve parent-adolescent relationships, reduce parental stress and improve parent-adolescent communication are required. Parenting training programs can be difficult to obtain for many families due to location of the service, cost and waiting lists (McGrath, Lingley-Pottie, Emberly, Thurston, & McLean, 2009;Reid & Brown, 2008). Parents are often discouraged by the treatment options available to them (Shanley, Reid, & Evans, 2008).
Available behavioural change parent training programs include Adolescent ParentWays (Taylor et al., 2015), Triple P (Nowak & Heinrichs, 2008) and Parenting Adolescents Wisely (Kacir & Gordon, 2000). Taylor et al. (2015) have argued that despite the effectiveness of evidence-based programs, further parenting solutions that are more accessible, affordable and relevant, are required. Increasing options of care and accessibility of parent training programs will lead to better outcomes for families.

| The parent-adolescent communication toolkit
The Parent-Adolescent Communication Toolkit (PACT) is an online intervention targeted at parents of adolescents to improve the communication of parents with their adolescents, and their relationship.
The PACT intervention (Toombs, Unruh, & McGrath, 2013) was developed with close collaboration between parents of adolescents and the research team. PACT takes principles from Gottman's couple relationship intervention (Declaire & Gottman, 2001;Gottman & Ryan, 2005;Gottman & Silver, 2015) and combines these with the Strongest Families model of care (McGrath et al., 2009 to create a strategy for parents to improve communication with their adolescents. Table 1 briefly describes each module's content. PACT was modelled on Gottman's relationship repair strategies given the applicability of his relationship repair theories to a parent-adolescent dyad with some modifications and the high success rate of this relationship therapy. Simple communication strategies and practical suggestions are combined with activities such as writing and self-reflection exercises for couples. Relationship repair skills can be implemented independently, without the aid of a therapist. By simplifying his research findings, Gottman provided an alternative approach to traditional faceto-face therapy (Gottman & Ryan, 2005). Many of Gottman's couple communication modules such as Nurturing Fondness and Admiration, Creating Shared Meaning and Turning Towards Each Other (Gottman & Silver, 2015) can be applied to a parent-adolescent dyad.
The Strongest Families empirically validated distance care model (McGrath et al., 2009 for child mental health was used as a framework for the PACT intervention. Strongest Families offers telehealth and web-based interventions to families requiring support for childhood behavioural and anxiety disorders. Strongest Families programs implement programs for parents and children assisted by highly trained and monitored non-professional coaches that are reachable by telephone or email. The Strongest Families program is highly effective at reducing typical treatment barriers that exist for families seeking support . Programs offer evidence-based skills that are customized to meet parent requirements. Strongest Families facilitates accessible, convenient and confidential care, in a novel and effective approach to family mental health treatment (McGrath et al., 2009.
By combining the Strongest Families model of care with Gottman's relationship repair strategies, the PACT intervention offers an alternative to traditional parenting interventions. It provides a low cost, convenient measure for parents seeking additional support for parenting their adolescents without the stigma of seeking treatment. The intervention normalizes parent-adolescent conflict, and provides specific skills to reduce these concerns. By delivering the PACT skills online, parents are able to seek information at their own pace, on their own time. PACT is designed for parents to complete without the aid of a therapist.

| PACT delivery via Individualized Research and Intervention Software
The PACT intervention is delivered using Individualized Research and for customizable intervention content to suit the specific needs of the PACT intervention (Wozney et al., 2016).
To meet study demands, IRIS can modify the intervention content presentation, order and time of presentation. IRIS can deliver questionnaires, collect data and offer study completion reminders to participants. The software can track participant activity, such as time spent on each page and will track participant progress and activity by date.

| The purpose and hypotheses of this study
The main purpose of the current study was to assess the usability of the new content and online format of PACT. We assessed the intervention usability using parent feedback to provide ideas regarding potential improvements, and alternative constructs that could better facilitate participant learning in future modifications to the PACT intervention.
PACT is a new intervention and the method of presenting the intervention content had not been assessed. The second aim of this study was to assess two different methods of intervention content delivery to yield the best participant outcomes possible. Participants were randomly assigned to either a sequential method of content delivery, meaning content must be completed in a rigid, predetermined order, or participants had unrestricted access. Participants with unrestricted module access completed modules in any order of choice Introduction Outlines the program and the major features.

Assessment
Taking the parent-adolescent communication test How to determine where your conversations go wrong.

Building relationships
Module 1: Pay attention How to react positively to your adolescent's attempts at emotional connection.
Module 2: Give affection and respect Expressing good feelings about your adolescent through compliments, praise and positive observations.

Module 3:
Create shared meaning Finding shared Creates greater stability in relationships, allows pursuit of goals together. Create rituals to connect, have symbolic and emotional meaning.

Module 4:
Give goals room to grow Recognize and honour the dreams and feelings within your adolescent.
Module 5: Accept your teen and his/her influence Be open to persuasion from your adolescent without giving in.
Module 6: Accept one another Accept your adolescent for who he or she is, not the person you want him/her to be

Positive communication
Module 7: Compromise Avoid gridlocking by working out a decision that both you and your adolescent can agree upon and be happy with.

Module 8:
Start softly Learn to start talking about a complaint without criticizing or insulting.

Module 9:
Repair your communication Deescalate negative feelings during a difficult encounter with your adolescent.

Turn around negative communication (Module 10)
Criticism Avoid attacking your adolescent's personality or character rather than specifics.

Contempt
Avoid insulting and psychologically abusing your adolescent.

Stonewalling
Avoid removing yourself from the conversation mentally.

Defensiveness
Avoid defending yourself from insults. Nine main strategies.

Flooding
Avoid overwhelming your adolescent with too many complaints.
Talk about difficult issues (Optional Module 11) Using the intervention skills to talk to your adolescent about sex, drugs, divorce and mental health. Offers additional resources based on these.

Summary
Summarizes the program and how to solve problem.
functioning of participants. In summary, the hypotheses for this study were: • Parent usability ratings for the PACT content delivered using the IRIS platform would be high.
• Parents randomized to the sequential module access would have higher completion rates and usability ratings than those randomized to the unrestricted module access group.
• Parent and adolescent would use significantly more positive communicational strategies following the PACT intervention, as rated by the IWK-PACC and emotional functioning (DASS) posttest scores would be significantly higher than pre-test scores.

| METHOD
This study was completed using a pre-posttest design with a 6-week intervention. Parent participants were randomized into two groupssequential chapter access or unrestricted module access. Figure 1 is a diagram of the study design.

| Study randomization
All parent participants completed identical PACT content, although how the intervention content was delivered was randomly assigned.
Parents either had unrestricted module access or sequential module access, with each module unlocked only after completion of the previous module.

| Website
The study was conducted using a customized IRIS web-based platform. All usability questionnaires and intervention content were presented using IRIS. Parent participants completed usability study portion (content presentation, data collection and preliminary descriptive analyses) entirely through the PACT website.

| Participants
Eighteen parent-adolescent dyads completed the study. Adolescent participants were between the ages of 13-17 years and were a resident in their parent's home. Parent participants agreed to commit to 150 Posters distributed in community the 6 weeks required of the study. A convenience sample of 20 dyads, recruited by word of mouth in the community, was chosen to assess the overall usability for this pilot study and for this sample to provide in depth comments on aspects of the intervention that should be modified before further implementation. Two dyads completed consent and were randomized, but did not complete any intervention content. Exploratory analyses completed from this sample were used to determine significant trends in the data in relation to the effectiveness of the intervention.
No parent had participated in another parenting or communication behavioural intervention (including any prior PACT study), or received support for a mental health problem in the previous 6 months. Parents who reported any severe psychological impairment for themselves or their adolescent were excluded from the study. All parent participants had access to the Internet. Only one parent-adolescent dyad per family could participate. Parent participants were primarily birth parents, well educated (most achieved a 2-year college diploma or higher) and had a secondary parent in their family. No parents earned less than $20 000 per year. Parent-adolescent dyads were primarily mother-daughter.

| Hypothesis 1: Parent usability ratings of PACT delivered using the IRIS platform would be high
Participant usability module ratings were rated highly by both participant groups with Table 2 depicting the mean module ratings for the sequential access and unrestricted access participant groups.
The total ratings per each of the 10 modules (obtained by averaging all participant raw scores for each chapter) were analysed using a one-way ANOVA and did not differ significantly from one another at the p < .05 level for the ten modules [F (9, 98) = 0.434, p = .914)].

| Participant usability feedback
Parents provided written feedback for each module. Few modifications were suggested. Parents found the "Relationship Memory Bank" and "Being Specific with Praise" particularly helpful. The audio-visual of the intervention content was difficult to access on tablets and phones, and greater diversity in the family's illustrated in the videos was recommended. Parents also suggested the number of questions, the number of examples, and the repetitive content be reduced.

| Hypothesis 2: Parents randomized to the sequential module access group would have higher completion and usability ratings than parents in the unrestricted module access group
Two dyads did not complete the posttest study questionnaires and were removed from subsequent analyses. Remaining parent participants varied on module completion rates, with eight participants (50%) completing all ten modules of PACT. Modules of PACT are dispersed in four sections. Sixty-two percent of participants completed the first section, and 50% completed sections two, three and four.
Participation decreased by 50% after Module 4. Figure 2 depicts the percentage of parent participants who completed each module by randomization to the sequential or unrestricted module access groups.
Participant global usability ratings by each access group were analysed to determine if the sequential module access group rated the content higher than the unrestricted module access group. These global ratings were obtained by adding participant scores across modules and dividing by the total score possible for how many modules

IWK-PACC was divided into three primary subscales: Building
Closeness and Admiration (BCA), Reducing and Repairing Conflict (RRC) and Increasing Conflict (IC). The two communication scales, BCA and RRC, were combined and the negative IC was reverse scored to produce a total IWK-PACC communication total. Table 3

| DISCUSSION
Participant usability ratings of PACT in this study were encouraging.
Parents rated all of the PACT modules quite highly, thus supporting Hypothesis 1, with a small decreased rating for the audio-visual content. Parents rated PACT as easy to use, the content to be relevant and easy to learn. All modules were rated highly. The only consistent feedback provided by parents was that the videos did not work on some devices (e.g., tablets and smart phones) and partially impaired their completion of this PACT content. More audio-visual components were requested, depicting a greater variety of family structures, and communication situations, as videos only showed interactions within a mother-daughter dyad. Parent responses indicated that the content was somewhat repetitive. Small modifications to PACT to address this feedback are required before the next phase of testing.
Parents who were randomized into the sequential module access completed more content and rated the usability of PACT higher than participants in the unrestricted module access group. The significantly higher usability ratings by the sequential module access group provide support for this type of structure for PACT. The sequential access style of content delivery, often called tunnelling, decreases distraction of the user and ensures the user completes necessary content (Danaher, McKay, & Seeley, 2005). Due to both high usability ratings and increased benefits noted by participants, the sequential module access is the recommended style of information architecture.
Increased completion rates of the sequential module access also support the use of this design in future studies. Parents in the sequential module access group completed more of the PACT content than those randomized to the unrestricted module access group. Only two parents or 11% of participants did not complete all of the modules in comparison to a previous PACT study (unpublished data) in which six participants (30%) did not complete all modules.
Participant retention is a priority for future studies of PACT. Fifty percent of the parent participants did not continue the intervention after module 4 even though this module was rated as highly as the earlier modules. Many web-based interventions have similar attrition rates (Eysenbach, 2005) and low completion rates reduce the opportunity to achieve strong program outcomes. More email reminders from PACT could be provided through an automated IRIS system. The attrition rate may also have been influenced by parents' perception of content repetition for modules four and five where skills are applied in more complex communication. Streamlining of content to reduce repetition will be helpful.
PACT presents foundational skills related to building a positive relationship before more challenging skills designed to reduce conflict.  Increased customization may also decrease attrition rates when combined with the sequential chapter access design due to increased relevancy and structure of the content. A systematic review by Christensen, Griffiths, and Farrer (2009) Fathers tend to be less willing to participate in parenting interventions and often have a much higher attrition rate and lower satisfaction than mothers (Lee & Feldgaier, 2015). In this study, five of the 18 parents were fathers. They did not identify any gaps in the PACT content but the small sample size did not permit comparison by gender.

| Study limitations
The main limitation of this study was the small sample size and inclusion of parent-adolescent dyads with relatively strong relationships at study outset. During the recruitment phase, six parents expressed interest in completing the research study but could not participate due to the unwillingness of their adolescent.

| CONCLUSION
This study confirmed the usability of the PACT intervention and provided evidence for the best structural architecture for content delivery. Participants identified modifications to the intervention content to improve it for future use. Exploratory analyses of change in parents' perceptions of their relationship with their adolescent following PACT indicated that the PACT intervention may be a viable tool for improving parent-adolescent communication.

ETHICAL APPROVAL
Ethical approval for this study was obtained from the Research Ethics Board at the IWK Health Centre, in Halifax, Nova Scotia, Canada. • substantial contribution to conception and design, acquisition of data or analysis and interpretation of data;

AUTHOR CONTRIBUTIONS
• drafting the article or revising it critically for important intellectual content.