Learning from usability testing of an arts‐based knowledge translation tool for parents of a child with asthma

Abstract Aim Digital, art‐ and story‐based resources can be viable and engaging knowledge translation strategies in health care. Understanding the usability of these approaches can help maximize their impact. The aim of this work is to understand what aspects of ‘My Asthma Diary’, an art‐based digital knowledge translation tool for parents of children with asthma, has an impact on usability. Design Sequential explanatory mixed methods pilot study. Methods Eighteen parents of children with asthma reviewed ‘My Asthma Diary’ in a paediatric emergency department and completed a usability questionnaire. Follow‐up interviews were conducted with five parents and analysed with qualitative description. Results We identified four themes which complemented the quantitative results: (a) the eBooks are relatable and mirror personal experience; (b) the digital format is convenient and easy to navigate; (c) the narrative structure aids learning; and (d) the narrative and illustrations are synergistic. We summarize core usability considerations for subsequent research and creative knowledge translation tool development in other contexts.

. Arts-based KT can leverage multiple learning styles, emotional understanding and forms of knowledge beyond the explicit, propositional knowledge characteristic of didactic teaching approaches (Archibald, Caine, & Scott, 2017;. These attributes have utility in various health and illness contexts and may be particularly pertinent to enduring illness contexts such as childhood asthma because of the experiential knowledge that individuals bring to each healthcare encounter and management scenario (Archibald, Hartling, Caine, Ali, & Scott, 2018).
Most asthma management, the most common chronic disease of childhood, occurs outside of healthcare settings (Deis, Spiro, Jenkins, Buckles, & Arnold, 2010;Nicholas, Dell, Fleming-Carroll, & Selkrik, 2009). This requires parents to undertake a great extent of day-to-day care. Consequently, ensuring that parents are equipped to manage asthma day-to-day is integral to improving childhood asthma outcomes. However, research continues to show that parents have pervasive unmet information needs around childhood asthma (Archibald, Caine, Ali, Hartling, & Scott, 2015;McMullen et al., 2007). This points to shortcomings in how research evidence on asthma management has been mobilized and communicated to parents (i.e. often through standard information sheets or verbal information provision only) (Archibald et al., 2015). These shortcomings contribute to poor asthma control and avoidable burdens on the family (e.g. frequent exacerbations, missed school and work) and healthcare system (e.g. high use and cost) (Berg, Anderson, Tichacek, Tomizh, & Rachelefsky, 2007;Coffman, Cabana, Halpin, & Yelin, 2008;McMullen et al., 2007).

| Process of eBook development: 'My Asthma Diary'
In response to the need for more complex educational materials that reflect parental experience, we created 'My Asthma Diary: Coping with Childhood Asthma', a 28-page art and story-based eBook developed in response to parental information needs identified from an interpretive descriptive study (Archibald et al., 2015). The process of developing 'My Asthma Diary' involved multiple stages, which we report on in previous manuscripts (Archibald et al., 2015(Archibald et al., , 2018) and summarize briefly here. In stage one, we conducted a state of the science literature review  to identify domains of parental information needs to inform a semi-structured interview guide and to identify whether a subsequent primary research study was needed. These domains included asthma basics (e.g. basic pathophysiology and knowledge of triggers including environmental modification), treatment modalities (e.g. short-and long-term medications), coping (including self-efficacy) and medical expectations (e.g. when to seek medical assistance) . In stage two, we conducted F I G U R E 1 Example page from eBook prototype 1. Black and white line drawing version with sans-serif font an interpretive descriptive study of asthma information needs with 21 parents of a child with asthma (Archibald et al., 2015). In this study, we identified four predominant information needs (recognizing severity, acute management and inhaler use, prevention vs. crisis orientation and knowing about asthma) and two mediating factors (beliefs about the nature of asthma, interactions with healthcare providers) (Archibald et al., 2015). We then used the parental stories, experiences and knowledge need to structure the narrative for 'My Asthma Diary' and used an interdisciplinary, collaborative process to develop four asthma eBook prototypes (Archibald et al., 2018).
These four asthma eBooks were then presented to a unique sample of parents for usability testing (current manuscript).
Each eBook prototype contained the same narrative, structure and evidence-based content and differed in the aesthetic style of presentation (i.e. illustration style, text font and colour) as visible in Figures 1-4. We detail the development of the eBook in a separate publication (Archibald et al., 2018) and report here on the usability of these resources.

| Usability testing
Usability testing is a common practice in health informatics-the array of assessment and communication activities integrating technology with a human interface (Bastien, 2010). The purpose of usability testing is to determine whether a developed product meets its intended purpose and, as such, is crucial to any user-centred design process (Bastien, 2010). Central to usability testing are questions pertaining to effectiveness (e.g. does the produce do what it intended?), efficiency (what is the resource investment for the system to be used as it was intended?), satisfaction (positive responses from end-users) and errors encountered (Alshaman & Mayhew, 2009;Bastien, 2010;Stinson et al., 2010). Although examples of user-based usability assessment (i.e. approaches that involve direct end-user participation) are readily available in the health literature, most revolve around website development (Alshaman & Mayhew, 2009). More research is needed on what makes Web-hosted (digital) arts-based KT resources usable, in part because of the emergent status of such approaches . Such data will be useful to guide similar strategies in different populations and contexts, particularly given the growing popularity of arts-based KT approaches across disciplines.
Methods of assessing usability are guided by the intricacy of the KT strategy and end-user characteristics. Kastner, Lottridge, Marquez, Newton, and Straus (2010) conducted usability evaluations of three components of a clinical decision support tool for osteoporosis disease management. Using a standardized in-person process with a trained moderator and a think-aloud procedure, the authors collected and analysed video, audio and statistical data to iteratively inform the tool's revision. Like Ben-Zeev et al. (2013) whose usability testing for a smartphone self-management system F I G U R E 2 Example page from eBook prototype 2. Graphic novel style black and white line drawing version with comic font for schizophrenia focused on ease of system use, decisions regarding the importance of real-time feedback were made in reference to the complexity of the delivery system and related content. Stinson et al. (2010) conducted usability testing of an online selfmanagement program for adolescents with juvenile idiopathic arthritis. Using an iterative 2-cycle user-based approach, they conducted qualitative semi-structured interviews and observation methods to detect user performance and revise the self-management prototype. This cyclical approach, inclusive of observation, was necessary given the complexity of the '12-module interactive multi-component treatment protocol' (Stinson et al., 2010, p. 3), which consisted of over 310 content pages, forums and surveys, among other features.
Conversely, Hartling et al. (2010) used an iterative qualitative approach when developing storybooks for childhood croup. The authors employed user-centred design principles to understand end-user perceptions of the stories and storybook prototypes and revised these according to the feedback obtained through participant review and focus group interviews. By focusing on the presentation, interest, style and clarity of the croup stories, the authors were able to maximize the 'accuracy, credibility and relevance' (p. 9) of the storybooks and to identify considerations that align with established usability principles (e.g. identify clear goals and purpose) (HHS.gov, n.d).
Although usability procedures used in the health sciences vary extensively, they should reflect evidence-based practices in usercentred design and draw on the plethora of international standards that have been developed over the past 25 years (Bevan, 2001). For instance, the U.S. Department of Health and Human Services provides best practices for Web-based content, design and usability, including research-based guidelines on design process (e.g. give useful content, establish user requirements) and user experience (e.g. increase Web credibility). When adapted, these principles can inform the development and usability evaluation of digital, arts-based KT strategies.
Applying usability principles to the development and refinement of KT tools can enhance their relevance, uptake and impact (Graham, Kothari, & McCutcheon, 2018). Yet little research exists on the usability of arts-based KT approaches, despite growing use of these approaches across disciplines. In this study, we sought to understand which aspects of 'My Asthma Diary' have an impact on its usability, to contribute to the growing evidence base of arts-based KT tools and help inform future effectiveness studies. In this paper, we report on a usability study examining parent's perceptions of 'My Asthma Diary'-a parent-driven, arts-based KT eBook developed for parents of children with asthma (Archibald et al., 2018) and discuss how the lessons learned can be applied to other creative KT endeavours with different populations.
F I G U R E 3 Example page from eBook prototype 3. Blended colour illustration with sans-serif font

| DE S I G N
We conducted a sequential explanatory mixed methods usability pilot for the purposes of complementarity (i.e. elaborating on and clarifying quantitative results using qualitative methods) (Greene, Caracelli, & Graham, 1989) using a complementary strengths stance (Morse, 2003). We drew from two traditions: technology development, where a basic usability assessment using quantitative methods was conducted (Ancker, Chan, & Kukafka, 2009;HHS.gov, n.d), and a generic qualitative approach, where a deeper exploration of the meaning of the quantitative responses and additional perceptions of the eBooks were elicited.
Determining optimal sample size for usability testing is the subject of long-standing debate. Bastien (2010) identifies that four to five participants may uncover approximately 85% of usability issues encountered with an electronic interface. While notable support for this approach exists (e.g. Kastner et al., 2010), others purport that asking, 'how many users is needed' is perhaps the wrong question to ask. Given that usability testing is seeking to uncover mismatches between the user interface and the users' mental model, an alternative method of considering sample size relates to the number of tasks an end-user is being asked to complete (Straub, 2007). As very few unique tasks (e.g. access information icons, navigate sequentially through story) were involved in navigating through 'My Asthma Diary', we hypothesized that a quantitative sample of 16-20 participants and a qualitative sample of 4-7 participants would sufficiently uncover most usability issues.

| Quantitative
Parents of a child with asthma who presented to one of two emergency departments in an urban centre (population: approximately 880,000) were approached on a convenience basis to discuss the study. Eligible parents were English speaking and identified as a primary caregiver for a child with asthma or asthma-related concern.
Parents were invited to review one of four eBook prototypes (i.e. prototype three), complete a paper-based 27-question usability questionnaire and consent to a follow-up interview. Parents were fully informed about the study prior to providing verbal consent to participate.
The questionnaire included four pre-and post-test visual analog scale (VAS) measures (i.e. level of asthma knowledge, comfort with asthma knowledge, confidence in managing asthma day-to-day, confidence in managing asthma exacerbations) anchored at 0 ('not at all') and 10 ('very much'), 15 descriptive VAS measures specific to three dominant usability constructs (i.e. effectiveness, efficiency, satisfaction) and four open-ended questions. The VAS was developed based F I G U R E 4 Example page from eBook prototype 4. Graphic novel style colour illustration with comic font on recognized usability constructs and additional measures adapted for arts-based KT (e.g. relatability).

| Qualitative
Consenting parents were contacted for a semi-structured telephone interview. The interview guide was developed to reflect usability constructs having an impact on ease of use in the user-centred design literature, to explore quantitative findings and to identify 'incidents'.
Like Kastner et al. (2010), we defined an incident as a problem or error that either: (a) halted the typical function or navigation within the eBook (i.e. critical incident); or (b) did not inhibit completion (i.e. general incident). Incidents were categorized by type and content.
Consenting parents were provided an online link to the asthma eBooks (Table 1) and time to review the books prior to commencing the interview. Interview questions moved from general to specific, beginning with an exploration of parental perceptions of the eBook prototype three, which parents had previously reviewed. After exploring general usability questions, parents were asked the following specific questions regarding their perceptions of the prototypes: (a) Do you prefer any one version of the eBook and if so, which one and why? (b) Can you rank the eBooks according to your preference? (c) What do you like and dislike about each version? Interviews were audio-recorded.
Qualitative and quantitative integration occurred at the levels of sampling, data collection and analysis. Qualitative participants were sampled from the quantitative arm, enabling exploration of participant responses obtained on the VAS measures. At the level of data collection, the questions from the semi-structured interview guide were developed to explore, confirm or contest the quantitative findings. Analytic integration is discussed under the analysis heading.

| ANALYS IS
We used IBM SPSS Software to analyse the quantitative data using non-parametric (Wilcoxon signed-rank test), parametric (paired t test) and descriptive statistics. Statistical significance was set at .05. Open-ended questions were content analysed. Qualitative data were analysed inductively, guided by qualitative description.
Audio recordings were listened to repeatedly, detailed notes and analytic memos were made, and data were loosely grouped thematically. Themes were revised and categories regrouped as analysis progressed. Rigour was enhanced through detailed memos and through team meetings where emerging findings were discussed and rigorously questioned. Problems encountered while using the eBooks were classified as critical or general incidences (i.e. usability issues that do or do not impair use), and participants were asked for suggestions to improve the eBooks function in relation to occurring incidences. We then used a sequential mixed analysis approach to analysing and integrating qualitative and quantitative data (Teddlie & Tashakkori, 2009). Data were analysed in sequence, and then, qualitative and quantitative findings were reviewed in reference to one another, to interpret the overall usability of the eBooks. Parents, on average, reported that it took 9 min and 24 s to read the eBook (SD 4.00, range 5-20), which they regarded as an appropriate length (mean = 8.44, SD 1.21, range 6.57-9.86). Descriptive statistics illustrate positive ratings on constructs related to effectiveness, efficiency and satisfaction. Outliers were encountered; for instance, participants occasionally provided very low ratings on the enjoyment, contribution of drawings to information and quality of information subscales. Open-ended responses and followup qualitative interviews suggest that participants may not have accessed the information icons (e.g. buttons that when activated, delivered additional information) embedded throughout the tool, relying exclusively on the storyline to receive information. One participant who provided a low enjoyment rating found the story was 'scary and intimidating' for first time parents dealing with asthma (Table 2).

| Quantitative
Given the small quantitative sample and lack of random sampling, we conducted pre-post evaluations of knowledge gain (p = .001), confidence in managing asthma day-to-day (p = .055) and confidence in managing asthma exacerbations (p = .021) using the related-samples Wilcoxon signed-ranks test (i.e. non-parametric paired t test equivalent). We ran paired t tests to obtain data on mean differences and standard deviations. Paired t tests revealed significant improvements in knowledge (t(17) = 3.09, p = .007) between pre (mean = 6.81, SD = 1.98) and post (mean = 7.78, SD = 1.45) measures; and significant improvements in confidence in managing asthma

| Qualitative and mixed analysis
Of eight parents who consented for follow-up, five (two fathers, three mothers) completed a telephone interview. All self-identified as the child's primary caregiver; had been managing asthma for a period ranging from 1-8 years; and had educational attainments at the high school (n = 1), technical school (n = 3) and university (n = 1) levels. Interviews lasted an average of 36 min (range: 24-41.83 min).
Iterative data collection and analysis confirmed that participants were identifying few usability issues; it appeared unlikely that further incidents would be encountered through additional data collection.
Parents had an overwhelmingly positive response to the eBooks. They consistently reported that the information was clear Participants expressed that the storyline reflected the reality of living with childhood asthma, thereby increasing its relatability.
Specifically, participants noted that the ending of the story, wherein the central character (a 6-year-old boy) achieves asthma control but does not outgrow asthma, was particularly salient, believable (participants 1-5) and hopeful (participant 3). As participant 1 emphasized, the 'growing out of asthma' discourse was prevalent among her family and friends; the concluding sentiments in the eBook provided a needed reflection of the reality of living with asthma. For others, the turning point signified by the asthma clinic referral (participant 4), or the seasonal variations in asthma (participants 1, 5) present in the narrative were particularly reflective of parents' experiences.
One participant identified that the use of multicultural names and characters effectively 'reflects the communities we are living in' and signified inclusivity (participant 5). While other participants did not explicitly identify this as a factor having an impact on relatability, it may have influenced the high quantitative (mean = 7.46) and unanimously positive qualitative relatability ratings among the diverse participant sample.
Namely, all participants, including the fathers, were able to relate to the story, despite the narrator being a mother. As participant 2 explained, he was still able to relate to the narrative because stories need not be interpreted literally and because having a mother as the central character may indeed reflect a reality of the caregiving experience. to keep her occupied while awaiting further treatment. Participants felt it would be easier to refer back to an eBook than paper-based information, and some identified specific content that was most useful to refer back to (e.g. emergency kit, list of triggers).
Most participants preferred the digital delivery to a paper-based format (participants 1, 2, 4, 5). One participant (participant 3) expressed nostalgia for paper-books but concurrently stated that the eBook was a highly useful and convenient modality for asthma education. Similarly, in an open-ended response, one participant requested paper copies so that the information could be re-visited. Overwhelmingly, participants recognized the digital format as appropriate and reflective of the current digital era: a belief that 'everyone has a computer' (participant 4) or would use their hand-held device to access information (participant 2) rendered the eBook a contemporary educational modality. For one participant (participant 5), the digital approach enhanced the eBooks' credibility: 'you see some of the brochures and you just know that they've been there for 20 years and are stale and out of date, not that reliable… because the format was digital you think, okay this is new, or newer, than a dusty old brochure or pamphlet and likely easier to update'.

| The narrative structure aids learning
The storyline helped understand the parent's per-spective… and triggers your memory.

(participant 4)
Participants found the story to be an effective communication approach because it helped organize information in a manner that 'flowed'. The narrative gave structure to information that otherwise would be a series of facts. Parents expressed satisfaction with the quality of information provided; embedding information within a sequential narrative allowed participants to follow and make sense of the information (participants 1-5). In this way, the narrative 'made the 'this wasn't just a generic asthma brochure but was a story of a family.
That reinforced the seriousness of the condition' (participant 5).
The narrative was structured according to the four seasons to reflect seasonal variations in asthma in Canada. This was meaningful for participants, mirrored participants' own uncertainty and variable trajectories and for one participant, 'emphasized the need to be consistent year round' (participant 1). In addition to the seasons, participants often identified the discussion of triggers (participant 1, 4, 5), specifically pertaining to the family pet, as reflective of their own experiences (participant 5). When the storyline did not reflect the specifics of the participant's experience, the eBook was regarded as useful in 'getting the conversation going' (participant 1).

| The narrative and illustrations are synergistic
The narrative and the visuals work together.

(participant 4)
The illustrations enhanced the eBooks' aesthetic and aided in its functionality. Participants expressed that the illustrations contributed to comprehension, engagement and overall appeal (participants 1-5). They referred to the interplay between the story and illustrations, a finding supported by participants' quantitative ratings (mean = 6.97, contribution of drawings to information; mean = 8.1 contribution of story to information; mean = 9.04, ease of understanding), which suggests that the narrative and visual components of the eBook work together synergistically. Indeed, participants reinforced this notion through statements such as 'the storyline triggers your memory… and the pictures help us think' (participant 4).
The illustrations reinforced and complimented the eBook content. Participants emphasized how the illustrations improved information clarity and made the research-based information more comprehensible (participants 1-5); the amount of detail in the illustrations reinforced the key learning points (participants 1, 3 and 4) and at times, helped re-orientate participants to what their children were experiencing. A particularly effective illustration in this respect was a visual rendering of constricted bronchial tubes (Figures 1-4) were 'dead-on' (participant 1), participant 4 found the mother looked exceedingly worried; she expressed concern that this could worry her child if they read the eBook together. Interestingly, the same participant felt that the eBook could emphasize the experience of uncertainty more thoroughly. Overall, the qualitative results support and expand on the quantitative findings: the story contributed more to the information than the illustrations, but both were integral to the effectiveness, efficiency and satisfaction of engaging with the eBook.

| Prototype preference
Color is more attractive and captures our imaginations.

(participant 2)
Participants unanimously preferred the coloured versions of the eBooks (prototypes 3, 4) to the black and white versions (prototypes 1, 2). The black and white versions were seen as 'unfinished' (participant 5), less relatable and the illustration details (i.e. Figures 1-4) were less discernable. Prototype 1 was the least appealing prototype for all participants, followed by prototype 2-the darker lines of prototype 2 were seen to enhance its appeal.
Two of the four participants who ranked the eBooks favoured the 'eye-catching' (participant 1) and 'familiar' (participant 4) graphic novel style of prototype four, while the remaining two preferred the illustration style of prototype three. Participants were divided into whether the comic-like depiction was engaging, or whether it appeared juvenile.
Participants had mixed perspectives about font. Participant 1 preferred the boldness and ease of reading of the comic font (prototypes 2, 4), where participant 2 found the comic font difficult to read, stating, 'there is no sign of where words start or stop'. Two participants commented on how the text font 'matched' the illustration styles for all prototypes; the sans-serif font was regarded as a 'softer, more conversational' match for the diary (participant 5).

| Incidents
We encountered two general incidents. First, 80% of participants did not notice the

| Limitations
There were limitations to our study. We did not conduct real-time, in-person usability testing in a naturalistic environment, which could have revealed different usability issues (Alshaman & Mayhew, 2009).
Determining sample size for usability testing is a continued debate in the field with little sign of resolution (Bastien, 2010), and the qualitative sample was limited to those parents consenting to follow-up.
However, the lack of occurrence of new usability issues attested to appropriate sample size for the study purpose, recognizing that we may have encountered additional usability issues with a larger sample. The quantitative arm did not have a control group and was not randomly sampled, which would have been a more critical issue if assessing effectiveness of the intervention, rather than the usability effect as designed.

| D ISCUSS I ON AND CON CLUS I ON
Participants in our study found the combined use of story and art facilitated engagement, understanding and appeal, thereby supporting the use of a multimodal dissemination approach. Effective stories create an emotional connection with readers (Kirkpatrick, Ford, & Castelloe, 1997) and enable vicarious re-experience-the capacity to experience and feel through representation (Barone & Eisner, 2012).
Visual art can move beyond the confines of language to elicit and evoke different and deeper understandings. Evoking an emotional response is central to the aesthetic experience and to learning more generally; as such, balancing aesthetics with the integrity of the research data are integral consideration with arts-based KT (Archibald et al., , 2018Hartling et al., 2010).
Our findings reflect favourably on using a user-centred design design is fundamental to creating useful and meaningful products (Shoup et al., 2015;Taddio et al., 2013).
There are useful, transferable lessons gleaned from usability testing of arts-based and digitally delivered KT tools. Usability testing provides essential data on whether creative KT tools achieve their purpose. Using an asynchronous approach, we were able to determine whether 'My Asthma Diary' met its purpose while simultaneously identifying key considerations for others interested in developing creative educational materials in other populations and contexts. These considerations are summarized in Table 3.
Central to usability considerations is an understanding of enduser needs (e.g. understood through formative research), fit with purpose (e.g. does it communicate research in a meaningful and comprehensible way?), efficiency (e.g. is the tool an appropriate length, particularly when considering user attributes and contexts of application) and satisfactory display (e.g. is using the tool enjoyable?). Regardless of the creative modality employed, consideration to user-centred design principles is necessary to ensure that KT tools are functional and appealing (Bastien, 2010). This is particularly pertinent given the emergent status of arts-based KT tools, the lack of existing data on how attributes unique to arts-based KT tools have an impact on their use, and the extensive resources (personnel, financial, time) required for their development .

CO N FLI C T O F I NTE R E S T
The authors have no conflicts of interest to report.

E TH I C A L A PPROVA L
The Research Ethics Board at the University of Alberta granted approval for this study. Informed consent was obtained prior to parent's involvement in the quantitative study arm. Parents consented to follow-up qualitative interviews following completion of the questionnaire. Verbal consent was obtained prior to completing the telephone interviews.