A realist evaluation of a regional Dementia Health Literacy Project

Abstract Background A Dementia Health Literacy Project was undertaken in the north coast region of NSW, Australia, after it was identified as having a high prevalence of dementia. A Dementia Support Kit was produced with service user engagement to provide useful information to people with dementia and their families. Objective To evaluate the Dementia Health Literacy Project using a realist evaluation framework. Setting and participants The setting was the region of the north coast of New South Wales. Eight people diagnosed with dementia and their carers, 13 members of social groups of older people in the local area, and 22 local GPs and other health‐care and service providers participated in this study. Results Two context‐mechanism‐outcome configurations were identified: (a) co‐design workshops where the stakeholders’ opinions were equally valued (context) led service users to feel listened to and prompted them to provide feedback (mechanism) to develop a practical resource that they would use (outcome); and (b) use of health professionals to distribute the resources (context) that they consider useful and valuable (mechanism) resulted in the target audience receiving the resources (outcome). Discussion and conclusions The Dementia Health Literacy Project produced a Dementia Support Kit that is likely to provide locally relevant and useful information for people with dementia and their carers. The results highlight the value of the co‐design approach in producing and disseminating dementia health literacy resources. Further evaluation is required to confirm the impact of the Kit over time on service users’ behaviour and consequently on their health outcomes.


| Dementia and health literacy
Health literacy is a measure of how well a person can find, understand and use health information. According to the Australian Commission of Safety and Quality in Health Care, health literacy includes an understanding of the "systems, processes, people, information and practices that make up the health and health care industries and sector." 5 Approximately 60% of the Australian adult population are at risk of low health literacy 6 and it is a particular concern among older adults. Surveys in North America found that over 70% of adults aged older than 65 years did not have sufficient health literacy skills and were therefore unable to interact successfully with the health-care system at a time of their lives when they were increasingly dependent on it. 7 Low health literacy is associated with a number of poor health outcomes, including increased rates of chronic illness, early mortality, decreased use of health services and increased costs associated with health care. 8 It is also a risk factor for hospital admission among elderly people. 8 This is true regardless of people's cognitive performance, newspaper reading frequency, health status or level of vision. 9 A study by Clark et al 10  People with dementia are vulnerable to low health literacy. Many older adults are fearful of the disease and try to distance themselves from it. 11 Corner and Bond 12 found that fear of the condition resulted in older adults failing to seek information from health professionals. In dementia, low health literacy is associated with poor advance care planning and poor health outcomes in general. 13,14 This occurs because low health literacy affects how people access and use health care. It also affects relationships between patients and their health-care teams, and the ability of a person to self-manage their condition. 15

| Strategies to build health literacy among people with dementia, their families and carers
People with dementia report receiving little or no information, or unclear printed information about their condition and the services that are available to them. 16 Using evidence-based health literacy strategies can support people with dementia, their families and carers to access, understand and act on the information and services they need. 17 Health literacy best practice has been described as: (a) information that meets health literacy standards, (b) information that is available in a variety of ways and (c) service user involvement: 6

| Information that meets health literacy standards
Health literacy standards include using plain language, consistent font size, left justified text, chunking text into short paragraphs containing only the most important information, and using pictures rather than text where possible.

| Information available in a variety of ways
Health information needs to be delivered in a variety of ways to accommodate a variety of needs. 18 This includes clear written information, visual aids, video decision support tools and online or digital information. 13,14,19 Providing health information by video has also been shown to be an influential medium. 13,14

| Service user involvement
Person-centred care involves partnering with patients and their carers to explore options and select the most appropriate care pathways, to understand the emotional journeys that people experience along their care pathways, and working together to improve these experiences. 20 Partnering with patients and their carers has been shown to improve clinical quality and outcomes, people's experience of care and the business and operations of delivering care. 6 Codesigning health information with service users is likely to produce resources that are readily usable by the target population.
It is recommended that clinicians, health system planners and policy makers promote the uptake of these strategies into routine care to improve health outcomes for all patients, including those at risk of low health literacy. The 2016 North Coast Primary Health Network's (NCPHN) annual Needs Assessment found that access to local services and support was limited by service users' fragmented understanding of available resources. In particular, commissioned dementia services were required in the area. Consequently, service mapping and health literacy were prioritized. A Dementia Health Literacy Project was undertaken to develop a Dementia Support Kit that would meet health literacy best practice standards.
Consultations with an industry-specific advisory group comprising clinical advisors, representatives of government agencies and nongovernment partners working in the area of aged care and dementia suggested that there was a fragmented understanding of locally available services which was impacting on the ability of local services to work together in a systematic and coordinated way. The advisory group reported that those who access services do so late in their disease trajectory, when cognitive impairment is already making it difficult to make informed decisions about proactive planning and self-management.

| The dementia health literacy project
The Dementia Health Literacy Project was undertaken in the north coast region of NSW, Australia, after it was identified as having a high prevalence of dementia. The Dementia Health Literacy Project adopted an experience-based co-design approach to design a Dementia Support Kit to provide useful information to people with dementia and their families and carers. Engaging those with the lived experience of dementia is consistent with the contemporary trend towards co-design and co-production of resources for health-care quality improvement. 23 This co-design approach is a collaborative "method of designing better experiences for patients, carers and staff" 24 by engaging communities, service providers and designers to solve real-world problems. 25 The aim is to work towards, and test, solutions with groups of people who will be directly impacted by these solutions. One of the many benefits of co-design is improved satisfaction because of a better fit between the users and the solutions. 26 Patients can also provide novel solutions that are relevant to their issues because they may be less rigid in their thinking compared to the professionals, although professionals' solutions may be more technologically viable. 27 In any case, the combination of diverse cognitive approaches and different knowledge sets promotes new ideas or the adoption of old ones in new contexts. 28 Co-design challenges include overcoming power imbalances and variations in commitment, 29 allocating sufficient time and resources to co-design, clarifying what is actually negotiable 30 and the nature and timing of service user demands. 26 This paper reports an evaluation of the Dementia Health Literacy Project using a realist evaluation framework. Ethics approval was provided by Southern Cross University Human Ethics Research Committee (Approval number: ECN-17-064). The RAMESIS II guidelines have been used to inform this report. 31

| ME THOD
Realist evaluation is a way to understand how programmes work.
There is an underlying assumption that no programme works for everyone all the time and that the context in which the programme occurs influences its outcome. 28 It aims to investigate how and why an intervention works, for whom, to what extent, in which respects, in what circumstances and over what duration. 32 That is, it aims to understand the underlying causal processes or mechanisms that generate particular behaviours and how people adapt to them. This is achieved by distinguishing salient contexts that are more or less conducive to producing the types of behaviours or adaptations of interest (outcomes).
Outcomes include short-, medium-and long-term changes, both intended and unintended, that arise from an intervention. 28 In realist evaluation, context-mechanism-outcome configurations are identified and can be used to explain outcome pattern variations. 33 It was an appropriate approach for evaluating the Dementia Health Literacy Project because it could provide an understanding of how variations in mechanism and context influenced outcomes, and how those variations could be managed to improve the dementia health literacy of the target population. Realist evaluation begins with a hypothesis that can be generated from a number of sources, including literature reviews and data from large-scale surveys. In this research, a literature review was used to generate the following hypothesis: Engaging service users to co-design health resources that meet health literacy standards increases health literacy, meets community needs and empowers people to feel in control of their health, to access health services and to make informed, shared decisions about their care.

| Data collection
In realist evaluation, no strategy is ruled out in testing the hypothesis. In fact, the accumulation produced by different methods can strengthen the results. Three data collection strategies were used in this study (see Table 1):

| Documentary analysis
A number of documents were collated including a health literacy literature review, the Senior Project Officer's project summary, the Dementia Support Kit and emails and meeting notes between the Health Literacy Project Officer and the research team.

| Semi-structured interviews
Semi-structured interviews were conducted with seven service users who had previously participated in the co-design workshops.
Three were conducted face-to-face; the remaining four were conducted by telephone. Interviews lasted between 30 and 60 minutes each.
The Dementia Health Literacy Project set out to develop a Dementia Support Kit for the Tweed area. It was conducted in two stages: (a) designing a Dementia Support Kit, and (b) piloting and evaluating the Kit.

| Stage 1: Designing a dementia support kit
The

| Stage 2: Piloting and evaluating the dementia support kit
GPs and other health-care and service providers Copies of the Dementia Support Kit were distributed to 24 clinicians and service providers via a local clinical society event and a regional  interviews were conducted with those who were phoned and faceto-face with three members of the co-design group.

| Data analysis
Analysis comprised several steps. First, semi-structured interviews were recorded and transcribed verbatim with participants' consent.
Survey data were analysed using the descriptive statistics functions

| Participants
The evaluation involved the following participant groups:

| Context-mechanism-outcome configurations
Two key context-mechanism-outcome configurations were identified in this study: It will be my "go to" for information.
Everything I need in one place.
Excellent lay-out.

2.
Use of health professionals to distribute the resources (context) that the health professionals consider useful and valuable (mechanism) resulted in the target audience receiving the resources (outcome).
A total of 22 clinicians and service providers completed the Clinicians' Survey: nine from nursing professionals, five from service providers, three from allied health professionals, two from medical practitioners, one from a pharmacist and one from a social worker.
Responses were positive about the Kit, in particular that it provided valuable and easily accessible information for service users with a diagnosis of dementia: • 100% of respondents who reviewed the Dementia Support Kit found it easy to read and understand • 92% of respondents thought that the Dementia Support Kit would be beneficial for their clients' knowledge of services and support.
• 71% of respondents said that they would definitely give a hard copy of this Kit to their clients • 35% said that they were very likely to print a copy of the Kit for their clients; 35% said that they would definitely print a copy for their clients.
Concerns raised by this group included maintaining currency of a printed resource and that printing in their office/clinic would render it black and white and less appealing to read. Vision and colour perception deteriorate with age. 37 Consequently, sufficient colour contrast is needed to enhance readability. 38 Concerns were also raised about accessibility to the Kit using weblinks. Although one in five people aged 65 years in the region reportedly use the Internet to look for health information online 39 links to webpages would not be accessible to those in remote areas with poor Internet facilities. One respondent was concerned that the Kit did not accommodate culturally and linguistically diverse populations. In such cases where the health-care professional did not rate the Kit as valuable, accessible or relevant to the target audience, then they were unlikely to distribute the resource. Table 2 is a realist matrix summarizing the context-mechanismoutcome configurations at work in the Project. 40

| D ISCUSS I ON
This evaluation set out to test the hypothesis that: Engaging service users to co-design health resources that meet health literacy standards increases health literacy, meets community needs and empowers peo- What resources, opportunities, constraints were provided, by whom and in what circumstances?
What reasoning was prompted in response? What changes in behaviour/state of affairs were generated?
Co-design workshops where stakeholders' opinions were valued. Facilitated by Project Officer, with service users diagnosed with dementia, their carers and services workers.
Service users felt listened to and were prompted to provide honest feedback A practical resource that service users were likely to use. Produced A4 booklet and small fridge magnet containing locally relevant information.
Health-care and service providers are well placed to distribute the Kit.
Health-care and service providers considered the Kit to be useful and valuable Health-care and service providers did not consider the Kit to be useful and valuable. Some had concerns about currency, the diminution of the resource if printed in black and white, and its suitability for CALD populations.
The target audience receive the Kit.
The target audience do not receive the Kit.
disease" (p. 3) 11 which could further explain why many did not take the toolkits home with them.
It is important to use evaluation tools that are tailored to the target population. Research projects that involve people with dementia and/or carers need to take their specific needs into account (eg, response times, existence of co-morbidities). This may involve extended timeframes for collecting feedback. For some participants, the time between the first and second rounds of the HLQ was too short for service users to provide meaningful answers to some questions. Moreover, the HLQ may not be the most appropriate tool for this target group. It is a long survey, containing 23 questions. Some questions were perceived as similar by participants. For example, participants may not have easily understood the difference between Question 4: "Feel able to discuss your healthcare concerns with a healthcare provider" and Question 20: "Ask healthcare providers questions to get the information you need." Project evaluation is enhanced when feedback is collected from all stakeholder groups. "Emergent" evaluation approaches, such as qualitative, participatory, empowerment and critical, all highlight the importance of stakeholder involvement in all stages of the project. 41 In such approaches, the evaluation process is "iterative and responsive to changing circumstances and information" (p.1277). The Dementia Health Literacy Project set out to engage key stakeholders at all stages of the project. There were particular challenges associated with a stakeholder group that included people with dementia.
Some could not recall being given the Kit to review or found survey questions confusing. Another challenge was the use of community groups and organizations that had no specific focus on dementia.
Some health service users in these groups did not understand the relevance of the Kit for them. Evaluation was also limited by the small number of medical practitioners (one GP and one medical officer working in a hospital) who participated in the Clinicians' Survey.
The support of GPs and other health-care and service providers is essential for wide distribution of the Kit to its target audience.
Those who perform assessments and make diagnoses of dementia are best placed to distribute the Kit to the right people at the right time. Without such support access could be limited to service users' locating the Kit themselves via Internet searches.
Ultimately, it is the target group's decisions that determine whether the desired outcome is achieved, that is, the interaction between what the Project provides and the reasoning of its intended target population that prompts the desired behaviour. This resource was intended to provide dementia health information in such a way that could influence the target group's decision making. If successful, the resource has the potential to reduce some of the negative health impacts associated with low levels of health literacy, such as increasing rates of chronic illness, decreased use of health services and increased costs associated with health care. 8 The interventions used in this Project-co-design workshops and a pilot and evaluation of the Dementia Support Kit-confirmed the real benefits of service user engagement at every stage of the project. The co-design groups not only strengthened the likely impact of the final product but also made participants feel valued. The Project provided an opportunity for the Project Team to have one-on-one connected conversations with participants. Two causative mechanisms were identified in this evaluation: 1. When service users feel that they are listened to they are prompted to provide honest feedback. It is likely that strong service user engagement at all levels of the project will produce a resource that meets community needs and empowers people to feel in control of their health. Using location-specific and current information (ie, information gaps in the local region that were identified in the needs analysis) can facilitate service users' access to dementia health services in their local area.

2.
When health-care and service providers consider a resource use-

| Impact and sustainability
Further investigation is required to fully assess the impact of the

| CON CLUS ION
The Dementia Health Literacy Project successfully produced a Dementia Support Kit that is likely to provide locally relevant and useful information for people with dementia and their carers. The results highlight the value of the co-design approach in producing and disseminating dementia health literacy resources. The co-design approach that underpinned the Project not only ensured that service users felt listened to and therefore were prompted to give honest feedback, but also ensured its usability by the target group. Health-care and service providers need to appreciate the usefulness of the Kit to their patients and clients if they are to distribute the Kit to the target group. Further evaluation is required to confirm the impact of the Kit over time on service user behaviour and consequently on their health outcomes.

ACK N OWLED G EM ENTS
We acknowledge the support and contributions from Sharyn White from the North Coast Primary Health Network, Taya Prescott from the North Coast Local Health District and Sue Daly.

CO N FLI C T O F I NTE R E S T
The authors declare no conflicts of interest.