Supporting the involvement of older adults with complex needs in evaluation of outcomes in long‐term care at home programmes

Abstract Background It is important to involve older people in evaluating public programmes that affect their lives. This includes those with physical and cognitive impairments (such as dementia) who may need support to live at home. Many countries have implemented new approaches to support older people to live well at home for longer. However, it can be challenging to involve disabled people in service evaluation, so we are unclear whether services are meeting their needs. Aim This study explored how a cascading methodology, offering different supports enabled the involvement of home care users with cognitive and physical impairments in the assessment of their care‐related quality of life. Method We used multiple tools from the Adult Social Care Outcomes Toolkit (ASCOT) with n = 63 older adults who were recipients of home care in the Illawarra. We also offered different physical and cognitive supports as needed. Results We started with the standard ASCOT questionnaire to assess the care‐related quality of life, but then offered alternative formats (including Easy Read) and supports (including physical and cognitive assistance) if the older person needed them to participate. This allowed us to involve a greater diversity of older people in the evaluation, and changed what we found out about whether their care needs were being met. Conclusion There is a need to implement more flexible and inclusive methods to increase the involvement of vulnerable users of long‐term care in the assessment of service outcomes. This is important to ensure that the perspectives of all service users inform the delivery of person‐centred care. It is also critical to understand the extent to which programmes are meeting the needs of vulnerable service users. Patient or Public Contribution Service users with dementia were involved in the design of the ‘Easy Read’ questionnaire used in the study.


| INTRODUCTION
With the global ageing of the population it is becoming increasingly important that we find ways to engage and involve all older people in policymaking and evaluating public programmes that affect their lives. 1 However, involvement and engagement can present challenges with large numbers of older people living with chronic and degenerative conditions including dementia. 1,2 One key area that could benefit from greater patient involvement is the evaluation of aged care policies and programmes. Of particular need are those that emphasize ageing at home, with longterm care services delivered in community settings. 3,4 Reforms in aged and disability care in many countries have supported the introduction of consumer-directed care (CDC) models on the basis that they can improve autonomy and choice for service users. [5][6][7][8] However, there is mixed evidence concerning the effectiveness of these models, [8][9][10] especially with regard to supporting vulnerable service users, such as those with dementia, mental health issues, financial or social disadvantage and/or low literacy. 6,7,[11][12][13][14] One reason that we lack clear evidence about the effectiveness of different models of care in this group is the failure to use inclusive research methods to evaluate service-related outcomes. Service evaluation from the patient perspective is especially complex given the vulnerability and diversity of impairments of users of aged and disability services. Previous research has highlighted significant challenges with establishing reliable methods to collect care outcome data for people ageing with cognitive and communication impairments. For example, the cognitive impairments inherent with dementia present challenges for recruitment, 15 managing consent 16 and engagement in research processes. 16,17 Impairments can also make results difficult to interpret due to vagueness in speech, decreased vocabulary, poor reasoning of verbal information, confabulations or 'pseudo-reminiscences', perseverations, and confused word associations. 18,19 There is, however, a growing recognition that, with the right support, people with dementia are capable of expressing their views, needs and concerns. 20,21 It is recognized that understanding the experiences of people with dementia is important for evidence-based programme and service delivery. 17,22,23 This is especially so in a CDC aged care market where the success of programmes must be understood through the rubric of meeting peoples' preferences and needs.
There is a need for methodological innovation to support the development of valid, accessible and reliable assessment approaches that promote the involvement of older populations living with cognitive and communication impairments-who are the key audiences for these programmes. For people with dementia, new tools have been developed to support assessment of the health-related quality of life, which have been designed to capture the perceptions of people with dementia as they relate to their physical, mental, social and health status. 24 However, for high users of care services in the home or for those who live in residential aged care services, there is also a need to capture people's views about the aspects of their quality of life most impacted by care services. This is called social care-related quality of life (SCRQoL). 25

| Assessment of ScRQoL
The Adult Social Care Outcomes Toolkit (ASCOT) is a valid and reliable measure focused on areas of quality of life that can be attributed to care services-SCRQoL. 25 The eight domains cover the core or lower order domains, including personal cleanliness and comfort, accommodation cleanliness and comfort, food and drink, feeling safe and also higher-order domains, including social participation, occupation and control over daily life. The eighth domain, dignity, asks respondents to consider how treatment by care staff makes them feel. 25 These domains are directly relevant to the goals of the CDC Home Care Package (HCP) in Australia, the country in which this study was conducted. 4 For each domain of ASCOT, there is one item with four response options, relating to four conceptual outcome states (ideal state, no unmet needs, some unmet needs and high unmet needs). ASCOT has been used extensively to assess care-related outcomes in community-dwelling service using populations in the United Kingdom, 26-28 Europe 29-31 and more recently in the Australian older population. [32][33][34][35] Despite its high validity and reliability, 36-38 barriers exist to engaging sections of the aged and disabled cohort to report on their own outcomes. 39 As such, it is important to better understand the value of utilizing different formats and degrees of support to better represent the direct perspective of clients with cognitive and communication impairments.
This study outlines the value of alternative methods to promote involvement in the assessment of care-related outcomes that is inclusive of the voice of the most vulnerable service users. In this study, we trialled the use of two alternative ASCOT questionnaire formats within a cascading inclusive methodology that offered assistance and support as it was required. The aim was to understand what supports older service users with varying degrees of cognitive and physical impairments needed to participate in reporting their own care-related outcomes, and the value of this reporting to illuminate the service use experiences of this vulnerable cohort. 4. Are some service users unable to be supported to report against their own outcomes using the proposed methods?

| METHODS
This study explored the utility of two versions of ASCOT to promote the inclusion of a greater diversity of service users to selfreport their current SCRQoL. The ASCOT SCT4 is a standard, selfcomplete questionnaire, designed to assess current SCRQoL in the context of receiving care-related services with four response options. 25 The ASCOT-Easy Read (ASCOT ER) is a modified version of the ASCOT SCT4, initially developed for use with people with autism and intellectual disability, 40

| Recruitment and procedure
The study was conducted in collaboration with a service provider for community-dwelling older people with complex needs through the Australian government-funded HCP Programme. 42 Inclusion criteria were current receipt of HCP supports by this service provider and being resident in the Illawarra-Shoalhaven region of NSW (Australia).
The service provider had been previously involved in the adaption of the ASCOT-ER tool for use in the older population 41 and was keen to understand to what extent the ASCOT-ER would enable the greater voice of their clients reporting on their care-related outcomes.
The service provider made initial contact with n = 299 potentially eligible participants through a mailed out research information pack.
The pack included a letter from the provider encouraging participation in the research. It contained a plain language information flyer that outlined the aims of the research and what would be involved for those who were willing to take part. Potential participants were advised, if they consented to take part, a researcher would make arrangements to come to their home to discuss their experiences and test two different questionnaire formats. The flyer included a photograph of the lead researcher to create a personal connection with potential participants, and invited them to make contact via phone, email or mail if they were willing to participate.
In response, 63 (n = 63) potential participants (or their carers) made contact by phone or email. The first author answered their questions and provided more information about the study. This included advising them that they would be sent a self-complete questionnaire (the ASCOT SCT4), but if they could not complete it on their own they could do it with support when they met with the researcher face-to-face. At this point in the process, 14 (n = 14) participants either declined or were deemed ineligible to take part due to not meeting the inclusion criteria. One participant, whose carer had initiated the phone contact, was deemed unsuitable for participation due to their advanced dementia and was identified at this point as needing alternative methods to assess their outcomes.

| Ethics approval and consent process
Approval for the conduct of the study was provided by the University On commencement of the research interview, all participants were supported to read through the information, ask any questions and discuss their understanding with the researcher. In any cases where participants showed any confusion, written proxy consent of a care partner was also obtained. During the interviews, all participants were also observed to monitor process assent through their willingness and interest to discuss and answer the questions. 16 Demographic, functional and cognitive data was then collected directly from participants in a supported interview format. Ths which included age, language spoken other than English, gender, carer status, carer coresidence, education level, self-reported diagnosis of dementia, monthly family finances and level of HCP. Cognitive status was screened using the Mini-Cog© (a score of <3 was used as an indication of cognitive impairment). 44,45 General functional ability was assessed using the Home and Community Care (HACC) functional screener. This tool has a maximum total score of 16, with a lower score indicating more difficulty managing daily activities of living. 46,47

| The cascading methodology
The interviews conducted within this study acted as both a way of collecting data about the experience of completing either version of the ASCOT questionnaire and, providing support and assistance to those who needed it to ensure their participation in the study was 'manageable'. Whether or not the participants were able to selfcomplete the ASCOT SCT4 directed the way the method cascaded during the research interview. If participants were able to independently complete the ASCOT SCT4, they were subsequently interviewed about their responses. If assistance to complete the ASCOT SCT4 was required on the basis of visual or physical impairment, this was provided by the researcher. However, if assistance was required due to the comprehensibility of the questionnaire, they were instead provided with the Easy Read format (ASCOT-ER). Assistance was again provided on an 'as needed' basis, but this time included support for comprehensibility, as well as to aid the focus and/or orientation of participants to promote complete responses for each domain if required.
Detailed field notes were also taken by the researcher regarding the types of assistance and support required by all participants to complete the ASCOT SCT4 or ASCOT-ER questionnaires.

| Data analysis
All interviews were audio-recorded and transcribed. Transcripts and field notes were placed in NVivo 11 and deductively analysed, using the framework for research cohesiveness to identify the supports that were required to support a manageable, meaningful and comprehensible assessment experience for the participants. 41 Participants were classified as either being 'independent' or 'assisted' for three parts in the research process: 1. Replying (as part of the recruitment process) 2. Providing consent and 3.
Completing the ASCOT questionnaire. This was further analysed to identify whether the assistance was required to 'manage' the demands of the process (e.g., holding a pen, writing or talking on the phone, maintaining focus on topic, prompting to complete responses for each domain) or related to 'comprehensibility' (e.g., ability to make sense of the domain through the text explanation or the pictorial). Negative scores indicate a state worse than death. 10 The ASCOT-ER differs from other questionnaires in the toolkit by asking two separate questions in the 'Safety' domain.

| Statistical analysis
This allows participants to distinguish between safety in the home and that experienced in the local area. To enable the calculation of an overall current SCRQoL score using the weighted preferences, and comparability with the other ASCOT measures, in this study, the 'Safety' score indicating the highest unmet need for each participant was utilized. Further information about the ASCOT tools, resources and scoring can be found online. 48

| RESULTS
Overall, there were 63/299 responses to the mailed research invitation (21% response rate). However, n = 15/63 of these chose not to take part in the study. A total of 14/15 of these responses were family carers advising ineligibility (e.g., due to the participant being in hospital or admitted to residential care). One other response, also from a carer, confirmed that the HCP recipient was immobile and nonverbal. As such, this participant was identified as needing alternative methods to assess their outcomes.
In total, 48 (n = 48) respondents agreed to participate. Five did not complete the study requirements due to sickness (n = 1), unwillingness to trial alternative research methods (n = 2) or identification of the need for other assessment methods during the research trial (n = 2).

| Demographic characteristics of participants
Overall, demographic, functional and current SCRQoL data were obtained from n = 43 participants. The demographic characteristics of those able to rate their current SCRQoL through completion of either the ASCOT SCT4 or the ASCOT-ER are displayed in Table 1.
Participants in the study were mostly: old (mean age: 84.2 years), female (69%), supported by a family carer (67%; 23% coresident), had more than a high-school education (53%) and reported adequate financial circumstances (usually had some money left over at the end of each month) (65%). People who spoke a language other than English were not well represented and less than 25% of respondents were receiving the highest level of care package (Level 4).

| Value of support to make assessment more manageable for all service users
Overall, 35/48 participants made an attempt to complete the ASCOT-SC4 self-complete survey. Of those, 24/35 were able to complete the SC4 independently. For the other 11/35, participants requested physical assistance due to functional impairment (e.g., visual, physical, fatigue). Others also requested support due to a reported lack of confidence, or a desire to clarify the meaning of some questions before choosing a response.
For those 12/48 who were unable or unwilling to attempt completion of the ASCOT-SCT4, 8/12 were successfully assisted by the researcher to complete the Easy Read format, 2/8 were unable to complete even with assistance and 2/8 declined to make an attempt even with support.
For those eight participants who were aided by the Easy Read format, success was associated with the researcher providing support Finally, there were three participants who were unable to be supported to use the Easy Read format. One participant was identified when their carer replied to the research invitation, and two others were unable to be successfully supported during the trial.
See Table 2 for details of needs for assistance and support for successful completion during the various stages of the research and assessment process.

| Value of easy read format to make assessment more comprehensible
The flowchart in Figure 2 of the cascading inclusive methodology shows how it allowed for more people with cognitive impairment to complete the questionnaire than if the study relied only on the standard questionnaire format (ASCOT SCT4).
Statistical analysis showed significant differences in the profile of those who were able to be supported to complete the ASCOT SCT4 versus the ASCOT-ER. That is, those who could not complete the ASCOT SCT4 but were successfully supported to complete the ER format to rate their current SCRQoL were older (p < .000) and also more cognitively (p < .001) and functionally impaired (p = .018).
While the size of the study prohibited statistical analysis of differences, comparisons of descriptive data indicate there were also some differences in the self-reporting of met and unmet needs across the eight domains for the two groups (ASCOT SCT4 vs. ASCOT-ER).
For the ASCOT-ER group, scores in seven of the eight domains, and their overall current SCRQoL scores, were lower than those who were able to complete the ASCOT SCT4 (see Table 3 for details). The two domains where self-reported needs were higher for the ASCOT-ER group than for the ASCOT SCT4 group were 'Personal cleanliness and comfort' and 'Social participation'. The exception was for 'dignity in care', with the ER group indicating more 'ideal' and 'no needs' states than the ASCOT SCT4 group that is those with more cognitive and functional impairment rated the kindness and respect with which they were treated by care staff as in a mostly 'ideal state'. For both T A B L E 1 Characteristics of participants using ASCOT SCT4 versus ER

| Additional supports needed to promote more manageable assessment
In this study, the successful completion of the ASCOT questionnaires in both formats (ASCOT SCT4 or ER) benefitted from the support of the interviewer. The cascading inclusive methodology provided value to participants who needed both physical and/or cognitive support.
In addition, unlike other inclusive methods that propose a one-sized fit all adjustment (such as an Easy Read format for all participants), the cascading inclusive methodology adjusts according to need. For example, almost a third who completed the standard format (ASCOT SCT4) also benefitted from assistance to make their participation more 'manageable'. This was mostly to support physical impairments (e.g., vision, physical limitations, fatigue) though some also reported a lack of confidence, or a desire to clarify the meaning of some questions before choosing a response. In contrast, for those requiring the ER format, assistance was required by all participants and was frequently more intensive, including support to maintain orientation, focus, comprehension and completion of the questionnaire.
The need to adapt methods and provide support for people with dementia to participate in research is established. 51,52 Adaptation is useful, specifically for older people with cognitive impairment to promote meaningfulness in the context of their life stage. 41 This study reinforces that the ER format was useful to improve 'comprehensibility' of the domains, the use of a supported interview protocol was also essential to ensure engagement with the questionnaire was 'manageable'. 41

| Limitations and future research
This study was conducted for the purposes of understanding the methods that may be required to increase the involvement of vulnerable older people in reporting outcomes from their aged care services. It was conducted in a small sample of home care users from a service provider in a single geographical region in Australia. The study was very time intensive and the researchers required dedicated training to provide the needed support. This may limit the future transferability of some of these methods beyond the research environment. Future research is therefore required to apply these innovative methods to assess outcomes in a representative sample of service providers and users in different geographical locations. The results obtained about users SCRQoL may not be representative of the population of the provider, the region or the broader home care population. However, this was not the aim of the study.
We had limited success in assessing the outcomes of people with more moderate or advanced dementia. This suggests there is still a further need for methodological innovation to promote their inclusion in service evaluation. One approach that may have merit is the multi-methods ASCOT CH4 54 -which has been used to promote greater inclusion of the voice of older people with cognitive and communication impairment living in institutional/care home settings but has not yet been feasible in a community setting.
The use of a mail out invitation to take part in the research was also not useful in supporting participation in home care users who speak languages other than English, those without the support of an informal carer and those with more moderate and advanced dementia. As such, alternative methods for recruitment are needed to overcome the challenges of recruiting people with dementia for service evaluation research, even when using multiple methods. 15 Future research should focus on supporting larger-scale research in a representative population. It should also consider the value of training service providers to adapt cascading supportive approaches to support their use of assessment tools as part of routine care planning and service evaluation. Finally, further methodological innovation is needed to ensure that the preferences and needs of those living in community settings with more moderate and advanced impairments are understood and met.

| CONCLUSION
Overall, this study highlights the value of a cascading inclusive methodology that has methods and supports that adjust to the cognitive and physical needs of community-dwelling older people with complex needs. Offering people with cognitive and physical impairments a range of methods and supports to meet their needs allowed us to include the direct perspective of vulnerable people in reporting on their own needs. This is crucial if we are to evaluate the success of programmes and policy changes that aim to support older people living with cognitive and/or communication impairments as one of their primary target groups.