Quality indicators for dementia and older people nearing the end of life: A systematic review

Robust quality indicators (QIs) are essential for monitoring and improving the quality of care and learning from good practice. We aimed to identify and assess QIs for the care of older people and people with dementia who are nearing the end of life and recommend QIs for use with routinely collected electronic data across care settings.


BACKGROUND
The number of older people and people with dementia who have palliative care needs is increasing. 1 The prevalence of dementia at death is around 30% in high-income countries, and with the aging population this number is rising globally. 2,3 Delivering high quality care to meet complex needs arising from the interaction between physical and psychological needs, polypharmacy, clinical uncertainty, and care preferences can be challenging. [4][5][6][7] Monitoring and improving health care of populations require quality indicators (QIs), which can be obtained from routinely collected electronic datasets. Routinely collected electronic data can enable assessment and comparison of the care provided to people over time, across different care settings, nationally and internationally, while avoiding generating additional data collection tasks. 8 QIs enable identification of services that deliver excellent care and those that might require improvement. At an aggregated level, QIs can also be used to assess the effectiveness of new interventions and services. 9 QIs should ideally be accessible from routinely collected electronic datasets, supported by highquality evidence and endorsed by key stakeholders. 8,9 Development and testing of QIs are resourceintensive and time-consuming processes. Numerous QIs have been developed to assess care of older people and those with dementia nearing the end of life. However, the properties and robustness of these QIs have not been systematically synthesized. To our knowledge, no systematic reviews have focused on QIs for the care of older people and those with dementia approaching the end of life. [10][11][12] Although dementia is not synonymous with old age, there are similarities between the two patient cohorts toward the end of life in terms of demographics, and health and social care needs and service use. 13,14 Most people living with dementia are older than 60 years 15 and people with dementia comprise 40%-70% of older adults living in long-term care facilities. 16,17 Older people and people with dementia follow a similar, prolonged pattern of decline toward the end of life. 18 The aims of this systematic review were: (1) to identify and assess the psychometric properties of QIs for the care of older people or people with dementia nearing the end of life and (2) to recommend QIs measurable using routinely collected electronic data across care settings.

Search strategy
We searched MEDLINE, EMBASE, PsycINFO, Web of Science, and CINAHL from inception to February 14, 2020. No study design or language limitations were applied. References of key studies and grey literature (i.e., publications produced by organizations outside the traditional academic publishing such as reports, working papers, and white papers) were searched. Search terms for each database are provided in Table S1. Definitions are provided in Table 1. Eligible studies (Table S2) were those describing the development, review, and/or testing of QIs for the care of adults with dementia and/or those who were older, and who were nearing the end of life.
When QIs were developed and evaluated over time and/or reported across multiple papers, data were extracted from the most recent publication, considering any refinements made. Publications reporting the application of existing QIs to clinical practice were used for reference chaining but excluded from the final list of papers. We excluded studies with adults younger than 60 years. Papers focusing predominantly on cancer and other disease-specific QIs (e.g., chronic obstructive pulmonary disease and end-stage renal failure) were also excluded. Service performance related QIs were excluded. Study authors were contacted to request additional information as required. The review was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta analyses (PRISMA) reporting

Key Points
• We identified 976 quality indicators (QIs); 508 QIs were judged to be robust, 231 moderate, and 237 poor in terms of their psychometric properties. The majority of the QIs were not designed for use with routinely collected electronic datasets. • Few QIs have been developed regarding the legal, ethical, spiritual, and cultural aspects of care. • 71 robust QIs for care of older people and people with dementia nearing the end of life, which can be used with routinely collected electronic data, across care settings are provided.

Why Does this Paper Matter?
The use of robust quality indicators with routinely collected electronic datasets will promote monitoring and improving the care provided to older people and people with dementia approaching the end of life.
guideline (Table S3) and the protocol was registered with PROSPERO (CRD42020170296). 22 Ethical approval was not required.

Study selection and data extraction
Identified references were downloaded into EndNote reference manager for de-duplication. Title and abstract screening was performed by one reviewer (EY), and all papers were double-screened at the full-text stage (EY, JL, LT, and RLC). Papers with ambiguous content or with discrepancies regarding eligibility were discussed with a second reviewer until a consensus was reached.
Data from the included studies were extracted (EY, JG, and JA) into an Excel spreadsheet developed for the review (Link S4). Data extracted about each study included bibliographical information, aim, design, setting, country, population, and data source. Data extracted about each QI were based on a systematic review focusing on end-of-life cancer care QIs 12 and the care domains adapted from the National Consensus Project's guidelines for quality palliative care (developed in the United States and used by numerous organizations). 23 Information retrieved included (1) QI type (process: what care is given and received; or outcome: changes in health status or quality of life 11 ); (2) care domain ("operational"; "physical"; "psychosocial"; "spiritual and cultural"; "communication, advance care planning, ethical and legal"; and "other [including QIs that cover multiple care domains]" 12,23 ); (3) numerator and denominator descriptions; and (4) information on psychometric properties (acceptability, evidence base, definition, feasibility, reliability, and validity). If the information required was not available in the included paper, backward and forward reference checks were made using Google Scholar.

Data analysis
The psychometric properties of each QI were doublechecked and discussed between assessors (EY, JG, and JA) and other author (KES) where necessary. The six psychometric properties (acceptability, evidence base, definition, feasibility, reliability, and validity) have been shown to be important for assessment of individual QIs 24-26 and similar criteria have been used previously for assessing sets of QIs. 27 Assessment of psychometric properties was based on well-established criteria with four possible ratings: positive (positive supporting evidence [e.g., ≥70% of an expert QI development panel, for example, healthcare professionals and policymakers, agree that the QI is valid]), intermediate (doubts about the psychometric property [e.g., some but not all aspects of QI were clearly defined]), negative (disproving evidence about the assessed psychometric property [e.g., QI data collection, analysis, or interpretation not feasible]), and unknown (no published information available to make a judgment about the psychometric property [e.g., reliability of QI was not reported]). 10,12,24 Further information on the definition and assessment of psychometric properties is provided in Table S5.
Ratings of the psychometric properties were used to classify each QI's overall usability as robust, moderate, or weak. QIs with three or more positive ratings across six psychometric categories were classified as robust. The final recommended shortlist included (1) QIs relevant across care settings (e.g., not just hospice), (2) QIs that were not reliant on a specific survey/data assessment tool, (3) robust QIs, and (4) QIs measurable using routinely collected electronic data. We used reference chaining to locate if QIs had been developed or tested for QIs are explicitly defined, measurable items of practice performance that, alongside (review) criterion and performance standards, can be used to judge the quality of care provided by a health or social care organization. 19,20 A QI requires explicit and defined components, including a numerator and a denominator. QIs can be classified by process (the quality of the care process received by the patient) and outcomes (often concerning the clinical outcomes of care). 21 Population level QIs which are applicable across the patient population, and not limited to a subpopulation or a care setting. While measures such as patientreported outcome measures concern the quality of care provided at an individual level, QIs are meaningful measurements when they are interpreted for understanding the quality of care provided at an aggregated level.

Routinely collected electronic data
Administrative data that were not predominantly collected for research purposes, those including electronic medical records, and data collected for insurance purposes such as mandated minimum datasets.
use with routinely collected electronic data such as electronic health records or insurance records. Where duplicate or related QIs were identified (e.g., depression screening for people diagnosed with dementia), the QI with (i) better psychometric properties and (ii) a broader denominator (greater sensitivity; applicable to more people) was chosen. 28
Overall, 508 QIs were judged to be robust, 231 moderate, and 237 weak in terms of their psychometric properties. Of the 508 robust QIs, 315 were unique QIs (i.e., not duplicated in other papers which developed QIs). When categorized into care domains, most of the unique, robust QIs focused on physical aspects of care (n = 204, 64.8%), followed by communication (n = 43, 13.7%), operational • QIs which covered multiple domains of care (e.g., proportion of people who had a list of things which should be covered in the notes included depression assessment, transferring notes on discharge, medications, and previous admissions) Abbreviations: ICU, intensive care unit; QI, quality indicator.

Shortlist of recommended QIs for use with routinely collected electronic datasets
The shortlisted 71 QIs came from seven papers (Table S7). 29 these QIs referred to a process of care, while the rest (n = 27, 23.9%) were outcome QIs. The final recommended QIs concerned physical (n = 29), operational (n = 17), communication, advance care planning and legal (n = 15), psychosocial (n = 9), and spiritual and cultural aspects of care (n = 1) (Figure 2). QIs concerning physical aspects of care varied from use of specific medications, appropriateness of medical interventions, and screening of symptoms and conditions. QIs on operational aspects of care mainly related to admissions, referrals, eligibility for specific care plans, or funding and F I G U R E 2 Shortlist of recommended QIs for use with routinely collected electronic datasets (n = 71) place of death. QIs in the communication, advance care planning, and legal category concerned communication of specific treatments to patients and their families and transfer of information between care settings. QIs relating to the psychosocial aspect of care were mainly regarding the screening and treatment of depression, people's ability to communicate and socialize with others, and included three QIs related to carers. The single QI in the spiritual and cultural aspect of care concerned availability of translators (Table S7).
The shortlisted QIs included 35 (49.3%) which were applicable both to older people and people with dementia nearing the end of life, while 31 (43.7%) were for people with dementia and five (7.0%) were only developed for older people. Forty-three QIs were designed to be used for people nearing the end of life, while 28 QIs were designed to be used for people who had died (Figure 2).

Main findings
In this systematic review, we identified and assessed all available QIs for the care of older people and those with dementia nearing the end of life. We examined the psychometric properties of each QI, and their potential applicability at a population level using routinely collected electronic datasets. We produced a recommended shortlist of 71 QIs with robust psychometric properties which can be used for understanding, assessing, and improving care of older people and those with dementia nearing the end of life. Of the 71 shortlisted QIs, most concerned physical (n = 29) or operational (n = 17) aspects of care, while only one concerned spiritual or cultural aspects of care.
This distribution is similar to findings of systematic reviews of QIs for end-of-life, 10 cancer, 12 or dementia care. 11 Many of the shortlisted QIs in the physical care domain focused on multimorbidities, symptoms, and treatments, which reflect a biomedical approach to care of these populations. 11 These QIs tend to be rated as more robust compared with the QIs in the other care domains. QIs about the operational aspects of care, such as those relating to admissions or eligibility for financial aid, were also common. Operational QIs such as hospital admissions before death are widely used by researchers and policy makers; hence, their psychometric properties may be better established. 67,68 Information needed for understanding spiritual or cultural aspects of care is also less readily available in routinely collected electronic data, but may be captured in other ways such as through patient-reported outcomes and experience surveys. Thus, combining ways of measuring quality is key to gaining a broader overview of the quality of care. 12 Our shortlist also included a substantial number (n = 24) of QIs regarding communication; advance care planning; and ethical, legal, and psychosocial of aspects care. These QIs reflect receiving a diagnosis, discussion of treatments, communication of treatments to other professionals, receipt of treatment, and carers' well-being. While application of the QIs in the communication, advance care planning, and ethical and legal domain may be less straightforward to extract from electronic systems, psychometric robustness of these QIs should encourage further adaption of these QIs into practice. We identified three QIs concerning carers available from routinely collected electronic data. Personal carers often have unidentified and unmet care needs. 69 Using QIs related to carers is a promising starting point for recognizing their needs at a population level.
The smallest number of QIs (n = 19) concerned spiritual and cultural aspects of care. We identified one QI from this care domain for use with routinely collected electronic data, which referred to the availability of interpreters and translated materials. More QIs in the spiritual and cultural care domain have been developed in recent years. 50,62 Their implementation into practice is currently limited. Spirituality and cultural beliefs are intrinsic aspects of person-centered end of life and palliative care and have been highlighted as one of the priority areas for palliative care of people with dementia. 23,70 Building the evidence base and incorporating information regarding spiritual and cultural aspects of care in routinely collected national datasets may support the development of robust QIs in this area.

Characteristics of QIs
Over three quarters (79.7%) of the total identified QIs concerned processes of care, while 20.3% concerned outcomes. This may be because processes of care are often easier to measure and improve. 9 Policy makers and health and social care providers can thus be more in control of setting benchmarks and making necessary changes to improve process QI measurements. Conversely, it is harder to determine the causal relationship between the care provided and the outcome experienced by people, which may explain the smaller number of outcome QIs.
In terms of psychometric properties, more than 50% of the QIs received a "positive" rating for their evidence base, acceptability, definition, and feasibility. Availability of information needed for assessing psychometric properties varied. Evidence base, acceptability, and definition of the QIs were well documented. In comparison, information on feasibility, reliability, and validity was harder to locate among the published work, reflected in higher percentages of "unknown" ratings for these psychometric properties. We also rated just 0.3% of the QIs as "negative" for their validity. Scant reporting of QIs with poor validity is likely to be explained by lack of validation of the QIs after their development. However, some level of publication bias may also exist, and some validation articles might have not been retrieved from our search.
While most QIs (n = 747, 76.5%) were applied prospectively to a population who were likely to be nearing the end of life, a smaller proportion (n = 229, 23.5%) of QIs were designed to be used retrospectively after death. Prospectively identifying a population approaching the end of life is challenging. QIs have been developed which signal potential palliative care needs or risk of poor outcomes such as functional and cognitive impairment and caregiver distress. However, these measures may not be available in routinely collected administrative data or health records and are not collected reliably across care settings. 52 The mandate of standardized minimum datasets across settings may address this limitation in the future. 8 Development of techniques such as linking different datasets and use of text-mining could also maximize the potential of obtaining QIs covering a wider range of care domains from routinely collected electronic data. 71 Nevertheless, controls put in place for safety of patient data, organizational culture, complex governance arrangements including costs, technical barriers, and lack of transparency, and communication about routinely collected electronic data may create unintended barriers for its efficient use. 72

Strengths and limitations
The majority of systematic reviews for palliative care QIs have drawn on evidence from studies involving people with cancer. 10,11 This is the first systematic review to identify and assess QIs applicable to older people and people with dementia nearing the end of life. We used previously applied and recommended ways of assessing psychometric properties of each individual QI, 12 rather than assessing the usability of overall QI sets. This approach enabled us to unpick QIs which were robust and usable within QI sets. All papers were doublescreened at full-text stage, and the bibliographical data extracted and assessment of psychometric properties of each QI were also double-checked.
The evidence base for identified QIs was often unclear. In some cases, this was referred to as "based on existing evidence and expert opinion" without any further details. There is also a chance that information on the psychometric properties of some QIs has not been published. We made efforts by frequent discussions, reference chaining, and contacting the authors where needed, when making decisions on psychometric properties. QIs applicable across care settings have advantages such as comparison and continuity of measurements for a large population and are easier to apply. However, we acknowledge that setting-and condition-specific QIs have their own benefits. QIs, especially those focusing on processes and healthcare utilization, do not necessarily equate to achieving a good quality of care. Measures which may be relevant at a population level may not always translate to each individual's experience and capture issues relevant to patients and families.

Implications for policy, research, and practice
Using QIs can help deliver high quality of care. 9 The number of older people and those with dementia who are nearing the end of life is growing. 1 With advancing technology, we can retrieve more information needed for evaluating palliative and end-of-life care from routinely collected electronic datasets, while minimizing data collection burden. In light of the recent COVID-19 pandemic, the value of having linked datasets across care settings has been highlighted. 8 Our findings should be used to inform the development of infrastructure needed for population-level data collection. Use of the recommended QIs may provide an overview of the quality of care provided to a large proportion of the population who are in significant need of palliative care, yet may consistently experience suboptimal care. Determining how many QIs to implement is difficult. Experts advise on being selective and having a smaller number of QIs per care domain. 9,73 QIs are often pointers for actions needed to improve or maintain the quality of care. Therefore, having fewer robust QIs (rather than many collected through routinely collected electronic datasets) combined with other quality measures is more meaningful for achieving high quality of care. 12 Significant resource is spent for developing new QIs. We suggest that instead of developing new QIs, future research should prioritize the following: (1) revising and adapting existing QIs where possible (especially for physical aspects of care); (2) developing the evidence base for psychosocial, cultural, and spiritual aspects of care and further testing (feasibility, reliability, and validity) of existing QIs; (3) involving patients and families in further development and selection of QIs 74 ; (4) implementing shortlisted QIs into practice 75 at local, national, and international population levels; and (5) combining the interpretation of the QIs with other quality measures such as patient-reported outcomes for monitoring and improving quality of care and enabling learning through comparison.

CONCLUSIONS
We provide a shortlist of 71 robust QIs for older people and people with dementia nearing the end of life that are robust, which can be applicable across care settings and measurable using routinely collected electronic datasets and applied across care settings. Future research should focus on testing and developing psychometric rigor of existing QIs and implementation of robust QIs into practice, to guide our understanding of quality of care provided to these populations, to identify unmet needs, and to improve service provision.

ACKNOWLEDGMENTS
We acknowledge Dr Lesley Henson for her advice in the initial stages of the study.

SUPPORTING INFORMATION
Additional supporting information may be found online in the Supporting Information section at the end of this article.