Accuracy of general hospital dementia diagnoses in England: Sensitivity, specificity, and predictors of diagnostic accuracy 2008–2016

Introduction Recognizing dementia in general hospitals allows for tailored care. We aimed to assess hospital dementia diagnosis accuracy, changes over time, and predictors of correct identification. Method Retrospective cohort study of people over 65 years, using data from a large mental health care database as gold standard, linked to 2008–2016 English hospital data. Results In 21,387 people who had 138,455 admissions, we found sensitivity and specificity of dementia recording, respectively, to be 78.0% and 92.0% for each person's complete records, and 63.3% and 96.6% for each nonelective admission. Diagnostic sensitivity increased between 2008 and 16. Accurate general hospital recording of the presence of dementia was lower in ethnic minority groups, younger, single people, and those with physical illness. Discussion Dementia diagnosis recording in general hospitals is increasing but remains less likely in some groups. Clinicians should be aware of this inequity and have a higher index of clinical suspicion in these groups.


Introduction
There are increasing numbers of people with dementia [1], and they are more frequently admitted to general hospitals than those without dementia [2], due to their greater burden of physical and mental comorbidity, poorer nutritional status, and difficulties managing medication and seeking timely medical care. In the United Kingdom, around two-thirds of people with dementia are thought to have received a diagnosis [3], but this frequently comes late in the illness [4], which limits the provision of appropriate care and opportunities for patients to make future plans at an early stage. Increasing timely diagnosis is part of many countries' dementia strategy [5], and hospital admission may be an opportunity to improve dementia diagnosis.
Recognition of dementia in hospital inpatients is also important as hospital medical records should accurately reflect the person's clinical condition so that tailored inpatient care and discharge plans can be provided, particularly considering the effect of dementia on existing health conditions [6]. People with dementia may forget the contents of the agreed management plan, and lack of mental capacity means that people with dementia are often unable to make health care decisions [7], potentially requiring others to make decisions according to best interests principles [8].
Furthermore, understanding the accuracy of dementia diagnoses in general hospitals will also inform clinical research as routine medical records are an increasingly important method of case ascertainment for epidemiological studies. Hospital data in the United Kingdom have been used to address numerous research questions [9][10][11], but there are concerns about inaccurate or missed diagnoses [12], and previous studies found this to vary according to sociodemographic and clinical characteristics [13]. The sensitivity of dementia reporting in hospital discharge records has been estimated to be 70% in the United States [14], 51% in Finland [15], and between 26% and 43% in Swedish studies [16][17][18]. These studies have used cohort study assessments or clinical examinations as gold standard, but they have been relatively small, and none have examined data later than 2008 nor examined trends over time. Recent health policy to increase timely diagnosis [19] and greater health care professional awareness of the condition may have increased accuracy of subsequent diagnostic recording.
We sought to investigate the accuracy of recorded diagnoses of dementia in general hospitals in the United Kingdom, using data up to 2016. In particular, we aimed to 1. analyze the sensitivity and specificity of dementia diagnosis recording in general hospitals, using secondary mental health care data as gold-standard diagnostic status; 2. examine time trends in sensitivity and specificity of general hospital dementia diagnosis between 2006 and 2016; and 3. explore sociodemographic and clinical correlates of diagnostic accuracy.

Ethics statement
The Oxfordshire Research Ethics Committee C (reference 08/H0606/71 1 5) approved the data resources for secondary analysis.

Study setting and data source
We conducted a retrospective observational study using data from two linked data sets of routinely collected clinical data, described below in sections 2. The Clinical Record Interactive Search (CRIS) data resource provides pseudonymized electronic medical records from South London and Maudsley, one of Europe's largest secondary mental health care providers, which delivers a range of psychiatric care, including dementia assessment and management in memory clinics to a catchment area containing 1.2 million residents in four South London boroughs. Memory clinics are the primary dementia diagnostic service in the United Kingdom whose practice is to take referrals from other health and social care services (usually primary care) of people who have been identified as having possible dementia. There is no routine dementia screening in the United Kingdom.
In CRIS, pseudonymized data are extracted from structured fields in patients' electronic clinical records and from unstructured text within clinical records (including correspondence and case notes) with a natural language processing algorithm using General Architecture for Text Engineering software [20], which generates text strings associated with diagnostic statements. The accuracy of the General Architecture for Text Engineering software is described in detail in a previous publication [21]; it has been found to have precision of 0.99 for diagnosis. The CRIS data set has been used to examine a variety of dementia-related research questions [22,23]. Data are available for all clinical records from January 1, 2006, and CRIS is linked to the Hospital Episode Statistics (HES) database, described in the following.

National Health Service Digital HES
This data set contains clinical information about National Health Service care, collected directly by hospital providers and has been used in a number of research studies [24][25][26]. The data of interest for this study are records of general (nonpsychiatric) inpatient admissions to any hospital in England and the clinical diagnoses recorded on each hospital discharge summary by the treating clinical team. Diagnoses are recorded as International Statistical Classification of Diseases and Related Health Problems, 10th Revision [27] codes and each admission has up to 20 diagnostic codes. The method of admission (elective or nonelective) is also recorded [28]. Diagnoses recorded in HES are those clinically identified during the admission, obtained from correspondence with primary care, or derived from preexisting clinical records such as previous hospital medical records-some record systems prepopulate diagnosis fields with previously recorded chronic conditions. There was no routine practice of dementia assessment in English hospitals until 2012 when the U.K. Department of Health recommended case finding in older inpatients for possible dementia, by asking if any person who was admitted had a change in their memory lasting a year to the extent that it influenced functioning. This would be followed, if dementia was suspected, by referral to memory services [29], although we have no data reporting the extent of adoption of this practice.

Study participants
We retrieved records from CRIS (South London and Maudsley) of all patients aged 65 years or over who had been assessed (as part of ongoing follow-up or as first clinical contact) during the study window from January 1, 2008 to March 31, 2016. We did not include patients whose first electronic record of dementia was during 2006-2007 as we aimed to identify people with newly diagnosed dementia rather than those with a history of the condition. Those whose first CRIS recording was before 2008 would include many whose dementia was diagnosed before the inception of the data set and who were being followed up during 2006-2007. These data were linked to HES records over the same period. All mental health and dementia diagnoses in CRIS were extracted from structured fields in the electronic medical record where clinicians are required to record International Statistical Classification of Diseases and Related Health Problems, 10th Revision [27] codes or from unstructured text using the General Architecture for Text Engineering software, including dementia diagnosis (coded in CRIS as F00x-F03x). We retrieved the dates of, and diagnoses recorded for, each general hospital admission during the study window, including diagnosis of dementia (coded in HES as F00x-F03x, G30x, G31.0, or G31.8). We excluded those who had dementia in their CRIS records but were later diagnosed as having mild cognitive impairment (F06.7), as we judged this to mean these people had the dementia prodrome state [30] rather than clinical dementia.

Covariates
We extracted data from CRIS on participants' age, sex, ethnicity (white, Asian, black African/Caribbean, or other), marital status, and last recorded dementia subtype (Alzheimer's disease; vascular dementia; Lewy body dementias; other dementia [encompassing any other specified dementia type]; and unspecified dementia [where dementia etiology was not recorded]). We estimated the socioeconomic status from the 2010 Index of Multiple Deprivation, which is based on 37 indicators related to the patient's most recent address [31], with a higher score indicating more socioeconomic deprivation. Dementia severity was estimated from the most recently recorded Mini Mental State Examination (MMSE) [32] score at the time of hospital admission. Other aspects of clinical presentation were derived from CRIS using the Health of the Nation Outcome Scale (HoNOS), which is a standard instrument applied routinely in mental health care with adequate-to-good psychometric properties [33]. It comprises 12 subscale rating problems with agitation; self-injury; alcohol/drug use; cognition; physical illness; hallucinations; depressed mood; relationships; daily living function; living conditions; occupation or activities; other problems. Each domain is rated 0 (no problem) to 4 (severe or very severe problem). As 2 is seen to indicate a clinically significant problem, we dichotomized the HoNOS scores in each domain to facilitate interpretation: scores of 0 and 1 were grouped as no/minor problems and scores of 2 indicated problem in that domain. We did not use the cognitive subscale in our primary analyses due to its correlation with MMSE or the "other" subscale due to its nonspecific clinical meaning.
Age, sex, and ethnicity statuses were taken from the baseline recording, and other covariates were recorded at the time closest to the first hospital admission.

Analytic approach
We used the CRIS database record as the gold-standard definition of dementia because it includes records from the area's memory clinics, which are the principal U.K. dementia diagnostic services [19,34] in which people are assessed by trained psychiatrists in consultation with the broader clinical team. Those not seen in memory clinics would usually have been assessed by psychiatrists in other secondary mental health care services. Included patients were all assessed as part of routine clinical practice. They had all received an International Statistical Classification of Diseases and Related Health Problems, 10th Revision diagnosis of dementia (therefore fulfilling standardized gold-standard criteria) or another mental disorder during the study window. Although formalized dementia screening assessment was not administered to all participants, dementia would have been considered as a differential diagnosis for people aged over 65 years with psychiatric disorder, and those with suspicion of dementia would have received standard diagnostic workup. We henceforth describe as "sensitivity" the proportion of people with dementia in CRIS who are correctly identified as having the condition in HES and as "specificity" the proportion of people without a dementia diagnosis in CRIS who are correctly identified as such in HES.
A single cohort would not be adequate to analyze sensitivity and specificity because CRIS and HES assessments rarely take place simultaneously, and for those with CRISdiagnosed dementia, the date on onset is uncertain, and for those without such a diagnosis at their last CRIS assessment, we could not be certain that dementia did not develop later. Therefore, we analyzed people with and without a CRIS dementia diagnosis separately. To assess sensitivity, we examined all HES records after the CRIS dementia index date, which was the date of the first dementia diagnosis in the CRIS database and up to March 31, 2016. For specificity, we examined all HES records from January 1, 2008 and before the CRIS index date, which was the date of last assessment in the CRIS database for people without dementia. All statistical analyses were undertaken using STATA 14.2 (2017).

Sensitivity of HES dementia diagnoses
We calculated the following:

Additional analyses
Twenty-seven percent of people with dementia and 61% of people without dementia had missing data on at least one covariate. To avoid a loss of efficiency, we imputed missing covariate values using multiple imputation by chained equations [35]. Five imputed data sets were created using STATA's mi package by replacing missing values with simulated values from a set of imputation models using a model constructed from all potential covariates and outcome variables. We conducted logistic regression on each imputed data set and combined coefficients using Rubin's rules [36].
We conducted a post hoc sensitivity analysis using the cognitive subscale of HoNOS rather than MMSE because of a large amount of missing MMSE data for people without dementia.

Results
The study sample comprised 21,387 people. Of these, 8246 had dementia diagnosed in CRIS (South London and Maudsley) during the study period and 13,141 did not. The sociodemographic and clinical characteristics of the study sample and percentage of missing covariate data are summarized in Table 1. The mean age at dementia diagnosis was 82.2 years and 60.4% were female. For the people without dementia, mean age at index date was 77.9 years and 55.4% were female. The majority were from white ethnic background and African/Caribbean people formed the largest ethnic minority group. People in the sample were mostly married or widowed and Alzheimer's disease was the dementia subtype for around half of people with dementia and vascular dementia for a quarter. The median time between dementia diagnosis in CRIS and subsequent general hospital admission was 1.4 years (interquartile range 0.5, 2.7 years), and the time between CRIS assessment of people without dementia and prior general hospital assessment was 1.7 years (interquartile range 0.6, 3.5 years).

Sensitivity of general hospital diagnoses of dementia
Of the 8246 people with dementia who were admitted to hospital, 6429 (sensitivity 5 78.0%, 95% confidence interval 77.1, 78.9) had dementia diagnosis at any time in their general hospital records ( In the fully adjusted multivariable model (Table 3), independent predictors of a person with dementia having it detected during subsequent general hospital admissions were increasing age, lower MMSE score, having previously recorded agitated behavior, problem with daily activities, and

Specificity of general hospital records
Of the 13,141 people who did not have dementia diagnosed by CRIS (South London and Maudsley) and who were admitted to hospital before their last contact, 12 (Fig. 1, full data in Supplementary Appendix A.2).
In the multivariable analysis (Table 4), dementia was more frequently entered on HES records of people without a CRIS dementia diagnosis if the person was older, had lower MMSE or problems with agitated behavior, activities of daily living or living conditions, and in those with more general hospital admissions. These identified predictors were also found in our sensitivity analyses accounting for missing data using multiple imputation (Supplementary Appendix B) or the cognitive HoNOS subscale rather than MMSE (Supplementary Appendix C).

Discussion
In this study examining the accuracy of general hospital diagnoses of dementia, we report that overall sensitivity and specificity of hospital dementia diagnoses were 78.0% and 92.0% for each person's complete hospital records and 63.3% and 96.6% for each individual nonelective hospital admission. The rate of dementia diagnosis in HES is increasing but missed diagnosis is more likely in people who are from ethnic minority groups, single, younger people, and those with better cognitive function, less agitation,  or activity of daily living impairment, and with more physical illness. Dementia being entered on HES records for a person who is subsequently assessed by secondary mental health care services without dementia being recorded is also becoming more common over time and is more likely with older age, worse cognitive function and problem with agitation, daily living activities, or living conditions. Having more hospital admissions was associated with higher rate of dementia recording. Our sensitivity estimates are similar to other studies [14][15][16][17][18]. Previous studies have indicated milder dementia is less likely to be detected in data sources [37][38][39][40], while being married [40], female, or living in a care home has also been found to increase the chance of dementia being diagnosed. Our study, in a more ethnically diverse population, adds that unrecorded dementia diagnosis in general hospitals is particularly likely for people from ethnic minority groups, who are around half as likely to have a record of dementia. For some, this may be because of impaired communication between them or their family and the assessing clinician [41]. As dementia awareness is lower in minority ethnic groups [42], patients and their families may be less likely to report the emergence of dementia, and clinicians may misattribute symptoms. Our findings suggest that further efforts are required by clinicians in general hospitals to identify dementia cases in people from minority ethnic groups, by reducing language barrier through use of interpreters, using culturally appropriate cognitive assessments, and potentially targeted case finding. The increased risk of missed diagnosis in the presence of physical illness suggests dementia may be misattributed to physical comorbidity. Our findings that unrecorded diagnosis is also more likely in younger people and those with better cognition and activities of daily living suggest that milder dementia is more often missed. The presence of agitation was probably associated with diagnostic recording as this symptom in an older person can be a trigger for thorough dementia assessment despite the absence of overt cognitive symptoms. We found that single people are less likely to have dementia detected, consistent with previous research findings [13], likely due to the absence of an informant's collateral history. Particular effort should be made to seek supporting information from informants in these groups, and inability to obtain such information should not preclude thorough diagnostic assessment.
Our novel finding of increasing general hospital recording of dementia is important, as recognition of dementia during hospital admissions allows the clinical team to make appropriate adjustments to their communication style, incorporate family members views on health care decisions, initiate specific treatment for dementia's symptoms and consider the effects of dementia on management of other comorbid conditions. The observed increase in recording probably reflects increasing health care professional awareness of dementia, increasing coding accuracy [22,43] and greater communication between primary and secondary care. Furthermore, efforts in 2012 by the UK Department of Health to increase diagnosis rates in secondary care by case finding in older admitted people [29] may have also increased diagnosis in general hospitals, as supported by our finding of increased diagnostic sensitivity during that year.
Our specificity estimate of 92% was lower than figures of 98% [18] and 99% [16] from other studies. However, our analysis of specificity should be interpreted with caution, and the true figure may in fact be higher. Our analysis is based on a cohort of people in contact with secondary mental health care services who may be more likely than a general population to have symptoms resembling dementia. "Falsepositive" dementia diagnosis (i.e., diagnosis in HES when later assessment did not result in dementia diagnosis in CRIS) is a possible unintended consequence of the drive for earlier dementia diagnosis and potentially harmful. However, we found that older age, worse cognitive function, and problem with daily living activities and agitation predicted "false-positive" recording of dementia, and these are hallmarks of dementia, so some of these may actually represent correct diagnosis of dementia in the general hospital and incorrect diagnosis (i.e., failure to detect dementia) by CRIS, in which case the specificity is underestimated.

Strengths and limitations
This is the largest and most up-to-date analysis of hospital register dementia diagnoses, with sufficient data to allow the first analysis of changes in accuracy over time. We used a very large secondary care mental health register as gold standard against which to test accuracy of general hospital diagnosis, with natural language processing used to increase the accuracy of the CRIS register by picking up people whose diagnosis had been written in text records rather than in structured diagnosis fields.
Missed dementia in the CRIS record is possible, although it is based upon the assessment of trained psychiatrists from dementia services. We therefore restricted our sample to people aged over 65 years whom the mental health care service would have been likely to assess for dementia. As the CRIS data source is retrospective, we are not able to validate its accuracy by assessing participants, as used in other studies [44] as it would rely on information, in particular collateral history and cognitive examination, obtained for individual patients. Records are likely to be written in a way that reflects the clinician's overall clinical impression. Missed dementia diagnosis in CRIS may mean that sensitivity in this study is overestimated-we expect that people with dementia whose condition was missed in CRIS would also be more likely to have missed diagnosis in HES-and that specificity may be underestimated, as described previously.
National dementia recording rates are estimated to be around 72%, and estimates for people in this study's catchment area are similar (75%) [3], meaning that CRIS records will miss people with dementia because they have not presented to services. For individuals never seen in secondary mental health care services, therefore not in our CRIS cohort, HES diagnostic sensitivity may be worse as they may be more likely to have characteristics associated with lack of HES dementia recording. Finally, our sample was derived from a specific region in urban and suburban London, which could limit representativeness. However, this area has considerable ethnic and socioeconomic diversity, which allowed us to examine the effect of these factors on dementia recording, and the hospital records were from all of England, so our results are likely to reflect a range of hospital diagnostic practice.

Clinical implications and future research
UK efforts to increase dementia diagnosis rates in general hospitals have had success, but there is lower recording rates in some groups, likely due to communication difficulties, lack of an informant, or the presence of other causes of cognitive decline. It is therefore important that clinicians are aware of this inequity, and that they have a higher index of suspicion in these patient groups. Policymakers should consider more targeted case-finding approaches and providing training for hospital clinicians in dementia detection in these patient groups. Better sharing of diagnostic information between health care providers, such as automatic population of hospital databases with previously diagnosed conditions, would increase clinician awareness of comorbid conditions including dementia. Future prospective research should seek to identify in more detail the effect of factors such as native language, the presence of an informant, and physical comorbidities on dementia diagnostic accuracy.
Our study also clarifies the validity of hospital episode statistics as a tool for epidemiological and clinical research, and we found higher sensitivity than previous studies. We note the dynamic of increasing dementia recording over the past 9 years and that more hospital admissions improve diagnostic accuracy. However, using HES for case ascertainment may create systematic bias, especially with people from ethnic minorities, in whom dementia will be underestimated. These factors should be taken into consideration when researchers use these records.
G. Livingston. All authors read and approved the final article. Due to the data management requirements of the pseudonymized data source, no additional data are available for sharing. The lead author affirms that this article is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained. The lead author in this statement is the study guarantor.

RESEARCH IN CONTEXT
1. Systematic review: We systematically searched the literature and found six studies examining general hospital record dementia diagnostic accuracy, using data from before 2008, with sensitivity between 28% and 70% and specificity between 94% and 99%.
2. Interpretation: Our study is the largest and most recent analysis. The sensitivity and specificity of general hospital dementia diagnoses was 78.0% and 92.0%, respectively, for each person's complete hospital records and 63.3% and 96.6% for individual hospital admissions. Dementia recording increased between 2008 and 2016. Dementia was more likely to be missed in minority ethnic, older, single people, and in those with milder dementia, non-Alzheimer's dementias, or physical illness.