Mini-Cog for the diagnosis of Alzheimer's disease dementia and other dementias within a community setting

  • Protocol
  • Diagnostic

Authors


Abstract

This is a protocol for a Cochrane Review (Diagnostic test accuracy). The objectives are as follows:

To determine the diagnostic accuracy of the Mini-Cog for detecting Alzheimer's disease dementia and related dementias in a community setting.

To investigate the heterogeneity of test accuracy in the included studies and potential sources of heterogeneity. These potential sources of heterogeneity will include the baseline prevalence of dementia in study samples, thresholds used to determine positive test results, the type of dementia (Alzheimer's disease dementia or all causes of dementia), and aspects of study design related to study quality.

We will also identify gaps in the evidence where further research is required.

Background

Target condition being diagnosed

Alzheimer's disease and related forms of dementia are common among older adults with a prevalence of 8% in individuals over the age of 65, increasing to a prevalence of approximately 43% in adults aged 85 and older (Thies 2012). Given the increasing number of older adults in most developing countries, the prevalence of dementia is expected to increase considerably in the coming years. Currently, an estimated 5.4 million Americans are diagnosed with Alzheimer's disease and this number is expected to increase to 6.7 million in by 2025 (Hebert 2003). Alzheimer's disease and related forms of dementia are incurable and result in considerable direct and indirect costs, both in terms of formal health care and lost productivity from both the affected individual and their caregivers (Thies 2012). Despite being incurable, there are several benefits to diagnosing Alzheimer's disease and related dementias early in the disease course. Most individuals with dementia, and their caregivers, would prefer to know a diagnosis of dementia, and earlier diagnosis of Alzheimer's disease allows these individuals to make decisions regarding future planning while they retain the capacity to do so (Prorok 2013). A diagnosis of dementia is also necessary to access certain services and support for individuals and their caregivers, and pharmacological treatments such as cholinesterase inhibitors (Birks 2006; Rolinski 2012) or memantine (McShane 2006; Wilkinson 2011) which may help slow the progression of the disease.

The diagnosis of Alzheimer's disease is clinical and based on a history of decline in cognitive functioning in memory and deficits in at least one other area of cognitive functioning (e.g. apraxia, agnosia, or executive dysfunction). There must be a decline from a previous level of functioning which results in significant social or occupational impairment (DSM 2000; McKhann 2011). A definitive diagnosis of Alzheimer's disease can only be achieved at autopsy, but a clinical diagnosis using standardized criteria is associated with a sensitivity of 81% and a specificity of 70% when compared to autopsy-proven cases (Knopman 2001).

Approximately 50% to 80% of all individuals with dementia are ultimately classified as having Alzheimer's disease (CSHA 1994; Blennow 2006; Brunnström 2009). While patients with dementias share common characteristics, subtle differences can help to provide a diagnosis in the absence of neuropathological examination. A smaller proportion of dementias are associated with dementia with Lewy bodies (Brunnström 2009) or Parkinson's disease dementia (Aarsland 2005). Dementia with a mixed etiology is present in 10% to 30% of cases (Crystal 2000; Brunnström 2009; Feldman 2003). Vascular dementias may occur more abruptly or present with a step-wise decline in cognitive functions over time and account for approximately 15% to 20% of dementias (Brunnström 2009; CSHA 1994; Feldman 2003; Lobo 2000). Patients experiencing frontotemporal dementia account for a smaller proportion of dementias (4% to 8%) and often present with problems in executive function and changes in behaviour, while memory is relatively preserved (Brunnström 2009; Greicius 2002).

Index test(s)

The Mini-Cog is a brief cognitive test consisting of two components: a delayed three-word recall and the clock drawing test (Borson 2000). The Mini-Cog was initially developed in a community setting to provide a relatively brief cognitive screening test that was free of educational and cultural biases. Different scoring algorithms were tested to determine which combination had the optimal balance of sensitivity and specificity (McCarten 2011; Scanlan 2001). The Mini-Cog takes approximately three to five minutes to complete in routine practice (Borson 2000; Holsinger 2007; Scanlan 2001). The Mini-Cog has been reported to have little potential for bias in terms of education or language (Borson 2005; Borson 2000).

Clinical pathway

Dementia develops over a trajectory of several years. There is a presumed period when people are asymptomatic although disease pathology may be accumulating.  Individuals or their relatives may first notice subtle impairments of short-term memory. Gradually, more cognitive deficits become apparent with difficulty completing complex tasks such as banking or medication management. At this point, the attribution of cognitive and memory symptoms to normal aging may cause delays in the diagnosis and treatment of Alzheimer's disease. This underscores the need for accurate, brief cognitive screening tests such as the Mini-Cog to help distinguish the cognitive changes associated with normal aging from the changes that might indicate dementia. Most older adults with memory complaints will first present to their general practitioner or other primary health care provider. Primary health care providers may then administer brief cognitive screening tests and, depending on the results of the cognitive screening, an individual may then have additional investigations or cognitive tests to determine if dementia is present. In some settings, a positive result on a brief cognitive screening test may result in a referral to a dementia specialist such as a neurologist, geriatrician, or geriatric psychiatrists. Some countries have also recommended that brief cognitive screening tests be administered to all older adults in order to help screen for undetected or asymptomatic cognitive impairment (Cordell 2013).

Alternative test(s)

We are not including alternative tests in this review because there are currently no standard tests available for the diagnosis of dementia. 

The Cochrane Dementia and Cognitive Improvement Group (CDCIG) is conducting a series of diagnostic test accuracy reviews of biomarkers and scales (see list below). Although the CDCIG is conducting reviews on individual tests compared to a reference standard, we plan to compare our results in an overview.

  • 18F-FDG-PET (positron emission tomography F-2-fluoro-2-deoxy-D-glucose)

  • 11C-PIB-PET (positron emission tomography Pittsburgh compound-C)

  • sMRI (structural magnetic resonance imaging)

  • Neuropsychological tests (Mini Mental State Examination (MMSE); Montreal Cognitive Assessment (MoCA))

  • Informant interviews (Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE); AD8)

  • APOE e4

  • FP-CIT SPECT (fluoropropyl-carbomethoxy-iodophenyl-tropane single-photon emission tomography)

Rationale

Cognitive diagnostic tests are required to assess cognition and assist in diagnosing conditions such as mild cognitive impairment and dementia. Comprehensive neuropsychological evaluation conducted by psychologists or dementia specialists such as general psychiatrists, geriatric psychiatrists, geriatricians, or neurologists would be the most accurate clinical procedure for assessing cognition and diagnosing dementia in older adults. However, these specialized resources are scarce and expensive and as such are not practical for routine use in the evaluation of cognitive complaints (Pimlott 2009; Yaffe 2008). While there are some cognitive tests that can be performed by health care providers who are not dementia specialists, many of these tests are time-consuming and may not be practical to use routinely in primary care or community settings (Brodaty 2006; Harris 2009; Pimlott 2009). As such, brief but relatively accurate cognitive screening tests are required for health care providers in community and primary care settings to identify individuals who may require more in-depth evaluation of cognition.

Utilizing a standard diagnostic test also promotes effective communication between health care providers. The sensitivity and specificity of such tests vary depending upon the setting in which they are utilized (Holsinger 2007). Some studies have found that the majority of older adults with dementia in primary care are undiagnosed (Boustani 2005; Sternberg 2000). In addition, many primary care providers have difficulty in accurately diagnosing dementia, and mild dementia is particularly under-diagnosed (van den Dungen 2011). Early diagnosis and treatment of dementia has numerous clinical and economic benefits for the patient, their community, and the public healthcare system (Bennett 2003; Thies 2012). Accurate diagnosis of dementia is also important in order to initiate dementia therapeutics including both non-pharmacological and pharmacological treatments, such as cholinesterase inhibitors (Birks 2006; Rolinski 2012) or memantine (McShane 2006). A brief and simple cognitive screening test that could be used in routine community settings would allow healthcare professionals or lay people to screen older adults initially for the presence of dementia. Individuals that screen positive for cognitive decline on the Mini-Cog may then be further investigated for the presence of dementia using additional cognitive tests or other investigations. Given that the Mini-Cog is brief, widely available, easy to administer, and has been reported to have reasonable test accuracy properties (Brodaty 2006), it may be well suited for use as a cognitive screening test in community settings. Other cognitive screening tests that may also be suitable for use in the community or primary care settings include the Mini Mental State Examination (Holsinger 2007), the General Practitioner Assessment of Cognition (GPCOG), or the Memory Impairment Screen (Brodaty 2006). The current review will examine the diagnostic accuracy of the Mini-Cog in community settings. Separate diagnostic test accuracy reviews are being undertaken for primary and secondary care settings.

Objectives

To determine the diagnostic accuracy of the Mini-Cog for detecting Alzheimer's disease dementia and related dementias in a community setting.

Secondary objectives

To investigate the heterogeneity of test accuracy in the included studies and potential sources of heterogeneity. These potential sources of heterogeneity will include the baseline prevalence of dementia in study samples, thresholds used to determine positive test results, the type of dementia (Alzheimer's disease dementia or all causes of dementia), and aspects of study design related to study quality.

We will also identify gaps in the evidence where further research is required.

Methods

Criteria for considering studies for this review

Types of studies

We will include all cross-sectional, population-based studies with a well-defined population that utilize the Mini-Cog as an index cognitive screening test compared to a reference standard. Studies must utilize a reference standard to determine whether or not a dementia was present. Studies must utilize the Mini-Cog as a screening test, and not for confirmation of diagnosis. Studies may utilize the test on patients with a previously known diagnosis of Alzheimer's disease or a related dementia. Studies will administer the index and reference tests to individuals where their diagnosis is not already known, when possible.

Participants

Study participants must present in a community setting and may or may not ultimately be diagnosed with Alzheimer's disease or a related dementia. Participants may have cognitive complaints or dementia at baseline although their cognitive status should not be known to either the individual administering the Mini-Cog or the reference standard. Studies on participants with a developmental delay or disability which prevents them from completing the Mini-Cog will be excluded. Studies including participants in either primary or secondary care settings will be excluded as these are the topics of other reviews.

Index tests

Mini-Cog test

The Mini-Cog consists of a three-word recall task and the clock drawing test. The standard scoring system involves assigning a score of 0 to 3 points on the word recall task for the correct recall of 0, 1, 2, or 3 words, respectively. The clock drawing test is scored as being either 'normal' or 'abnormal'. A positive test on the Mini-Cog (i.e. dementia) is assigned if either the delayed recall score is 0 of 3, or if the delayed recall score is either 1 or 2 and the clock drawing test is abnormal. A score of 3 on the delayed recall or 1 to 2 on recall with a normal clock drawing is a negative test (i.e. no dementia) (Borson 2000).

Studies must include the results of the Mini-Cog. If multiple scoring algorithms are utilized, we will explore the differences in results through subgroup analysis.

Target conditions

Target conditions include any stage of Alzheimer's disease or other types of dementia including vascular dementia, dementia with Lewy bodies, Parkinson's disease dementia, or frontotemporal dementia.

Reference standards

While a definitive diagnosis can only be made post-mortem at autopsy, there are clinical criteria for diagnosis of most forms of dementia. All dementia diagnostic criteria require that an individual have impairment in multiple areas of cognition that result in impairment in daily functioning and are not caused by either the effects of a substance or general medical condition. The standard clinical diagnostic criteria commonly used for Alzheimer's disease dementia include the NINCDS-ADRDA for probable or possible dementia (McKhann 1984; McKhann 2011), the Diagnostic and Statistical Manual of Mental Disorders (DSM 2000), or the International Classification of Diseases (ICD 2010). Diagnostic criteria for other types of dementia will include the NINDS-AIREN criteria for vascular dementia (Roman 1993), and standard criteria for dementia with Lewy bodies (McKeith 2005) and frontotemporal dementia (McKhann 2001). The evaluation should also include a number of laboratory investigations, many of which are useful for excluding alternative diagnoses (Feldman 2008). Additional procedures to help confirm the diagnosis include specific findings on neuroimaging (either computed tomography or magnetic resonance imaging). These investigations are typically used to confirm the diagnosis, rather than rule out the possibility of dementia. While these clinical criteria for dementia are considered the reference standard for the purposes of our review, the sensitivity and specificity of these clinical reference standards may vary when compared to neuropathological criteria for dementia (Nagy 1998).

Search methods for identification of studies

Electronic searches

We will search MEDLINE (Ovid SP), EMBASE (Ovid SP), BIOSIS previews (Web of Knowledge), Science Citation Index (ISI Web of Knowledge), PsycINFO (Ovid SP), LILACS (Bireme), and the Cochrane Dementia Group's developing register of diagnostic test accuracy studies. See Appendix 1 for a proposed draft strategy to be run in MEDLINE (Ovid SP) and to view the 'generic' search that is run regularly for the Cochrane Dementia and Cognitive Improvement Group's developing register of diagnostic test accuracy studies. We will design similarly structured search strategies using search terms appropriate for each database. We will use controlled vocabulary such as MeSH terms and EMTREE where appropriate. We will make no attempt to restrict studies based on sampling frame or setting in the searches developed. This is to maximize sensitivity and allow inclusion on the basis of population-based sampling to be assessed at testing (see below, Selection of studies). We will not use search filters (collections of terms aimed at reducing the number needed to screen) as an overall limiter because those published have not proved sensitive enough (Whiting 2011a). We will not apply any language restriction to the electronic searches; we will use translation services as necessary.

A single researcher with extensive experience of systematic reviews will perform the initial searches. Two independent authors will conduct the screening of abstracts and titles.

Searching other resources

We will search the reference lists of all relevant studies for additional relevant studies. We will also use these studies to search the electronic databases to identify additional studies through the use of the related article feature. We will ask research groups authoring studies used in the analysis for unpublished data.

Data collection and analysis

Selection of studies

To be included, studies must:

  1. make use of the Mini-Cog as a cognitive diagnostic tool;

  2. include patients from a community setting who may or may not have dementia or cognitive complaints; and

  3. clearly explain how a diagnosis of dementia was either confirmed according to a reference standard such as the DSM IV-TR or NINCDS-ADRDA at the same time or within the same four-week time period that the Mini-Cog was administered. Formal neuropsychological evaluation will not be required for a diagnosis of dementia.

We will first select articles based on abstract and title. Two independent authors will locate selected articles and assess them for inclusion. We will settle disagreements by involving a third author.

Data extraction and management

Two study authors will extract the following data from all included studies:

  • Author, journal, and year of publication.

  • Scoring algorithm used for the Mini-Cog including cut-points used to define a positive screen. Method of Mini-Cog administration, including who administered and interpreted the test, and their training and whether or not the raters of the Mini-Cog and reference standard were blinded to the results of the other test.

  • Reference criteria and method used to confirm diagnosis of Alzheimer's disease or a related dementia.

  • Baseline demographic characteristics of the study population including age, gender, ethnicity, serverity of presenting symptoms, comorbidity, educational achievement, language, baseline prevalence of dementia, country, APOE status, methods of participant recruitment, and sampling procedures.

  • Length of time between administration of index test (Mini-Cog) and reference standard.

  • The sensitivity and specificity, positive and negative likelihood ratios of the index test in defining dementia.

  • Version of translation (if applicable).

  • Prevalence of dementia in the study population.

Assessment of methodological quality

We will use the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) criteria to assess data quality (Whiting 2011). The QUADAS-2 criteria contain assessment domains for patient selection, index test, reference test, and flow and timing. Each domain has suggested signalling questions to assist with the 'Risk of bias' assessment for each domain. The potential risk of bias associated with each domain is rated as being at high, low, or uncertain risk of bias. In addition, we will determine an assessment of the applicability of the study to the review study question for each domain using the guide provided in the QUADAS-2. We will utilize a standardized 'Risk of bias' template to extract data on the risk of bias for each study using the form provided by the UK Support Unit for Cochrane Systematic Reviews of Diagnostic Test Accuracy. See Appendix 2 for details. We will summarize quality assessment results using the methodological quality summary table and methodological summary graph in RevMan 5 (RevMan 2012).

Statistical analysis and data synthesis

We will perform statistical analysis as per the Cochrane guidelines for diagnostic test accuracy reviews (Macaskill 2010). We will construct separate two-by-two tables for Mini-Cog results for both Alzheimer's disease dementia and all-cause dementia where this information is available.

We will use data from two-by-two tables to calculate the sensitivity, specificity, and positive and negative likelihood ratios as well as measures of statistical uncertainty (e.g. 95% confidence intervals). We will obtain rates of false positives, true positives, false negatives, and true negatives from RevMan. We will present data from each study graphically by plotting sensitivities and specificities on a coupled forest plot. If there are explicit thresholds across included studies we will consider the bivariate random-effects approach for meta-analysis of the sensitivity and specificity (Reitsma 2005). However, we will not restrict meta-analysis to studies that share a common threshold for the Mini-Cog. If multiple thresholds are reported for the Mini-Cog we will use the hierarchical standardized ROC (HSROC) method of Rutter and Gastconis for meta-analysis (Rutter 2001). This model will weight studies according to their inverse variance and account for inter-study heterogeneity, and the threshold effect. We will use the statistical software package SAS version 9.3 to derive summary statistics from either the bivariate random-effects models or HSROC models ROC graphs. We will calculate the diagnostic odds ratio (DOR) for each study. We will calculate the values of sensitivity and specificity for the average operating point. The typical algorithm used to score the Mini-Cog is quite explicit in how the results should be scored. However, where variation from this scoring scheme is encountered in the literature, the analysis plan will change to incorporate multiple average threshold points.

Investigations of heterogeneity

The potential sources of heterogeneity include baseline prevalence of cognitive impairment in the target population, the cut-points used to determine a positive test result, the type of dementia (Alzheimer's disease dementia or all-cause dementia), and aspects related to study quality.

To investigate the effects of the sources of heterogeneity, we will perform a descriptive analysis by visual examination of the forest plot of sensitivity and specificity and the ROC plot. Additionally, we will perform a formal analysis in the HSROC model. This model can be extended to include reliable covariates to assess whether threshold, accuracy, or the shape of the SROC curve varies with patient or study characteristics. If the number of studies is limited, we will investigate the effect of each reliable source of heterogeneity by using covariates to estimate differences in both the accuracy and threshold parameters, while the underlying shape of the summary ROC curve should be assumed to be constant.

Sensitivity analyses

If not already explored as part of the investigation of heterogeneity above, we will perform a sensitivity analysis in order to investigate the influence of study quality on the overall diagnostic accuracy of the Mini-Cog test. In this analysis we will omit studies at high risk of bias.

We will also determine the impact of individual studies on summary outcome measures.

Assessment of reporting bias

We will not investigate reporting bias because of current uncertainty about how it operates in test accuracy studies and the interpretation of existing analytical tools such as funnel plots.

Appendices

Appendix 1. MEDLINE search strategy

The Mini-Cog search will utilize only one search concept: the index test (Mini-Cog):

1. "mini-Cog".ti,ab.

2. minicog.ti,ab.

3. (MCE and (cognit* OR dement* OR screen* OR Alzheimer*)).ti,ab.

4. or/1-3

The MEDLINE generic search run for the CDCIG DTA register:

1. "word recall".ti,ab.

2. ("7-minute screen" OR “seven-minute screen”).ti,ab.

3. ("6 item cognitive impairment test" OR “six-item cognitive impairment test”).ti,ab.

4. "6 CIT".ti,ab.

5. "AB cognitive screen".ti,ab.

6. "abbreviated mental test".ti,ab.

7. "ADAS-cog".ti,ab.

8. AD8.ti,ab.

9. "inform* interview".ti,ab.

10. "animal fluency test".ti,ab.

11. "brief alzheimer* screen".ti,ab.

12. "brief cognitive scale".ti,ab.

13. "clinical dementia rating scale".ti,ab.

14. "clinical dementia test".ti,ab.

15. "community screening interview for dementia".ti,ab.

16. "cognitive abilities screening instrument".ti,ab.

17. "cognitive assessment screening test".ti,ab.

18. "cognitive capacity screening examination".ti,ab.

19. "clock drawing test".ti,ab.

20. "deterioration cognitive observee".ti,ab.

21. ("Dem Tect" OR DemTect).ti,ab.

22. "object memory evaluation".ti,ab.

23. "IQCODE".ti,ab.

24. "mattis dementia rating scale".ti,ab.

25. "memory impairment screen".ti,ab.

26. "minnesota cognitive acuity screen".ti,ab.

27. "mini-cog".ti,ab.

28. "mini-mental state exam*".ti,ab.

29. "mmse".ti,ab.

30. "modified mini-mental state exam".ti,ab.

31. "3MS".ti,ab.

32. “neurobehavio?ral cognitive status exam*”.ti,ab.

33. "cognistat".ti,ab.

34. "quick cognitive screening test".ti,ab.

35. "QCST".ti,ab.

36. "rapid dementia screening test".ti,ab.

37. "RDST".ti,ab.

38. "repeatable battery for the assessment of neuropsychological status".ti,ab.

39. "RBANS".ti,ab.

40. "rowland universal dementia assessment scale".ti,ab.

41. "rudas".ti,ab.

42. "self-administered gerocognitive exam*".ti,ab.

43. ("self-administered" and "SAGE").ti,ab.

44. "self-administered computerized screening test for dementia".ti,ab.

45. "short and sweet screening instrument".ti,ab.

46. "sassi".ti,ab.

47. "short cognitive performance test".ti,ab.

48. "syndrome kurztest".ti,ab.

49. ("six item screener" OR “6-item screener”).ti,ab.

50. "short memory questionnaire".ti,ab.

51. ("short memory questionnaire" and "SMQ").ti,ab.

52. "short orientation memory concentration test".ti,ab.

53. "s-omc".ti,ab.

54. "short blessed test".ti,ab.

55. "short portable mental status questionnaire".ti,ab.

56. "spmsq".ti,ab.

57. "short test of mental status".ti,ab.

58. "telephone interview of cognitive status modified".ti,ab.

59. "tics-m".ti,ab.

60. "trail making test".ti,ab.

61. "verbal fluency categories".ti,ab.

62. "WORLD test".ti,ab.

63. "general practitioner assessment of cognition".ti,ab.

64. "GPCOG".ti,ab.

65. "Hopkins verbal learning test".ti,ab.

66. "HVLT".ti,ab.

67. "time and change test".ti,ab.

68. "modified world test".ti,ab.

69. "symptoms of dementia screener".ti,ab.

70. "dementia questionnaire".ti,ab.

71. "7MS".ti,ab.

72. ("concord informant dementia scale" or CIDS).ti,ab.

73. (SAPH or "dementia screening and perceived harm*").ti,ab.

74. or/1-73

75. exp Dementia/

76. Delirium, Dementia, Amnestic, Cognitive Disorders/

77. dement*.ti,ab.

78. alzheimer*.ti,ab.

79. AD.ti,ab.

80. ("lewy bod*" or DLB or LBD or FTD or FTLD or “frontotemporal lobar degeneration” or “frontaltemporal dement*).ti,ab.

81. "cognit* impair*".ti,ab.

82. (cognit* adj4 (disorder* or declin* or fail* or function* or degenerat* or deteriorat*)).ti,ab.

83. (memory adj3 (complain* or declin* or function* or disorder*)).ti,ab.

84. or/75-83

85. exp "sensitivity and specificity"/

86. "reproducibility of results"/

87. (predict* adj3 (dement* or AD or alzheimer*)).ti,ab.

88. (identif* adj3 (dement* or AD or alzheimer*)).ti,ab.

89. (discriminat* adj3 (dement* or AD or alzheimer*)).ti,ab.

90. (distinguish* adj3 (dement* or AD or alzheimer*)).ti,ab.

91. (differenti* adj3 (dement* or AD or alzheimer*)).ti,ab.

92. diagnos*.ti.

93. di.fs.

94. sensitivit*.ab.

95. specificit*.ab.

96. (ROC or "receiver operat*").ab.

97. Area under curve/

98. ("Area under curve" or AUC).ab.

99. (detect* adj3 (dement* or AD or alzheimer*)).ti,ab.

100. sROC.ab.

101. accura*.ti,ab.

102. (likelihood adj3 (ratio* or function*)).ab.

103. (conver* adj3 (dement* or AD or alzheimer*)).ti,ab.

104. ((true or false) adj3 (positive* or negative*)).ab.

105. ((positive* or negative* or false or true) adj3 rate*).ti,ab.

106. or/85-105

107. exp dementia/di

108. Cognition Disorders/di [Diagnosis]

109. Memory Disorders/di

110. or/107-109

111. *Neuropsychological Tests/

112. *Questionnaires/

113. Geriatric Assessment/mt

114. *Geriatric Assessment/

115. Neuropsychological Tests/mt, st

116. "neuropsychological test*".ti,ab.

117. (neuropsychological adj (assess* or evaluat* or test*)).ti,ab.

118. (neuropsychological adj (assess* or evaluat* or test* or exam* or battery)).ti,ab.

119. Self report/

120. self-assessment/ or diagnostic self evaluation/

121. Mass Screening/

122. early diagnosis/

123. or/111-122

124. 74 or 123

125. 110 and 124

126. 74 or 123

127. 84 and 106 and 126

128. 74 and 106

129. 125 or 127 or 128

130. exp Animals/ not Humans.sh.

131. 129 not 130

Appendix 2. QUADAS-2

DomainPatient selectionIndex testReference standardFlow and timing
DescriptionDescribe methods of patient selection: describe included patients (prior testing, presentation, intended use of index test and setting)Describe the index test and how it was conducted and interpretedDescribe the reference standard and how it was conducted and interpretedDescribe any patients who did not receive the index test(s) and/or reference standard or who were excluded from the 2 x 2 table (refer to flow diagram): describe the time interval and any interventions between index test(s) and reference standard
Signalling questions (yes/no/unclear)

Was a consecutive or random sample of patients enrolled?

Was a case-control design avoided?

Did the study avoid inappropriate exclusions?

Were the index test results interpreted without knowledge of the results of the reference standard?

If a threshold was used, was it pre-specified?

Is the reference standard likely to correctly classify the target condition?

Were the reference standard results interpreted without knowledge of the results of the index test?

Was there an appropriate interval between index test(s) and reference standard?

Did all patients receive the same reference standard?

Were all patients included in the analysis?

Risk of bias:

(high/low/unclear)

Could the selection of patients have introduced bias?Could the conduct or interpretation of the index test have introduced bias?Could the reference standard, its conduct, or its interpretation have introduced bias?Could the patient flow have introduced bias? 

Concerns regarding applicability:

(high/low/unclear)

Are there concerns that the included patients do not match the review question?Are there concerns that the index test, its conduct, or interpretation differ from the review question?Are there concerns that the target condition as defined by the reference standard does not match the review question?

Anchoring statements to assist with assessment of risk of bias

Domain 1: Patient selection

Risk of bias: could the selection of patients have introduced bias? (high/low/unclear)

Was a consecutive or random sample of patients enrolled?

Where sampling is used, the methods least likely to cause bias are consecutive sampling or random sampling, which should be stated and/or described. Non-random sampling or sampling based on volunteers is more likely to be at high risk of bias.

Weighting: High risk of bias

Was a case-control design avoided?

Case-control study designs have a high risk of bias, but sometimes they are the only studies available especially if the index test is expensive and/or invasive. Nested case-control designs (systematically selected from a defined population cohort) are less prone to bias but they will still narrow the spectrum of patients that receive the index test. Study designs (both cohort and case-control) that may also increase bias are those designs where the study team deliberately increase or decrease the proportion of subjects with the target condition, for example a population study may be enriched with extra dementia subjects from a secondary care setting.

Weighting: High risk of bias

Did the study avoid inappropriate exclusions?

We will automatically grade the study as unclear if exclusions are not detailed (pending contact with study authors). Where exclusions are detailed, we will grade the study as 'low risk' if exclusions are felt to be appropriate by the review authors. Certain exclusions common to many studies of dementia are: medical instability; terminal disease; alcohol/substance misuse; concomitant psychiatric diagnosis; other neurodegenerative condition. However if 'difficult to diagnose' groups are excluded this may introduce bias, so exclusion criteria must be justified. For a community sample we would expect relatively few exclusions. We will label post hoc exclusions 'high risk' of bias.

Weighting: High risk of bias

Applicability: are there concerns that the included patients do not match the review question? (high/low/unclear)

The included patients should match the intended population as described in the review question. If not already specified in the review inclusion criteria, setting will be particularly important – the review authors should consider population in terms of symptoms; pre-testing; potential disease prevalence. We will classify studies that use very selected subjects or subgroups as having low applicability, unless they are intended to represent a defined target population, for example, people with memory problems referred to a specialist and investigated by lumbar puncture.

Domain 2: Index test

Risk of bias: could the conduct or interpretation of the index test have introduced bias? (high/low/unclear)

Were the index test results interpreted without knowledge of the reference standard?

Terms such as 'blinded' or 'independently and without knowledge of' are sufficient and full details of the blinding procedure are not required. This item may be scored as 'low risk' if explicitly described or if there is a clear temporal pattern to the order of testing that precludes the need for formal blinding, i.e. all (neuropsychological test) assessments were performed before the dementia assessment. As most neuropsychological tests are administered by a third party, knowledge of dementia diagnosis may influence their ratings; tests that are self administered, for example using a computerized version, may have less risk of bias.

Weighting: High risk of bias

Were the index test thresholds pre-specified?

For neuropsychological scales there is usually a threshold above which subjects are classified as 'test positive'; this may be referred to as threshold, clinical cut-off or dichotomisation point. Different thresholds are used in different populations. A study is classified as at higher risk of bias if the authors define the optimal cut-off post hoc based on their own study data. Certain papers may use an alternative methodology for analysis that does not use thresholds and these papers should be classified as not applicable.

Weighting: Low risk of bias

Were sufficient data on (neuropsychological test) application given for the test to be repeated in an independent study?

Particular points of interest include method of administration (for example self completed questionnaire versus direct questioning interview); nature of informant; language of assessment. If a novel form of the index test is used, for example a translated questionnaire, details of the scale should be included and a reference given to an appropriate descriptive text, and there should be evidence of validation.

Weighting: Low risk of bias

Applicability: are there concerns that the index test, its conduct, or interpretation differ from the review question? (high/low/unclear)

Variations in the length, structure, language, and/or administration of the index test may all affect applicability if they vary from those specified in the review question. 

Domain 3: Reference standard

Risk of bias: could the reference standard, its conduct, or its interpretation have introduced bias? (high/low/unclear)

Is the reference standard likely to correctly classify the target condition?

Commonly used international criteria to assist with clinical diagnosis of dementia include those detailed in DSM-IV and ICD-10. Criteria specific to dementia subtypes include but are not limited to NINCDS-ADRDA criteria for Alzheimer's dementia; McKeith criteria for Lewy Body dementia; Lund criteria for frontotemporal dementias; and the NINDS-AIREN criteria for vascular dementia. Where the criteria used for assessment are not familiar to the review authors and the Cochrane Dementia and Cognitive Improvement Group this item should be classified as 'high risk of bias'.

Weighting: High risk of bias

Were the reference standard results interpreted without knowledge of the results of the index test?

Terms such as 'blinded' or 'independent' are sufficient and full details of the blinding procedure are not required. This may be scored as 'low risk' if explicitly described or if there is a clear temporal pattern to order of testing, i.e. all dementia assessments performed before (neuropsychological test) testing.

Informant rating scales and direct cognitive tests present certain problems. It is accepted that informant interview and cognitive testing is a usual component of clinical assessment for dementia, however specific use of the scale under review in the clinical dementia assessment should be scored as high risk of bias.

Weighting: High risk of bias

Was sufficient information on the method of dementia assessment given for the assessment to be repeated in an independent study?

Particular points of interest for dementia assessment include the training/expertise of the assessor, whether additional information was available to inform the diagnosis (e.g. neuroimaging, other neuropsychological test results), and whether this was available for all participants.

Weighting: Variable risk, but high risk if method of dementia assessment not described

Applicability: are there concerns that the target condition as defined by the reference standard does not match the review question? (high/low/unclear)

There is the possibility that some methods of dementia assessment, although valid, may diagnose a far smaller or larger proportion of subjects with disease than in usual clinical practice. For example, currently the reference standard for vascular dementia may under-diagnose compared to usual clinical practice (reference?) In this instance the item should be rated as having poor applicability.

Domain 4: Patient flow and timing (NB refer to, or construct, a flow diagram)

Risk of bias: could the patient flow have introduced bias? (high/low/unclear)

Was there an appropriate interval between the index test and reference standard?

For a cross-sectional study design, there is potential for the subject to change between assessments, however dementia is a slowly progressive disease, which is not reversible. The ideal scenario would be a same-day assessment, but longer periods of time (for example, several weeks or months) are unlikely to lead to a high risk of bias. For delayed-verification studies the index and reference tests are necessarily separated in time given the nature of the condition.

Weighting: Low risk of bias

Did all subjects receive the same reference standard?

There may be scenarios where subjects who score 'test positive' on the index test have a more detailed assessment for the target condition. Where dementia assessment (or reference standard) differs between subjects this should be classified as high risk of bias.

Weighting: High risk of bias

Were all subjects included in the final analysis?

Attrition will vary with study design. Delayed verification studies will have higher attrition than cross-sectional studies due to mortality, and it is likely to be greater in subjects with the target condition. Drop-outs (and missing data) should be accounted for. Attrition that is higher than expected (compared to other similar studies) should be treated as a high risk of bias. We have defined a cut-off of greater than 20% attrition as being high risk but this will be highly dependent on the length of follow-up in individual studies.

Weighting: High risk of bias

Contributions of authors

BF wrote a draft of the protoco and contributed to revisions of the protocol. DS, SG, NH, NS, CC and VN all contributed to revising the protocol and contributing the final protocol.

Declarations of interest

The authors have no conflicts of interest to declare.

Sources of support

Internal sources

  • No sources of support supplied

External sources

  • Canadian Institutes of Health Research, Canada.

    This project was supported by a Canadian Institutes of Health Research Grant KAL#114493

Ancillary