Design considerations in long-term intervention studies for the prevention of cognitive decline or dementia

Authors


ILSI Europe a.i.s.b.l., Av. E Mounier 83, box 6, 1200 Brussels, Belgium. E-mail: publications@ilsieurope.be, Phone: +32-2-771-00-14, Fax: +32-2-762-00-44.

Abstract

The results of randomized controlled trials of nutritional interventions aimed at preventing cognitive decline or dementia in older people have been largely disappointing. A reasonable argument can be made that this is at least in part because the design of primary prevention trials in older people is not straightforward and that the complexities of such trials are not readily apparent and commonly remain not fully recognized. This article analyzes some of the difficulties associated with identifying and enrolling study participants in long-term prevention trials, with available data from three large, recently published trials used as examples. This analysis also serves to identify examples of good practice and areas for further research. Randomized controlled trials remain the single most important tool in the epidemiological arsenal for identifying the effects of specific interventions, but consideration of critically important design features is essential.

INTRODUCTION

The increasing prevalence of dementia in all regions of the world1 poses substantial challenges for the biomedical community and society at large. Significant efforts and considerable sums of money have been spent in attempts to identify pharmacological interventions to prevent or treat dementia, but results have been modest.2 There is an increasing body of scientific literature that suggests nutritional factors may be important in the etiology of dementia.3,4

The majority of the available evidence for a link between diet and cognitive health in later life has been gathered from cross-sectional surveys or longitudinal cohort studies. Such evidence is extremely important for the identification of putative causal factors and the generation of testable hypotheses, but it does not serve to establish causal relationships between diet, nutrition, and cognitive performance in older people. Recently, there has also been a rise in the number of randomized controlled trials (RCTs) conducted in this area, and it is generally agreed that large, long-term RCTs are the most reliable means to detect the probably small-to-moderate effects of therapeutic or preventive interventions.5 However, there is growing concern with regard to RCTs examining diseases with long latency periods, such as cognitive change or dementia, that although they are an effective tool for removing residual confounding, they are unable to truly mimic the cumulative risk exposure over the life course.6,7

The performance of long-term trials investigating potential ways to prevent the onset of disease in otherwise healthy volunteers raises a series of questions that researchers conducting trials aimed at improving the health of individuals suffering from specific illnesses or slowing disease progression may not face.

For example:

  • • What level of concern about future health risks exists for healthy people?
  • • Who should be recruited to these studies?
  • • Are healthy volunteers appropriate subjects in terms of representing the population at risk for the condition?

This article describes some of the challenges associated with the conduct of long-term intervention studies for the prevention of cognitive decline or dementia. It focuses exclusively on nutrition-focused RCTs in older people. Arguments are illustrated with data from three large, recently published RCTs: the Folic Acid and Carotid Intima-media Thickness (FACIT) Trial,8 the Ginkgo Evaluation of Memory Study (GEMS),9 and the Older People and n-3 Long-chain Polyunsaturated Fatty Acid (OPAL) Trial10 (Table 1).

Table 1. Summary of studies discussed in this review.
ReferenceStudy populationNo. of subjectsInterventionDuration, yearsCognitive measureOutcome or main results
  1. Abbreviations: HR, hazard ratio; CI, confidence interval.

Durga (2007)8Community-dwelling men and women aged 50–70 (mean 60). Excluded individuals with low (<13 µmol/L) or high (>26 µmol/L) homocysteine. No B-vitamin supplements.818800 µg/d folic acid versus placebo3Cognitive function assessed at baseline and after treatment. Cognitive tests from characterizing: memory, sensorimotor speed, complex speed, information processing speed, and word fluency. Test components: word learning test, concept shifting test, Stroop color-word test, verbal fluency test, and letter digit substitution test.Folic acid improved global cognitive function (average of 5 domains) (mean difference in cognitive change Z-score: 0.05, 95% CI = 0.004–0.096; P = .03). Domain-specific analysis: information processing speed declined in both groups but less in folic acid group (mean difference 0.087; 95% CI = 0.016–0.158; P = .02). Memory improved in both groups, with greater improvement in folic acid group (mean difference 0.132; 95% CI: 0.032–0.233; P = .01). Sensorimotor speed declined in both groups but less in folic acid group (mean difference 0.064; 95% CI = 0.001–0.129; P = .05).
DeKosky (2008)9Community-dwelling men and women aged 75 and older (mean 79) with normal cognition or mild cognitive impairment. Excluded individuals with dementia. No Ginkgo biloba supplements.3069240 mg/d Ginkgo biloba extract versus placebo6.1Cognitive function assessed at baseline and every 6 months after randomization for incident dementia.No evidence of effect of Ginkgo biloba on overall dementia incidence rate (HR = 1.12, 95% CI = 0.94–1.33; P = .21) or on Alzheimer's disease incidence rate (HR = 1.16; 95% CI = 0.97–1.39; P = .11).
Dangour (2010)10Community-dwelling men and women aged 70–79 (mean 75). Excluded individuals with dementia and those scoring <24/30 on the Mini-Mental State Examination. No fish oil supplements.867200 mg eicosapentaenoic acid + 500 mg docosahexaenoic acid/d versus placebo2Cognitive function assessed at baseline and after treatment. Cognitive tests characterizing: memory, processing speed, executive function, delayed recall. Test components: word learning test, story recall, prospective memory, spatial memory, verbal fluency test, search task, digit-span recall, reaction speed, and Digit Symbol Substitution Test.No evidence of effect of n-3 long-chain polyunsaturated fatty acid supplementation on primary cognitive function outcome the California Verbal Learning Test (mean difference −0.5 words, 95% CI = −1.2–0.2; P = .14). No evidence of effect of intervention on any of the secondary cognitive function outcomes (domain-specific cognitive function z-scores).

PARTICIPANT RECRUITMENT

In the ideal scenario, 100% of individuals approached to join a study would agree, and thus the study sample would perfectly match the defined target group; the reality, however, is commonly different.

In the OPAL Trial, the records of adults aged 70–79 years registered at 20 general practices in England and Wales were prescreened; individuals with preexisting diabetes and dementia were excluded, and 13,834 potentially eligible individuals were identified. Letters of invitation to join the study were then sent to 5,309 individuals in batches until the study's sample requirements were met. The invitation letters included a screening question on the daily use of fish oil supplements; 2,304 (43%) individuals responded that they regularly took fish oil supplements and were defined as ineligible to join the study. The remaining 3,005 were identified as potentially eligible, and of these, 1,608 (54%) said they were not interested in joining the study; 225 (7%) individuals did not reply, and 224 (7%) refused for other reasons. In total, 948 (32% of those potentially eligible) expressed an interest in joining the study, of whom 867 (91%) were randomized.10 It took 7 months to recruit the sample of 798 as defined in the study protocol, although to reach the sample requirements of an eye-health component of the study, we over-recruited to 867 individuals and kept study enrollment open for a further 6 months.11

In the FACIT Trial, 40,000 questionnaires were originally sent out to potential study participants identified from municipal and blood-bank registries, and 5,775 (14%) questionnaires were returned. After a series of eligibility checks, 907 individuals were found to be eligible to join the study, and of these, 819 (90%) were randomized. Randomization of the study sample took 18 months.8

The GEMS investigators mailed out 243,000 brochures to potentially eligible individuals identified from voter registration and other purchased mailing lists, but only 47,329 individuals of this sample constituted the true sampling frame for telephone call follow-up because of institutional review board restrictions at three of the four study sites12; 14,609 telephone contacts were attempted, 7,709 telephone interviews were completed, and 3,756 (26% of attempted telephone contacts) attended a screening visit. Of those that attended the screening visit, 3,072 (82%) were randomized. Randomization of the study sample took 21 months.9

It is clear from these three examples that, once eligible individuals are identified, the proportion randomized into the study is high (82–91%), suggesting that individuals who were willing to go through the recruitment process were also generally interested in joining these studies, but the substantially smaller proportion of the total sampling frame of individuals that responded positively to the initial contact from the study team highlights the significant concern that individuals who are eventually randomized into long-term primary prevention studies may not truly match the original target population.

A review of enrollment into dementia-related trials found that recruitment rates varied from 1% to 82%.13 Perceptions of risk and benefits, including the presence or absence of a placebo and the nature of the treatment being tested, appear to affect enrollment rates substantially.13 The number and types of exclusion criteria also affect recruitment rates, and this and other important points have been collated into a set of helpful guidelines for the enrollment of older adults in primary prevention trials.12

Enrollment of older people into primary prevention studies is an enormous effort, requiring substantial human and financial resources, and it is becoming increasingly difficult in research-wary populations, especially minority groups. Identifying recruitment strategies to ensure that participants randomized into long-term primary prevention studies match the original target population, thereby enhancing the external validity of the trial results, remains an important area of future research.

WHO JOINS LONG-TERM STUDIES?

In the OPAL Study, 867 individuals aged 70–79 years were randomized; 55% of the individuals that entered the study were male, more than 18% had the highest level of school qualifications (A-levels) or had been to university, and 56% had hypertension. In people aged 70–79 years in the United Kingdom, the national average proportion of men is 46%,14 19% have A-level or university-level education,15 and approximately 65% of adults have high blood pressure.16 The baseline characteristics of participants in the FACIT Trial and GEMS (Table 2) show a similar pattern of recruiting a disproportionate number of men and those with higher educational status and better health than their age-matched peers.

Table 2. Characteristics at study baseline of participants in the Ginkgo Evaluation of Memory Study (GEMS), the Folic Acid and Carotid Intima-media Thickness (FACIT) Trial, and the Older People and n-3 Long-chain Polyunsaturated Fatty Acid (OPAL) Trial.
CharacteristicGEMSFACITOPAL
  • ‡ 

    College graduate or post-graduate.

  • § 

    Senior vocational or academic training.

  • || 

    Advanced-level or university qualifications.

  • ¶ 

    Self-reported hypertension and concomitant use of antihypertensive medications, systolic blood pressure ≥140 mmHg, or diastolic blood pressure ≥90 mmHg (see Brinkley et al.22).

  • †† 

    Use of antihypertensive medications, systolic blood pressure ≥160 mmHg, or diastolic blood pressure ≥95 mmHg.

  • ‡‡ 

    Systolic blood pressure ≥140 mmHg or diastolic blood pressure ≥90 mmHg.

  • Data from Durga et al. (2007)8 (FACIT), DeKosky et al. (2008)9 (GEMS), and Dangour et al. (2010)10 (OPAL).

Age (mean ± standard deviation)79 ± 360 ± 575 ± 3
Male (%)547155
High education (%)3839§18||
Hypertension (%)5422††56‡‡

The use of criteria to exclude individuals from joining studies may be particularly important in pharmacological trials because of contraindications, but it is also common in nutritional studies in which such issues are unlikely to be a primary concern. The use of extensive exclusion criteria may result in the selection of a healthier subsample of the total population, and some of the baseline characteristics presented in Table 2 are therefore not that surprising. Indeed, the fact that over 24 months of intervention in the OPAL study approximately one third of the expected number of deaths occurred in the randomized sample exemplifies the relative health of individuals recruited into such studies.10

Of potentially greater concern, however, especially for the design of long-term RCTs, is how the outcome of interest changes in this healthier subsample over the intervention period. Sample size requirements for cognitive function are based on existing data on the rate of change in cognitive function or the rate of dementia incidence. These data often relate to general population samples and may therefore not be appropriate for use in select populations, such as those recruited into long-term RCTs.

The GEM Study provides a helpful example of the need for caution when calculating sample sizes. The GEMS investigators used published, national, age-specific dementia rates to estimate a sample size requirement of 3,000 with an average follow-up period of 5 years to detect a 30% reduction in the rate of dementia,17 although the study protocol included a provision to allow continuation of follow-up until the required number of dementia cases occurred. In reality, the dementia incidence rate, especially at the start of the study, was low, and the intervention period was therefore extended from 5 to a median of 6.1 years (maximum 7.3 years) to ensure that the required number of dementia cases occurred.9 This flexible approach to study end dates provides a useful example of how to overcome some of the problems associated with the mismatch between the characteristics of those in the target and randomized samples.

WHO SHOULD COMPRISE THE TARGET SAMPLE?

By definition, there is a logical argument in primary prevention trials that the target sample should, at the start of the study, be free of the health outcome of interest. Therefore, such studies are necessarily designed to delay the onset of, slow the progression toward, or reduce the incidence of ill health. In the OPAL Study, an attempt was made to identify a cohort free of cognitive ill health and test whether supplementation would result in slower cognitive decline in the intervention arm than in the placebo arm. Potential participants were therefore screened at baseline using the Mini-Mental State Examination (MMSE), and any individuals scoring less than 24 out of 30 were excluded. The cut-off score of 24 has traditionally been used as a marker for possible dementia in adults, and although it is possible that individuals with an MMSE score of 24 or greater may have mild cognitive impairment (MCI), the purpose of the proposed cut-off was to exclude individuals with possible frank dementia rather than select a highly functioning group of individuals.11 Of the 948 baseline screening interviews conducted, only eight (<1%) potential participants had a MMSE score below 24, and the median MMSE at baseline of 29 (interquartile range [IQR] 28, 30) attests to a likely good level of cognitive function at study entry.10

The FACIT Trial investigated whether folic acid supplementation would improve cognitive performance in older people and, in contrast to the OPAL Study, had no cognitive function-related exclusion criteria at study entry. This is a sensible approach in a study designed to test whether an intervention will improve cognitive function, because if baseline cognitive health is too good, there is likely to be little room for improvement. In reality, only seven (<1%) of the 818 participants in the FACIT Trial had an MMSE score below 24 at study entry,18 and the median MMSE score at baseline of 29 (IQR 28, 30) was also high.8

GEMS took a different approach, which, again, provides a useful example of a potential direction for future studies. GEMS investigators excluded individuals with diagnosed dementia but actively recruited individuals with defined MCI17; of the total sample recruited (N = 3,069), 482 (16%) had MCI at baseline. The study was designed to test whether Ginkgo biloba would reduce the incidence of all-cause dementia, and the inclusion of individuals with MCI was intended to increase the likelihood of this incidence. Of the 523 individuals who developed dementia, the cumulative incidence rate in those with MCI at baseline was 41% (n = 199), whereas the cumulative incidence rate in those who were cognitively healthy at baseline was only 13% (n = 324).9

The GEMS example identifies a clear advantage in long-term intervention studies of identifying individuals who are at risk of developing the health outcome of interest. There is much debate about the utility of the MCI category, which is variably defined, and some have divided it into subtypes with different rates and types of progression to dementia.19 Furthermore, the use of MCI as an enrollment parameter may confuse potential participants who may be unsure about the meaning of the definition and be concerned that it represents a definitive state of predementia. However, the GEMS example demonstrates there are benefits (for the design of research studies) to being able to identify individuals who are at greater risk of ill health. Other researchers have suggested that frailty in later life might be a useful marker of subsequent poor cognitive health. For example, the ongoing Multi-Domain Alzheimer Preventive Trial enrolled subjects who self-reported to their general practitioner with a subjective memory complaint, had a limitation in at least one instrumental activity of daily living, or had a defined slow walking speed.20

There are several alternative approaches to the identification of asymptomatic individuals with higher risk of developing dementia. These approaches include the use of predictive genetic markers such as apolipoprotein E genotype or measures of familial risk, such as recruiting the adult children of individuals with dementia. The development of biomarkers or early neuropsychological markers is also an area of expanding interest.21

CONCLUSION

Long-term intervention studies for the prevention of cognitive decline and dementia in older people present significant design challenges. Enrollment of samples of older people that match the target population remains a significant hurdle, and the evidence reviewed here suggests this may be hard to overcome. The recruitment of individuals who are, in general, healthier than the target sample, and may also be more motivated and adherent, and have a healthier habitual diet and physical activity patterns, is likely to have a significant potentiating effect on disease progression and may decrease the external validity of study findings. Although primary prevention of cognitive ill health should remain the ultimate goal, there may be some scientific and policy advantages to conducting studies in older people who already show some signs of frailty or poor cognitive health. Identification of nutritional interventions that prevent or slow further decline in these at-risk individuals would provide exceptionally useful evidence for future primary prevention studies.

Acknowledgments

Declaration of interest.  This work was commissioned by the Nutrition and Mental Performance Task Force of the European branch of the International Life Sciences Institute (ILSI Europe). Industry members of this task force are Abbott Nutrition, Barilla G. & R. Fratelli, Coca-Cola Europe, Danone, Dr Willmar Schwabe, DSM, FrieslandCampina, Kellogg Europe, Kraft Foods, Martek Biosciences Corporation, Naturex, Nestlé, PepsiCo International, Pfizer, Roquette, Soremartec – Ferrero Group, Südzucker/BENEO Group, Unilever. For further information about ILSI Europe, please call + 32-2-771-00-14 or email: info@ilsieurope.be. Alan Dangour is the Principal Investigator of the OPAL Study. Funding for the OPAL Study was provided by the UK Food Standards Agency (NO5053). The opinions expressed herein are those of the authors and do not necessarily represent the views of ILSI Europe or the UK Food Standards Agency. The authors declare they have no conflicts of interest arising from the preparation of this manuscript. The coordinator for this supplement was Ms Agnes Meheust, ILSI Europe.

Ancillary