Criteria for validation and selection of cognitive tests for investigating the effects of foods and nutrients

Authors


Correspondence: ILSI Europe a.i.s.b.l., Avenue E. Mounier 83, Box 6, B-1200 Brussels, Belgium. E-mail: publications@ilsieurope.be, Phone: +32-2-771-00-14, Fax: +32-2-762-00-44.

Abstract

This review is an output of the International Life Sciences Institute (ILSI) Europe Marker Initiative, which aims to identify evidence-based criteria for selecting adequate measures of nutrient effects on health through comprehensive literature review. Experts in cognitive and nutrition sciences examined the applicability of these proposed criteria to the field of cognition with respect to the various cognitive domains usually assessed to reflect brain or neurological function. This review covers cognitive domains important in the assessment of neuronal integrity and function, commonly used tests and their state of validation, and the application of the measures to studies of nutrition and nutritional intervention trials. The aim is to identify domain-specific cognitive tests that are sensitive to nutrient interventions and from which guidance can be provided to aid the application of selection criteria for choosing the most suitable tests for proposed nutritional intervention studies using cognitive outcomes. The material in this review serves as a background and guidance document for nutritionists, neuropsychologists, psychiatrists, and neurologists interested in assessing mental health in terms of cognitive test performance and for scientists intending to test the effects of food or food components on cognitive function.

Introduction

There is increasing academic and commercial interest in the effects of particular foods and/or nutrients on cognitive function. This interest comes from the desire to understand how cognitive performance may be influenced by nutrition during the life course, from early neurodevelopment to age-associated cognitive decline due to neurodegeneration, and, in addition, from the desire to optimize or improve what may be called normal cognitive function in healthy individuals. Evaluation of this relationship requires a battery of valid, reliable, and sensitive measures of cognitive function that ideally can also act as valid surrogates or (bio) markers of the neural processes underlying major domains of cognition tests that can be used in experimental trials.

Tests of cognitive performance should assess how people respond to the cognitive challenges of everyday life and should discriminate between different states and individuals.[1, 2] Crucially, tests chosen to measure cognitive function should have the support of a consensus within the scientific community, so that findings can be used as evidence for consideration of nutritional benefit claims by expert panels such as the European Food Safety Authority (EFSA) and the US Food and Drug Administration (FDA).

The term “cognition” refers to the complex mental functions that enable humans to exert control over their environment and is critical for survival.[3] It includes domains such as memory (episodic, semantic, and working memory), language, attention, executive function, and information processing speed. Development of tests to measure various cognitive functions can be traced back over a century, and some tests have been developed or adapted to measure the different aspects of cognitive function in relation to drug or food intake and nutrient status.[4, 5] ILSI Europe recently published guidelines on methodologies for long-term nutritional intervention studies on cognition[6, 7] but subsequently decided that a more in-depth survey of cognitive domains and of criteria for choosing the most suitable tests for these types of trials was needed, a decision stemming from the broad choice of tests available as well as the move from traditional and established paper-and-pencil tests to automated computerized test batteries that offer customized tests tailored to the user.

Validation criteria for biomarkers or surrogate markers of health status have recently been reviewed by an ILSI task force.[8] An ILSI workshop held in Lisbon in 2012 summarized the approach for evaluating markers or measures used in the field of cognition. First, regarding analytical aspects, methodology for the measure should be accurate, standardized, and robust, and the measures should have good sensitivity and specificity for the outcome being assessed. Construct validity and retest reliability are required for cognitive measures, and the levels of confidence and norms for the tests should be established for target populations. Second, a causal relationship should exist between the marker and the health or disease state being studied. Measures should correlate with the endpoint and with changes in it, and be relevant to the outcome. They should have biological plausibility, even if the detailed mechanism of action of the marker is not fully understood. Third, with relevance to nutritional status or studies, it is desirable that the marker (in this case, the cognitive test score) show a response to dietary or nutritional intervention or lifestyle changes. For nutritional interventions with longer-term effects, or where change in cognitive performance is likely to be small or immeasurable, functional or structural brain measures may be surrogates for cognitive performance, thereby confirming the efficacy of the intervention.[9]

It should not be assumed that findings observed in one population are applicable to any other population without further validation of the marker in the new population. Innovation should not be deterred by these criteria, but novel tests or markers should be evaluated against current norms for age, education, and gender – as determined by established tests or against “gold standards” (such as diagnosis or diagnostic criteria) – to assess their validity.

This report aims to provide guidelines for those planning to study the effects of nutritional interventions on brain function as measured by cognitive performance. To this end, an overview of randomized controlled trials (RCTs) that assessed the effects of nutrients on global and domain-specific cognitive performance is provided, along with the cognitive tests used. Examples covered are those with substantial epidemiological evidence of associations between cognition and a specific nutrient, such as polyphenols, B vitamins, and n-3 fatty acids.

This review includes the cognitive domains of memory (encompassing verbal, visual, spatial, and articulated working memory), selective and sustained attention, executive function, information processing speed, and global cognitive function. Commercially available automated, computerized cognitive test systems, including the Cambridge Neuropsychological Test Automated Battery (CANTAB),[10] the Cognitive Drug Research (CDR) Computerized Assessment System,[11, 12] and CogState,[13] have been developed and improved for use in clinical trials of pharmaceutical agents or for other purposes and have been used across many different clinical settings. These batteries comprise measures of a number of different domains that can be combined in many ways or selected tests used in a stand-alone fashion. More recently, tests from these batteries have been shown to be sensitive to the effects of dietary interventions on cognitive function.[14-16]

The recommendations presented here show how essential criteria can be applied to assess, validate, and select the cognitive tests most suitable for a proposed nutritional intervention study (Table 1). Table 2 summarizes, by domain, the cognitive tests that have been used in a range of nutrient intervention trials to show frequency of use and outcome.[16-73]

Table 1. Application of essential criteria for validation of cognitive function tests
Cognitive domainEveryday functional or behavioral relevanceNeural mechanisms assessedAppropriate target populationsEstablished testing paradigmsUtility, validity, and reliabilityEstablished sensitivity to nutraceuticals
Acute or subchronic effectsLong-term effects
  1. Abbreviations: ADHD, attention-deficit hyperactivity disorder; DHA, docosahexaenoic acid; GI, glycemic index; MCI, mild cognitive impairment.
Memory 
Verbal, visual, spatialLearning, retention, recall, spatial information, pairing of object/verbal cues with spatial information, mappingHippocampal, medial temporal lobes, dentate gyrus, amygdalaHealthy individuals of all ages; individuals with MCI, mild or moderate Alzheimer's disease, or other severe forms of memory impairmentWorld list learning, immediate and delayed verbal recall, verbal and object recognition, paragraph recallWidely established to be practical and reliable; excellent face, criterion, and construct validationEnhancements with carbohydrates, glucose, protein, breakfast, low-GI vs. high-GI breakfast, soy isoflavones3 months: berry polyphenols in MCI; 6 months: high-dairy diet in habitual low-dairy consumers; DHA in healthy individuals
Short or long term
Attention (sustained, selective)Sustained performance on attention-requiring tasks over an extended period of time in a low-arousal context, selective attention for some aspects of information while ignoring othersSensory systems, superior colliculus and thalamus, frontal eye fields, lateral intraparietal areas, prefrontal cortex, reward and motor systemsHealthy children and adults; individuals with ADHDSingle letters, digits, or shapes presented serially where predefined stimuli must be responded to or ignored; simulated driving tests; simulated shift workSustained attention overlaps with “vigilance,” and arousal; selective attention overlaps with “divided attention”; other related concepts are tonic alertness and phasic alertnessConsistent improvements following caffeine, inconsistent results with carbohydrates; children and adults with ADHD perform worse on attention testsNot commonly used in long-term studies
Executive function 
Inhibition, planning, judgement, task switching, verbal fluencyTrip planning, handling finances, fixing things, using equipment, following recipes, making judgements, multitasking, using strategiesFrontal lobes and neural pathways connecting memory centers, cortex and subcortical connectivity, dorsolateral prefrontal cortexHealthy young adults; older women and men; cognitively impaired elderlyTests cover a broad range of abilities, some speed related, including interference, task switching, digit symbol matching, tracking tasks requiring working memory, problem solving, and mental organizational tasksCan Include or overlap with working memory and Information processing abilities. As these higher-order executive functions are complex, not all executive tests assess the same aspect of executive function, though many have good correlation

n3-fatty acids: improvement in cognitively impaired elderly.

Flavonoids: improvement in older men and women

2 years: B vitamins benefit elderly with MCI
Information processing 
Simple reaction times, choice reaction time, tracking, neglectResponse times, comparisons, seeing differences, picking up detail, keeping track, left-right response accuracyAscending reticular formation projections to the cortex, posterior lateral prefrontal cortex, superior medial frontal cortexHealthy individuals of all ages; individuals with ADHD, stroke, Parkinson's disease, MCI, mild or moderate Alzheimer's disease or other dementiasSimple and choice reaction time paradigms, tracking tasks, tasks requiring stimulus discrimination and processingTests widely established for utility, practicality and reliability; extensive validationBreakfast: consumption of low-GI breakfast cereals, energy drinks, drinks containing fat and glucose, and caffeine improved test results3 years: in healthy elderly with high homocysteine levels, folic acid supplementation resulted in significant benefit[17]
Global cognition 
Not domain specificGeneral intelligence (crystallized and fluid), overall mental functioningTotal brain volumeOlder adults with MCI, mild or moderate Alzheimer's disease, or other dementiasMultidomain tests, either brief screening tests or in-depth batteriesTests widely established for utility, practicality, and reliability; extensive validation for targeted age groups and conditionsNo improvement shown in cognitively healthy young or older adultsDHA phospholipids improved cognition in MCI
Diabetics
Table 2. Summary of cognitive tests used in polyphenol, B vitamin, and n-3 fatty acid (n3FA) intervention trials that reported significant findings
Cognitive domainCognitive testaNo. of studies with significant findings (P < 0.05)/no. of studies in which test was applied
  Isoflavone intervention studiesBerry juice, cocoa, resveratrol, or pine bark intervention studiesB vitamin intervention studiesbn-3 FA intervention studies
  1. Abbreviations: ADAS-Cog, Alzheimer's Disease Assessment Scale–Cognitive; BTACT, Brief Test of Adult Cognition by Telephone; CANTAB, Cambridge Neuropsychological Test Automated Battery; CDR, Cognitive Drug Research; CLOX, Executive Clock Drawing Test; CVLT, California Verbal Learning Test; DMST, Delayed Matching to Sample Test; DSST, DANTES Subject Standardized Tests; ID/ED, Intradimensional/Extradimensional Shift; MMSE, Mini-Mental State Exam; RAVLT, Rey Auditory Verbal Learning Test; RVIP, Rapid Visual Information Processing; TICS-m, Telephone Inventory for Cognitive Status–modified; VPAL, Verbal-Paired Associate Learning; WAIS, Wechsler Adult Intelligence Scale; WMS, Wechsler Memory Scale.
  2. a Tests are categorized according to Lezak et al.[72]
  3. b Column information also includes data from the reviews as cited in the reference list:[62, 67, 68, 70]
Composite memory   1/4,[17-20] 
Immediate verbal memoryCVLT1/2[21, 22] 0/2[23, 24]
 Hong Kong List Learning Test0/1[25]  
 RAVLT0/2[26, 50]1/2[28, 69]0/1[69]
 Common Objects Recall Test1/1[29]  
 Randt Memory Test  1/1[30] 
 Selective reminding0/1[29]0/2[31, 32]1/2[28, 33] 
 VPAL0/1[50]1/1[22]0/1[28] 
 WMS Memory 1 Test0/1[25]  
 WMS Paragraph Recall2/6[26, 29, 34-37]  
 Word list recall0/1[36]0/1[38] 
 Word presentation (CDR System)0/1[39]  
 Word recognition (CDR System)0/1[40]0/1[39]  
  3/152/73/70/3
Delayed verbal memoryCVLT0/1[21, 22]  
 Hong Kong List Learning Test0/1[25]  
 RAVLT0/2[26, 50] 1/3[41, 42]
 HVLT  1/1[43] 
 Selective reminding0/1[29]0/2[31, 32]1/1[44] 
 WMS Memory 2 Test0/1[25]  
 WMS Paragraph Recall0/4[29, 35-37]  
 Word recognition (CDR System)0/1[39]  
  0/90/42/21/3
Immediate spatial memoryBenton Visual Retention Test0/1[45]  
 Color matching0/1[45]  
 Contextual Memory Task0/1[46]  
 Corsi Block Tapping Test0/1[45] 0/1[47]
 CANTAB DMST1/3[34, 35, 53]  
 Doors Test0/1[26]  
 Faces (WMS-III)0/2[31, 32]  
 Identical Pictures Test0/1[49]1/1[48] 
 Novel Spatial Working Memory1/1[50]  
 Rey Complex Figure Test1/2[25, 29]  
 Spatial PAL (CANTAB)0/1[21] 1/1[16]
 Spatial Pattern Recognition1/1[46]  
 Spatial Working Memory1/1[46]  
 Spatial Working Memory (CDR System) 1/1[39]  
 Visual Pattern Recognition0/1[45]  
 Visual Spatial Learning Test1/1[29]  
 Visual Spatial Memory0/1[40]1/1[51]  
 WMS Visual 1 Test0/1[36]  
  4/154/81/11/2
Delayed spatial memoryLong-term episodic memory2/2[34, 53]  
 Rey Complex Figure Test1/1[29]  
 Spatial Pattern Recognition0/1[46]  
 WMS Visual 2 Test0/1[36] 1/1[42]
  3/40/1 1/1
Executive functionTrail Making Test A & B2/7[25, 26, 29, 36, 37, 49, 50]0/2[31, 32] 0/1[69]
 Category Generation Task0/1[26]  
 Cube Comparisons Test0/1[49]  
 IDED (CANTAB)3/3[29, 34, 35]  
 Mazes0/1[29]  
 Mental rotation task0/1[50]  
 Stockings of Cambridge (CANTAB)3/3[34, 35, 53]  
 Verbal fluency2/10[25, 26, 29, 34-37, 49, 50, 53]3/5[18, 19, 33, 43, 44]0/2[23, 54]
 Visual scanning0/1[55]  
 WAIS Block Design0/1[36]  
 WAIS Similarities Test0/1[36]  
 Word fragmentation completion0/1[57]  
 CLOX  1/1[43] 
  10/300/34/60/3
Working memoryDigit ordering0/1[45]  
 Digit span1/6[25, 26, 45, 50, 55]0/1[18]1/3[23, 42, 54]
 Meaningful sentence/noun retrieval   1/1[58]
 Numeric working memory (CDR System) 0/1[39]  
 Serial subtraction by 3's1/2[56, 59]  
 Serial subtraction by 7's0/2[47, 59] 0/1[47]
 WMS Arithmetic Test0/1[36]0/1[27] 
 Working memory   0/1[60]
  1/81/50/22/6
Attention & information processing speedAttention switching0/1[71]  
 Stroop Color Test1/2[31, 32, 45]0/2[29, 45] 0/3[39, 46, 47, 69]
 Simple reaction time0/2[39, 46] 0/1[47]
 Choice reaction time1/3[39, 46, 51] 0/1[47]
 Digit Cancellation Test0/1[34]  
 Digit Vigilance Test0/1[25]0/1[39]  
 Paced Auditory Serial Addition Test1/3[34, 35, 53]  
 RVIP1/3[56, 59] 0/1
 WAIS Digit Symbol1/4[31, 32]0/2[31, 32]  
 Visual vigilance0/1[46]  
 Visual selective attention   0/1[58]
 WAIS Letter-Number Sequencing Task0/1[50]  
 BTACT backward counting  0/1[61] 
 DSST  1/3[17, 28, 62]0/1[42]
 Leiter-R Test of Sustained Attention   0/1[39]
  3/122/151/40/9
Global (adult)TICS-m  2/3[19, 27, 61] 
MMSE1/4[43, 63-65]0/2[23, 24]
ADAS-Cog0/1[63] 
    3/70/2
Global (children)Kaufman Assessment Battery for Children   0/1[41]
Semantic/language skillsSynonyms  1/1 
Peabody Picture Vocabulary Similarities1/1[66]1/1[39]
 Boston Naming Test0/4[25, 26, 29, 36]  
  0/4 2/21/1
Psychomotor skillsFinger Tapping Test0/1[25]  
 Grooved Pegboard Test1/2[29, 49]  
 Movement Assessment Battery   0/1[52]
  1/3000/1

Episodic Memory

This domain captures cognitive function relating to the retrieval of information encoded in verbal, visual, or spatial format. Because the memory domain has been studied more extensively than other domains, the different types of memory are addressed in detail below.

Verbal memory

Description of the cognitive domain

Verbal memory reflects the ability to store (from minutes to years) and subsequently retrieve and to recognize previously presented verbal information.[74] The information is encoded primarily in the form of serially presented words, anecdotes or stories, facts, and declarative statements. Verbal memory tasks can be categorized according to the period in which information should be retained and are commonly referred to as either immediate (retrieval required within minutes of encoding) or delayed (retrieval ranging from half an hour to days, weeks, or even years later). This domain does not include semantic memory, which is the memory for concepts and vocabulary, which is slow to decline with age and may even show improvements with age due to improved vocabulary.

Validity of assessment tests

Age of acquisition of verbal material and educational attainment are both important factors that influence performance of verbal memory tasks.[4] Thus, validity of assessment tests needs to be established, taking these factors into account. Different studies assessing the effect of nutritional interventions on cognitive test outcomes have included such a wide variety of verbal memory tests or stimuli that results are often not comparable due to lack of standardization of the various tests against each other for construct validity, reliability, and sensitivity. Differences in sensitivity and task difficulty may result in floor or ceiling effects, depending on the population tested. Test stimuli may vary in terms of the concreteness, imagery, and frequency of the verbal material, both between tests and, of greater concern, sometimes between parallel forms of the same test. However, the Medical Research Council (MRC) Psycholinguistic Database provides a searchable resource for verbal stimuli, ranked on the basis of these factors, that can be used for new test development.[75] Length of wordlists, number of repeated learning trials for encoding (usually 1, 2, or 3 trials), controlled encoding, and implicit learning strategies are all factors that introduce between-test variations that affect cross-comparisons.

Reliability and correlation between tests

One of the oldest, most easily administered tests of verbal memory is the Auditory Verbal Learning Test (AVLT).[76] This test has high test-retest reliability,[77] and factor analytic studies show that the learning measures of the AVLT correlate significantly with learning measures from other tests.[78] Similarly, the California VLT (CVLT)[79] requires recall of word lists, but this test is designed to assess the use of semantic associations as a strategy for learning words. The CVLT-II has good test-retest reliability after 3 weeks (r = 0.82). Alternate-form reliability ranges from 0.72 to 0.79 for various measures.[80] Attention reportedly plays a significant role in CVLT performance,[81] and vocabulary ability alone has been shown to account for 13% of the variance in performance,[82] while correlations with other memory tests are “modest.” The Hopkins Verbal Learning Test (HVLT)[83] contains recall and recognition components that have good validity when compared with other measures of verbal memory.[84] However, unimpaired adults may achieve ceiling performance, and reliability coefficients over 9 months using alternate versions were moderate in healthy older adults (r = 0.5).[85]

Researchers should be aware of possible practice effects with repeated administrations of verbal memory tests. For example, the Selective Reminding Test[86] correlates well with other tests of verbal memory[87]; however, a substantial practice effect for most of the scores appeared with four administrations when different forms of the test were used, regardless of the order of the forms.[88] Improvements following repeated testing have also been observed with paired-words tests such as the Verbal Paired Associates (VPA) test.[89] For example, a short-term (7–10 days) test-retest reliability correlation of 0.53 and a significant 1.33-point gain in mean score were documented for hypertensive patients.[90] A 3-week retest of participants ranging from 17 to 82 years of age produced a relatively high coefficient of 0.72, with an average score gain of 1.31.[91] Despite these practice effects, the VPA has good construct validity, as demonstrated by strong associations with other tests of verbal memory.[78]

Significant practice effects are also observed on tests of story recall. For example, the manual for the Logical Memory subtest of the Wechsler Memory Scale (WMS)[92] reports that subjects in the 20- to 24-year age group make the greatest gains when retested within 4–6 weeks (+7.4 items on immediate recall, +9.4 items on delayed recall). Moreover, practice effects can be observed with lengthy retest intervals of up to 1 year.[93, 94] Despite concerns regarding practice effects, the Logical Memory subtest appears to have good construct validity. Correlational studies consistently demonstrate a relationship between immediate recall of Logical Memory and other learning tests,[78] and even stronger associations have been observed between delayed recall of Logical Memory and other learning tests.[95]

Sensitivity to nutritional interventions

There is more evidence of the effects of nutrition on verbal memory than for other cognitive domains. Studies with isoflavone interventions have shown positive effects on immediate verbal memory in 3 of 15 studies using the Common Objects Recognition Test (n = 1) and the WMS paragraph recall test (n = 2), while none (of 9) have shown positive effects on delayed verbal memory. Interventions with flavonoids, (berry juice, cocoa, pine bark, resveratrol) had positive effects on immediate verbal memory in the CVLT (n = 1/2) and on verbal paired-associate learning (PAL) (n = 1/1) in a total of seven trials and no effects on delayed verbal recall or recognition (as referenced in Table 2). All the studies were conducted in adults, mostly postmenopausal females, but varied in length from 1.75 hours to 12 months.

Evidence of the benefits of supplemental intake of folic acid and B vitamins on episodic memory has been shown in two[17, 43] of four RCTs, although a third showed a benefit in the subgroup of women with low B vitamin intake at baseline.[37, 38] The Folic Acid and Carotid Intima-Media Thickness (FACIT) trial showed the effects of 3-year folic acid supplementation on a composite verbal memory score in cognitively normal adults with high baseline blood levels of homocysteine. A reduced decline in delayed verbal recall was shown in the Homocysteine and Vitamins in Cognitive Impairment (VITACOG) trial with three B vitamins versus placebo in individuals over 70 years of age with mild cognitive impairment (MCI) and high baseline homocysteine levels[43] Fioravanti et al.[30] and Bryan et al.[33] showed positive effects of folic acid on verbal memory, although McMahon et al.[28] did not. However, other RCTs, such as Eussen et al.,[18] showed no effects of folic acid on verbal memory (Table 2). The facilitation of verbal memory by glucose, the primary energy source for the brain, has been demonstrated most consistently with demanding delayed memory tasks.[3, 18, 28]

A recent study[96] compared drinks containing glucose, protein, fat, or placebo in healthy young adults and found the protein drink to significantly enhance a memory index comprising scores from three verbal memory tasks and one visual memory task from the CDR System[97] compared with the other drinks at 60 minutes postingestion.

Consumption of berry polyphenols over a period of at least 12 weeks showed some benefit, particularly for verbal memory in the CVLT or for verbal PAL, at least in older adults with MCI,[21, 98] although some issues with placebo matching and equivalence of control groups have been raised.

Hence, verbal memory may be influenced by supplementation and macronutrient manipulations in young healthy participants, particularly if delayed memory tests are used, and in those with MCI, whereas immediate verbal memory is modulated by food components in older women (e.g., postmenopausal). Thus, beneficial effects of particular nutritional interventions (carbohydrates, glucose, protein, berry polyphenols) were measurable with various types of verbal memory tests, including word list recall, paragraph recall, and composite memory tests, including computerized versions, for various age groups with varying cognitive ability.

The most commonly cited tests of verbal memory included the Rey Auditory Verbal Learning Test (RAVLT), used in five RCTs with all the nutrients (isoflavones, polyphenols, B vitamins, and n-3) (Table 2). Similar popular tests used in various centers included the CVLT, the HVLT, the Hong Kong List Learning Test, and the Consortium to Establish a Registry for Alzheimer's Disease (CERAD) Word List Recall, either free recall and/or delayed recall. Other common tests included were the Selective Reminding Test (5 studies), verbal PAL (3 studies), and the WMS paragraph recall test (5 isoflavone RCTs).

Visual memory

Description of the cognitive domain

Visual recognition memory is the ability to correctly identify or reproduce a visually presented image. This is an aspect of memory that can apply to short-term memory (working memory) or longer-term memory (episodic memory). Some properties of this aspect of memory are comparable to verbal memory (i.e., the limit to the number of items that can be held in working memory) or the time-related decline in the ability to retain the image once encoded (i.e., forgetting). It is different from spatial location memory, which is the ability to identify where an object was originally located versus where on a computer screen an object was subsequently presented. In everyday terms, it is the ability to correctly identify or recreate a previously seen visual image such as a picture, an object, a scene, or a face.

Visual recognition tasks vary in terms of the test paradigms, but many have the same basic format: 15–20 images are presented, and approximately at least10 minutes later, re-presented together with similar but different images, called confounders or distractors. The subject is required to indicate whether he or she believes the image is the original one or a new one.

Spatial and object memory

Description of the cognitive domain

Spatial memory is a broad term that encompasses the ability to store and retrieve knowledge about the spatial features of one's environment, such as remembering a familiar route to work or where one's car is parked. Spatial memory is vital to daily living because it allows a sense of where critical objects in the environment are located and provides a mental map of space that can be used for navigation.[99] It should be acknowledged at the outset that spatial memory is a complex and multidimensional function interdependent on both attention and memory processes. A useful taxonomy employed here distinguishes between spatial working memory, memory for routes, and object location memory.[99] Spatial working memory refers to the cognitive ability to update information required to resolve a spatial task when contextual conditions change from one trial to the next.[100] The majority of everyday tasks require some element of spatial working memory, such as updating spatial information to orient oneself during a car journey. Spatial working memory is part of working memory that consists of two subsidiary slave systems – the “phonological loop” and the “visuospatial sketchpad” – that hold limited amounts of information active.[101] In turn, these slave systems fall under the control of the central executive system that is responsible for controlled behavior and thought. An example of a spatial working memory task is the Corsi Block Tapping Task. Although it was originally designed as a clinical diagnostic tool, computerized versions of the test in various forms are now available.[102]

Route learning (traveling from A to B) can be achieved by using a map and by navigating through an environment. Although tasks that test actual navigation in the real world have been developed,[103] researchers have increasingly employed virtual environment tasks similar to those routinely used in animal research, such as the Morris Water Maze[102] and the Radial Arm Maze, but developed for use in human trials, e.g., the Memory Island task of Piper et al.[104]

Object location memory is a static form of spatial memory that measures the ability to remember the fixed position of objects in the environment. Typically, the participant is initially presented with an array of objects to be remembered and is subsequently asked to reproduce the original positions of the objects or to recognize the “correct” arrangement of objects from foils. Numerous spatial memory tasks have been developed using variants of this methodology; for example, Kessels et al.[105] required participants to recall the location of previously presented objects on a grid. Mahoney et al.[106] developed a fictitious country task specifically for children that involved filling in a blank map with the locations of previously learned countries and demonstrated sensitivity to the effects of breakfast.

Validity of assessment tests

There are a variety of object recall and recognition tests available, many of which have been validated to various degrees. Glahn et al.[107] reported good convergent and divergent validity for the Visual Object Learning Test. Visual/spatial memory tests usually use abstract designs or nonsense figures as stimuli to minimize verbal mediation. However, it is almost impossible to completely eliminate verbal associations. This could explain why spatial memory tests are not lateralized as effectively as verbal memory tests.[108] Visual memory tests often require a visuomotor response, such as drawing, which can further complicate the evaluation of performance on these tests. It can be difficult to distinguish the extent to which performance deficits are due to impairments in motor skills, spatial memory, visual processing, or an interaction among these factors.

Reliability and correlation between tests

Very little research has explored correlations across spatial memory tasks. Performance on spatial memory tasks would be expected to correlate with performance on tasks of other types of memory, such as verbal and working memory and, to some extent, on tasks of executive function. Millis et al.,[109] however, reported that scores on the Faces subtest from the WMS-III do not correlate as well as other measures of visual memory on a “general visual memory factor” composed from the WMS-III.[110] This may reflect the possibility that facial recognition utilizes different aspects of visuospatial processing than other object recognition tasks. The immediate Visual Reproduction (VR) test correlates significantly with tests that predominantly involve visuospatial problem solving (executive function), whereas the association with other visual memory tests is stronger for the delayed component of the VR test.[111] Similarly, immediate and delayed recall of the Complex Figure Test correlates strongly with other tests of visual memory, although correlation with tests requiring reasoning and planning (executive function) has also been reported.[112] In addition, factor analysis has revealed that VR test performance is often affected by reconstruction motor skills.[110]

Sensitivity to nutritional interventions

Nutritional intervention has been found to influence visual memory. The Rey-Osterrieth Complex Figure Task, a pen-and-paper visual recall test in which participants copy and then recall a complex figure, was shown to be facilitated by a 25-g glucose load in one study.[113] Another study from the same authors showed a trend for glucose facilitation,[113] whereas a third study showed no effect of glucose or fat content.[114] Soy isoflavones improved visual memory in some studies,[29, 115] but not all.[25]

Spatial memory has been demonstrated to benefit from isoflavone consumption. Three of four tests of delayed spatial memory reviewed by Lamport et al.[116] showed significant effects of isoflavones, although only 4 of 15 tests showed positive effects of isoflavones for immediate spatial memory. Yurko-Mauro et al.[16] reported that errors on the PAL test, a spatial working and episodic memory test from the CANTAB, were reduced following docosahexaenoic acid (DHA) administration for 24 weeks in healthy adults with mild age-related cognitive decline. However, a change from baseline analysis was performed that could have introduced artefactual effects.[117] These effects were all observed in longer-term interventions. At the end of a 3-month study of the antioxidant flavonoid pycnogenol in elderly volunteers, Ryan et al.[39] found that individuals in the treatment group performed better on the CDR System spatial working memory task than individuals who received placebo. This improvement was associated with significantly decreased concentrations of plasma F2-isoprostanes.

Performance on spatial working memory tasks such as the Corsi blocks task has also shown some sensitivity to flavonoids,[118] and one study showed a benefit of phospholipids on a navigation task (Visueller Gedächtnistest, VISGED) in chronically stressed men.[119]

Tests reviewed in this domain varied widely and included many computerized variations of spatial PAL, spatial working memory, and spatial and pattern recognition tests. The CANTAB digit matching to sample (DMTS) task was used in four isoflavone trials.

Selective and Sustained Attention

Description of the cognitive domain

Attention is a prerequisite for good information processing. The more one's attention is directed toward information (either internally or in the environment), the more likely one will be able to retrieve it from memory at a later stage. There are different classifications of subdivisions within the attention system. Here, a subdivision between selective and sustained attention, based on the theory of Petersen and Posner,[120] and Posner and Petersen,[121] is used. Sustained attention refers to the ability to sustain performance on an attention-requiring task over an extended period of time in a low-arousal context.[122] Selective attention tasks require prioritizing some aspects of information while ignoring others by focusing on a certain location or aspect of the scene or task.[123]

Sustained attention overlaps with “vigilance,” a term that refers to the ability to sustain attention to a task over time.[122] There is also a link with “arousal,” a term referring to nonspecific brain activation in relation to sleep-wake states and related to sustained attention. Other related concepts in the context of the attention network model are tonic alertness and phasic alertness.[120, 124] These concepts correspond to the underlying internal state of alertness, which has a circadian rhythm (related to sustained attention), and replacement of the resting state with a new state that involves preparation for detecting and responding to an expected signal (related to selective attention), respectively. Finally, attentional processes in dual-task situations referred to as “divided attention” have been described as being separate from selective attention. However, brain imaging studies have failed to demonstrate clear differences in functional activation of brain areas during selective and divided-attention tasks,[125] suggesting that these two types of attentional task involve common processing mechanisms. Examination of the distinct cognitive components of some complex attentional tasks has been attempted. For example, copying speed (or motor skill) accounts for 72% of the variance in Digit Symbol Coding performance,[126] whereas general intellectual ability, memory, and learning are not associated with performance.[127] The natural decline in motor speed and reaction time appears to be the main factor accounting for an aging-related decline in performance of tests of attention.[72] Test-retest reliability for Digit Symbol Substitution tends to be high, with correlation coefficients in the range of 0.82 to 0.88.[89, 128] Various practice effects have been reported, but most are modest.[129]

Sustained attention tasks

Sustained attention tasks commonly require participants to sustain their attention to a continuous stream of stimuli (single letters, digits, or shapes presented serially, sometimes in combination with tones presented more rarely) for a longer period of time (usually 30 minutes or longer). There are two types of tasks. For the first type, participants are required to respond to all stimuli except predefined infrequent target stimuli. An example of this type is the Sustained Attention to Response Task (SART),[130] which is also called the Stop Task. In the second type, participants are required to respond only to prespecified infrequent target stimuli. An example of this type is the Continuous Performance Task (CPT).[131] Adapted versions of the SART and the CPT have been used in children with attention deficit hyperactivity disorder (ADHD).[132-134] Other paradigms that may have greater ecological validity but are less well validated are (simulated) driving tests and simulated shift work. Alternative tests for neurological or ADHD patients are the Elevator Counting and the Lottery task subtests of the Test of Everyday Attention (TEA).[135, 136]

Selective attention tasks

In selective attention tasks, participants are presented with multiple stimuli within each trial and are asked to respond to the relevant stimuli (targets) and ignore the irrelevant stimuli (distractors).[137, 138] Examples of selective attention tasks include the following: flanker tasks, in which visual targets are flanked by either congruent or incongruent stimuli; cue-target paradigms, in which participants receive a cue at the start of each trial indicating what the target will be; mixed tasks, which also involve executive function such as Posner's Attention Network Test (using cues and flankers); the Stroop task, in which the task and the visual aspects of the stimuli can be congruent or incongruent; and task switching or shifting, in which participants are required to switch between tasks every few trials.

Alternative laboratory tests that are more commonly used with neurological patients and ADHD patients are the map search and the telephone directory search tasks of selective attention from the TEA battery. These tests have some ecological validity.[135, 136]

Validity of assessment tests

Tests of attention such as the SART,[139] the CPT,[140] and several selective attention tasks[141] have been validated in terms of increased accuracy and faster reaction times from childhood to young adulthood, and in terms of more errors and slower reaction times from middle age onward. The attention tests described are validated in their sensitivity to attention deficits in patients with ADHD[130, 132-134, 142] and show reversal of attention deficits with methylphenidate.[143] Normative data for children with ADHD are also available for the TEA.[135] Motivation plays an important role in performance on sustained attention tests, and there is an increased risk of task inattention and response disengagement with prolonged testing.[139] Furthermore, many variants of attention tests exist with only minor differences; however, as discussed by Shalev et al.,[144] it is unclear to what extent different variants tap into the same construct of attention. Although attention, tracking, and motivation can be differentiated theoretically, in practice they are difficult to separate. Pure attention deficits appear as an impaired ability for focused behavior, regardless of the participant's intention. A lack of motivation can be defined as an inability to maintain a purposeful attentional focus. It can be argued that intact attention and some degree of motivation are prerequisites to perform a tracking task.

Sensitivity to nutritional interventions

Caffeine is well known to enhance aspects of both sustained and selective attention, as shown with behavioral measures[145] as well as with electroencephalography and event-related potentials.[146-149] Caffeine commonly reduces reaction time and increases accuracy and has also been demonstrated to reduce omission and commission errors. The Stroop test and DSSTs have been used in various intervention studies with isoflavones, polyphenols, and n-3 (Table 2), although some studies may refer to these tests as executive function tasks or processing speed tasks.

Information Processing

Description of the cognitive domain

This domain relates to the ability to direct mental focus to relevant information in order to process it rapidly and efficiently, as well as to make accurate and appropriate decisions in line with predefined external rules and/or internal knowledge.[5, 150]

The classic information processing tests are Simple Reaction Time and Choice Reaction Time, originally developed in the 19th century.[151] A further test, Rapid Visual Information Processing, is also widely used.[5] Simple Reaction Time tasks require the rapid response to a predefined stimulus that will occur at a predefined location. The major measures are the time to respond in milliseconds and the variability of the responses. Choice Reaction Time tasks require the rapid response to two or more predefined stimuli that will occur at a predefined location. The major outcome measures are the time to respond in milliseconds, the variability of the responses, and the percentage correct responses. In the Rapid Visual Information Processing test, a continuous series of digits appears on the screen at the rate of 100 per minute. The subject must detect sequences of three consecutive odd or three consecutive even digits, pressing a response button as quickly as possible, a test that clearly also involves sustained attention.

More recently, other neuropsychological tests of information processing have been developed that require more complex thinking. Such tests include the Trail Making Test (TMT) A, the Letter Cancellation Test,[2] and the Pattern and Letter Comparison speed test.[152] Currently available computerized test batteries (e.g., CANTAB, CDR System, and CogState) contain various information processing tasks. It should be noted that, although they may be more sensitive to changes, more complex information processing tasks suffer more from speed/accuracy trade-offs as well as from training/practice effects than simple information processing tasks.

Validity of assessment tests

Such tests have widely established validity, including face validity, criterion validity, and construct validity.[5]

Reliability and correlation between tests

The tests do show appropriate intercorrelations with the same domain. Although there are a range of important aspects of information that are captured separately by different tests (e.g., Simple Reaction Time and Choice Reaction Time), the correlations are appropriately in the general range of 0.5 to 0.6. A factor analytic study showed that the Paced Auditory Serial Addition Test (PASAT)[153] has more in common with tests of attention and information processing than with tests of memory, visuospatial skill, or verbal knowledge.[110] PASAT style tests may not be ideal for short-term repeated testing. Significant gains have been reported with successive administrations, indicating practice effects, although performance tends to plateau at the fourth and fifth adminstrations.[154] It can also be difficult to distinguish the degree to which tests of attention rely on psychomotor skill and manual dexterity. A pure psychomotor test should be a simple measure of fine motor skill.[155] Typically, these consist of timed speed tests that have an apparatus with a counting device. The Finger Tapping Test (FTT),[156] the Purdue Pegboard, and the Grooved Pegboard[157] are widely used tests of psychomotor skill and manual dexterity that have validated norms and are sensitive to various brain disorders.[72]

Sensitivity to nutritional interventions

A variety of aspects of information processing have shown sensitivity to improvements or impairments with a wide range of nutritional interventions, including caffeine, energy drinks, breakfast, macronutrients, and so forth. Measures involving Simple Reaction Time and Choice Reaction Time task paradigms have been found to be favorably affected by breakfast consumption,[14, 15, 158, 159] energy drinks,[160, 161] drinks containing fat and glucose,[96] and caffeine.[114, 162] Three large, long-term RCTs with B vitamins[17, 28, 61] assessed change in processing speed with digit substitution tasks. Only the FACIT trial[17] of folic acid supplementation in cognitively normal adults with elevated baseline homocysteine levels showed less worsening of processing speed (DSST) in the treatment group compared with the placebo group, while the McMahon trial showed more worsening with high-dose folic acid. The treatment dose may have resulted in circulating unmetabolized folic acid, which could be detrimental if B12 status is below normal.[62] Walker et al.,[61] by comparison, included subjects with low baseline homocysteine levels and found no improvement of processing speed. Gleason et al.[29] noted a beneficial effect on the Grooved Pegboard following 100 mg of isoflavones for 6 months, but Basaria et al.,[49] who used a dose of 120 mg for 12 weeks, did not.

Executive Function

Description of the cognitive domain

Executive function is a domain that includes complex or higher-order thinking processes related to a number of types of behavior. It includes processes such as planning, problem solving, verbal reasoning, inhibition (the ability to suppress automatic and habitual responses or behaviors), mental flexibility or set shifting (alternating between behaviors or information sources), updating (the ability to discard and replace information), multitasking, and initiation and monitoring of actions.[163, 164] All of the above processes are dependent on working memory, a psychological construct used to describe the temporary manipulation and maintenance of speech-based and/or visuospatial information and requiring the control of attentional resources.[101]

Executive function is measured with a variety of cognitive and behavioral measures to cover the different functions mentioned above, including set shifting and inhibition (Stroop Test, Wisconsin Card Sorting Test), planning, problem solving, and decision making (Pegboard Test, Stockings of Cambridge Test, Tower of London Test, Clock Drawing Test), attention, monitoring, and tracking (Trail Making B Test and Digit Symbol Substitution Test), and category fluency and abstract thinking (Frontal Assessment Battery). It should be noted that changes in information processing are likely to affect the Trail Making A and B Tests, the Letter Cancellation Test, and the Digit-Symbol Substitution test, whereas changes to higher “executive” function processes are more likely to only be detected by the Trail Making B test and the Digit Symbol Substitution Test. In addition, behavioral tests of planning include testing the ability in managing finances, using telephone directories along with the telephone, and managing appliances with complex controls. Some of these types of tasks have been included in cognitive test batteries, including the Behavioural Assessment of the Dysexecutive Syndrome (BADS) for adults and children (BADS-C)[165] and the Delis-Kaplan Executive Function System.

Validity of assessment tests

Many tests of executive function have been validated for particular populations, including the Symbol Digits Modality Test (SDMT), for which norms have been published for children, adults, and those with brain injury.[166] However, for some tests, such as the Stroop Test, there are a number of variations on the test construct that do not have good correlations. Therefore, results from different studies using different forms of the Stroop Test are not directly comparable. Color versions of the TMT are available for both children and adults, but the administration of the test can vary, especially with regard to the recording and scoring of errors, because the test is timed. Thus, there is an advised capping on the length of time allowed for completion of the TMT.[72] Problem-solving tests such as the Tower of Hanoi (TOH) and the Tower of London (TOL) measure several different cognitive components such as planning, inhibition, procedural learning, explicit reasoning, working memory, and visuospatial memory.[167] However, the abstract nature of many of these tests means that they lack ecological validity with respect to the planning requirements of daily life. Several methods have been devised to assess everyday planning, typically in the form of videotapes depicting everyday awkward situations such as negotiating a solution to a problem with a neighbor or identifying the missing information required to solve a problem.[168] Patients with frontal lesions are typically impaired on these tasks[169]; however, the exact cognitive processes required during these tests are unclear.

Reliability and correlation between tests

Because executive function covers a range of higher-order thinking processes and skills, these functions do not all correlate specifically (e.g., set shifting may not correlate with a test of planning). Thus, care is needed in selecting appropriate tests and interpreting results, especially when comparing studies using different test constructs in this domain. For example, the TOH and the TOL, although very similar tests of problem solving, do not measure precisely the same cognitive skills, as indicated by a low correlation of performance between the two tasks (r = 0.37).[170] Goel and Grafman[171] propose that the TOH predominantly assesses response inhibition rather than forward planning.[172, 173] Working memory contributes to solutions of harder problems because more subgoal information needs to be processed.[174] However, it is still possible to show good correlation between a task such as the SDMT and the Stroop Test or the TMT[175] or to show that they all predict the risk of cognitive decline in a similar way.[176]

Practice effects are often evident, possibly because executive function tests afford the development of strategies. For example, significant practice effects were observed on Part B of the TMT with four successive examinations spaced 1 week to 3 months apart[177] and with the written and oral formats of the SDMT after 1-month intervals.[166] Retest data for the Stroop Test are inconsistent, with some studies showing no practice effects but others showing considerable gains on the second administration.[129] Overall, the evidence suggests that tests of executive function may not be suitable for short-term repeated administration. Significant correlations have been shown between objective tests and Activities of Daily Living Scales or judgment and planning items on the Clinical Dementia Rating Scale.[178] Thus, the domain of executive function might effectively reflect a person's functional abilities or disabilities.

Sensitivity to nutritional interventions

Effects of nutrients on executive function have been examined in a number of RCTs. The n-3-fatty acids had a beneficial effect on only the letter fluency test of 12 cognitive tests administered to older adults with MCI in a 6-month RCT.[179] Furthermore, a recent meta-analysis of 10 n-3 fatty acids RCTs reported a benefit for attention and processing speed in individuals with cognitive impairment but no dementia, but this benefit was not observed in healthy older adults, nor were benefits for executive function, working memory, or episodic memory reported.[180] Older people with MCI had improved Executive Clock Drawing Test (CLOX) scores after 2 years of treatment with vitamin B supplements (B12, B6, and folic acid),[43] a finding supported by a decrease in the rate of brain shrinkage in the treated group. The CANTAB Intradimensional/Extradimensional (IDED) attention test, a measure of rule learning and reversal, was used in three isoflavone studies in young adults and older adult females, all of whom showed significant effects of flavonoid treatment.[34, 35, 53] The Stockings of Cambridge (SOC) test, used in the same three studies, showed a significant effect in two of them. The TMT was used in a number of studies[181] but was sensitive to flavonoid-mediated cognitive change in only one 6-month study.[29] Two of seven isoflavone studies showed a significant effect of treatment on verbal fluency (in postmenopausal women and older men).[29, 37] Finally, no effects of treatment were found for visual search, indicating either that these measures were not sensitive to the effects of flavonoids or that these cognitive functions are unaffected by them.[181]

The most commonly used test of executive function in the nutrient RCTs reviewed has been the Verbal Fluency Test, used in 18 studies, (9 of isoflavones, 5 of B vitamins, 2 of polyphenol, and 2 of omega-3), five of which showed significant effects (Table 2).

Global Cognitive Function

Description of the cognitive domain

Global cognitive function includes multidomains of cognition such as orientation, attention, memory, language, executive function, and motor skills. The term “global cognition” is used to refer to overall or general mental performance that taps into various neural networks involving different brain regions. Measures of global cognitive function are often used as screening measures to detect the general cognitive status of a population sample or individual. However, most global screening tests cannot truly be said to measure global cognitive function because they do not cover the full range of cognitive domains. Tests of global cognitive function are unlikely to be sensitive to small changes in cognitive performance over short time periods in healthy individuals, including older individuals at predementia stages, and may show ceiling effects in young and/or healthy adult populations with few cognitive deficits. However, their purpose is to distinguish those individuals with general cognitive deficits from those whose performance is in the normal range for age.

Tests of global cognitive function

Tests of global cognitive function include the Mini-Mental State Exam (MMSE),[182] the Alzheimer's Disease Assessment Scale Cognitive Subscale (ADAS-Cog),[183] the Montreal Cognitive Assessment (MoCA),[184] the Telephone Interview for Cognitive Status-Modified (TICS-M),[185] and the Cambridge Cognitive Examination (CAMCOG).[186] These tests are suitable for cognitive screening of older people to detect dementia and MCI as well as for the measurement of widespread, gross cognitive changes over time in longitudinal studies.[187] In Alzheimer's disease research, the MMSE has shown an overall progression rate of 0.24 points per month, although this was moderated by education duration, gender, disease incidence, and drug therapy.[188]

Other measures useful for screening and standardizing adult study populations include the National Adult Reading Test (NART),[189] which is used to acquire baseline measures of crystallized intelligence, and Cattell's Culture Fair (CCF) Test, which is a nonverbal measure of fluid intelligence.[190] The Wechsler Adult Intelligence Scale-Revised (WAIS-R) comprises a battery of tests of different cognitive functions. For children, Raven's Progressive Matrices, the Wechsler Intelligence Scale for Children,[191] the Moray House Test no. 12 for general IQ,[192] and the Stanford-Binet IQ test are used to capture general cognitive ability.

Validity of assessment tests

These IQ[193-195] and screening measures are all normalized, standardized, valid, reliable, and well-established tests. Test scores are used to establish the percentile of the population within which the score lies, based on age and education levels. The MMSE has good retest reliability,[129] although it suffers from ceiling effects in normal and highly educated populations and is less sensitive to early cognitive impairment such as MCI. The TICS-M covers domains similar to those covered by the MMSE but affords a more sensitive measure of memory. The TICS-M has been shown to have very high test-retest reliability in Alzheimer's and stroke patients.[196] Tests of general ability that are designed as screening tools for conditions of clinical cognitive impairment such as dementia (e.g., the MMSE) and MCI (e.g., the MoCA) are not suitable as measures of global cognition in healthy populations and are unlikely to detect change induced by nutritional intervention. The MMSE is not recommended for use in RCTs of nutritional interventions, especially those that include cognitively normal participants or individuals with low levels of education.[7] Many studies, however, still include this measure, as it is well cited in the literature, thus perpetuating its use.

Correlation between tests

Studies show that measures of global ability tend to correlate highly. For example, NART and WAIS scores are highly correlated,[73] and significant correlations (r = 0.94) have been observed between the MMSE and the TICS-M.[197] A number of studies have been published comparing the MMSE with other screening measures such as the MoCA and the TICS-M,[184, 198, 199] which typically show high correlations between measures. The substantial correlation between these global measures is not surprising, given that the initial development and validity of these tests is based on comparisons with previously established tests.

Sensitivity to nutritional interventions

Tests of global cognitive function may not be sensitive to dietary-induced changes in the short term and may be more useful if assessed over a long time period (e.g., 2 years or more).[6]

Very few studies that have used the MMSE as an outcome measure have documented significant changes over time in response to dietary intervention in nonelderly or elderly individuals in cognitively intact samples. For example, a recent Cochrane review concluded there was no difference between elderly participants supplemented with n-3 polyunsaturated fatty acid and placebo participants in MMSE scores at final follow-up.[200] However, intervention studies have demonstrated improvements in MMSE scores in individuals with MCI following supplementation with B vitamins[43] or with DHA phospholipids containing melatonin and tryptophan.[201] Two of three intervention studies with the TICS-M as the outcome measure showed effects of B vitamins,[27, 61] specifically in those with low baseline intake.[19]

Important Considerations in Test Evaluation

Most studies that use a combination of tests report significant effects on some measures and nonsignificant effects on other measures in the same participants. When a test battery is utilized, only a limited number of the tests will be sensitive enough to identify any performance differentiation between manipulations.[202] It may not be that the manipulation affects only those specific functions, but that the cognitive tests that assess all of the different domains are not equivalent in terms of their sensitivity to nutrient-induced performance change.[3, 203] It is therefore critical to evaluate the sensitivity of measures of specific cognitive functions with respect to their sensitivity to detect nutrition-induced changes.[202, 204] A null result may be due to a lack of sensitivity of the selected test or a lack of statistical power (i.e., a type 2 error or insufficient sample size, rather than a true absence of an effect). Two tests may claim to measure the same cognitive domain, but it may be that only one is sufficiently sensitive to detect the effects of a nutritional manipulation because of subtle differences in the level of difficulty or the cognitive processes recruited or engaged by the task. This highlights the critical importance of adequate descriptions of tasks employed in studies so that their sensitivity to nutritional intervention can be determined.[2] Length of intervention, dose of nutrient, and cognitive status and nutrient status at baseline will all also affect intervention outcomes, and thus discrepant results of efficacy from nutrient studies may be difficult to interpret. It is important to distinguish screening tests for detection of cognitive impairment from those used to assess changes in general or specific cognitive functions. Confounding of cognitive tests as either predictors or outcomes of nutrient effects also needs consideration. A predictive test may show associations cross-sectionally with a biomarker or blood test related to the nutrient being tested. For example, a high cholesterol level may be associated with, or predict, poor memory performance. Intervention with a nutrient, such as n-3, may improve memory performance. Therefore, cognitive markers can both indicate the need for intervention and be used to measure the outcome of an intervention.

There are numerous (thousands of) cognitive tests available, and with different study groups selecting preferred or more familiar tests, achieving consensus on suitable batteries of tests for nutrient intervention studies is not an easy task. At least 66 different cognitive tests were used in the 19 B vitamin RCTs used in a recent meta-analysis,[68] while Hoyland et al.[202] reported 132 cognitive outcomes in a review of 31 macronutrient intervention studies. Even companies with computerized test batteries offer customized tests for trials, promoting further inconsistencies between studies. Harmonization is also difficult to achieve due to cultural and language differences across the globe. Visual tests are in some ways able to overcome these discrepancies; however, when verbal abilities are shown to be most sensitive to change, it may be more appropriate to use piloted translations of verbal tests in different languages than to resort to visual tests that may access somewhat different neural circuits.

Conclusion

Cognitive tasks should be selected because of a known or hypothesized relationship between a particular food and a particular cognitive function or domain, rather than the availability or ease of administration.[205] In addition, those tasks that are suitable for repeated administration, appropriate to the sample to be studied, and relatively simple to interpret and administer are preferable. In assessing the suitability or choice of a cognitive test to measure the effectiveness of a nutritional intervention, several questions could be asked: 1) Which cognitive domain(s) is the nutritional intervention expected to affect? 2) Which everyday functional activities does the nutrient enhance or deplete? In addition, several questions concern the test(s) selected to assess the domain(s): 1) Is it a well-standardized test? 2) Is its sensitivity to the nutrient known, or is its ability to discriminate between the groups being tested? 3) Are the underlying neural correlates of the cognitive domain understood, and if so, how does the biological mechanism of action of the nutrient on cognition relate to these? Table 1 is included as a guide for the evaluation of criteria for the selection of cognitive tests for nutritional intervention studies.

In summary, both the state of standardization of cognitive tests and the use of these tests in recent nutritional intervention studies have been reviewed. The overall recommendation of this review is that the field of cognitive function assessment is in need of further alignment of measures in each domain to enable valid comparisons of study outcomes, particularly for intervention studies. If more published trials of nutrient efficacy are based on standardized tests across laboratories, evidence for the cognitive domains and underlying neural pathways influenced by the nutrient will increase, and the specific tests or markers that are most sensitive to the nutrients tested can be established.

Acknowledgments

The opinions expressed herein and the conclusions of this publication are those of the authors and do not necessarily represent the views of ILSI Europe or its member companies.

Funding

This work was commissioned by an expert group of the European branch of the International Life Sciences Institute (ILSI Europe). The expert group received funding from the ILSI Europe Nutrition and Mental Performance Task Force. Industry members of this task force are listed on the ILSI Europe website at http://www.ilsi.eu. Further information about ILSI Europe can be obtained by e-mailing info@ilsieurope.be or calling +32-2-771-00-14.

Declaration of interest

EAdB is an employee of Unilever, and JF is an employee of PepsiCo Europe. KW is employed by and is a stockholder of Bracket Global, which provides cognitive testing services to the clinical trial industry. CdJ, LD, LB, JS, and KW received an honorarium from ILSI Europe for their participation in this publication and/or reimbursement of their travel and accommodation costs for attending the related meetings. LSJ is a former employee of ILSI Europe. MEL is an employee of ILSI Europe. The remaining authors have no relevant interests to declare. All authors contributed to discussions and had input into writing the article. All authors have seen and approved the final version of the article.

Ancillary