Updating and Not Shifting Predicts Learning Performance in Young and Middle‐Aged Adults
ABSTRACT
The goal of this study was to investigate whether single executive function (EF) tests were predictive for learning performance in mainly young and middle‐aged adults. The tests measured shifting and updating. Processing speed was also measured. In an observational study, cognitive performance and learning performance were measured objectively in 851 adult students and analyzed using multiple linear regression. EFs and processing speed were measured via cognitive tests. Learning performance was evaluated after 14 months. The results show that updating performance is predictive for learning performance, with a small effect size, while shifting performance was not. This means that a single updating test has predictive value for learning performance acquired over a longer period of time. However, as the effect size is rather small, the test on its own does not serve as a proper selection tool for determining whether a student will be successful or not.
Many studies have tried to predict and explain learning performance (Spinath, 2012). Previous learning performance and scores on standardized tests (i.e., tests on math, reading, language) have received a lot of attention as predictors of future learning performance. These measures account for around 25% of the variance in learning performance measures (Robbins et al., 2004). This motivates the search for other so‐called “third factors” as predictors of learning performance. Of the many possibilities, executive functioning could serve as an excellent candidate (Knouse, Feldman, & Blevins, 2014), because it is important for normal performance (Salthouse, Atkinson, & Berish, 2003). In this light, numerous studies have investigated the relation between cognition and learning in children and adolescents in traditional education finding executive functioning predictive for school performance or academic achievement, respectively (e.g., Best, Miller, & Naglieri, 2011; Bull & Scerif, 2001; Roebers et al., 2014; St Clair‐Thompson & Gathercole, 2006). Missing is research in adults, despite the fact that lifelong learning is imperative nowadays because life expectancies are rising (The Netherlands: Centraal Bureau voor de Statistiek, 2014; Worldwide: United Nations, 2012), retirement age is increasing (OECD, 2016) as rapid changes continue to occur in our knowledge‐based economy. The competencies (i.e., knowledge, skills, attitudes) needed for employment change more quickly so that diplomas and certificates no longer have lifetime validity and, thus, there is an increasing need for development far into adult age often via continuing formal education (Eurydice, 2011). To this end, the possibility of predicting learning performance using simple and easy‐to‐implement cognitive tests in this population group would provide both students and educational institutes useful information about prospective success. The present study investigates the possible predictive relation between performance on single EF tests and learning performance in a unique adult population, composed mainly of young and middle‐aged adults.
Learning performance is an umbrella term for various measures that reflect performance in an academic setting. In traditional education performance is mostly referred to as school performance in primary education and academic achievement in secondary education, reflecting either performance measured by grade point average (GPA) or grades in specific domains for primary education (e.g., reading, mathematics). In the distance education (DE) investigated in the present study, there is no fixed curriculum and the student determines the study pace and program. Therefore, two measures are of interest: study progress in terms of successfully completed courses and academic performance in terms of average grade per course.
Cognition refers to mental processes such as memory, planning, and problem solving; from simple lower‐order processes such as processing speed to more complex higher‐order processes such as inhibition. Executive functions (EFs) are such higher‐order processes. They are top‐down controlled mental processes needed for concentration and attention, and using them requires mental effort. Not using them means that individuals act habitually, follow automated behavior, and give in to temptations (Diamond, 2013). Although many definitions for EFs and their constituent components exist, the definition provided above is generally agreed upon (Jurado & Rosselli, 2007). The concept of EFs are based on Baddeley's (1983) model of working memory and his later proposal (1996) on which functions are performed by the “central executive”. Miyake et al. (2000) statistically analyzed Baddeley's proposal and described these functions as inhibition, shifting, and updating. Inhibition is the ability to ignore distraction; shifting is the flexible switching between tasks or mental representations; and updating is about monitoring and altering the working memory contents by manipulating the short‐term storage. Together with the short‐term storage updating forms the working memory (E. E. Smith & Jonides, 1999). Some researchers believe EFs to have a unifying, central factor (Duncan, Emslie, Williams, Johnson, & Freer, 1996; de Frias, Dixon, & Strauss, 2006), while others believe EFs to depend on separate processes (Miyake et al., 2000; Salthouse, 2005). Miyake and colleagues (Miyake et al., 2000; Miyake & Friedman, 2012) proposed the unity/diversity framework, which aligns both views. They report on common EF variation (i.e., unity) and EF‐specific variation (i.e., diversity). Common EF is the ability to actively manage the tasks at hand and the task‐related information and to use this information to guide and steer lower‐level processing. EF‐specific variation is the variation which remains after controlling for common EF variation. When controlling for common EF variation, there is only specific variation left for updating and shifting (Miyake & Friedman, 2012). This means that the common EF ability is a basic need for all three EFs and is especially important for inhibition, as no EF‐specific variation remains after controlling for common EF (Miyake & Friedman, 2012).
As mentioned, most research on the relation between EFs and learning performance has been carried out in children and adolescents. The associations found between EFs and learning performance are highly consistent, irrespective of EF type, age, or measurement design (Jacob & Parkinson, 2015). The overall unconditional effect size is around 0.30, which drops by more than half when controlled for background characteristics and IQ (Jacob & Parkinson, 2015). Most consistent findings were found for inhibition and working memory—in which the EF updating plays a role—and their relation to mathematics and reading (e.g., St Clair‐Thompson & Gathercole, 2006).
With respect to shifting, some researchers report that shifting is not consistently related to learning performance (e.g., Espy et al., 2004; Sluis, Jong, & Leij, 2007) while a meta‐analysis of Jacob and Parkinson (2015) shows a clear and stable medium effect size of shifting on reading and mathematics, based on 28 studies. The reason why the findings of Espy et al. (2004) differ from Jacob and Parkinson's may stem from the fact that this research was conducted in preschool children, where performance on both EFs and outcome measures might be hard to measure. Another reason might stem from the fact that we tend to expect individuals with good EFs to have a high score on all three EFs. However, it is likely that individuals with high inhibition abilities have low shifting abilities as their strong intent to maintain a task and not be distracted makes it difficult to flexibly switch to another task (Miyake & Friedman, 2012).
A study (Best et al., 2011) investigating the relation between EFs and learning performance from ages of 5 to 17 in a large sample reported on complex EF, which can be seen as common EF, as mentioned earlier. Performance on complex EF was positively related to learning performance and was clearly and developmentally similarly related to reading and math, suggesting a domain‐general contribution of executive functioning. In conclusion, these results indicate positive relations between EFs and learning performance, although the relation between shifting and learning performance is less clear than for updating.
Research in adults is scarce, as most research on learning has taken place in formal traditional educational settings at the stages of childhood and early adolescence (i.e., K‐12). Research in what some call “adults” is mostly focused on learners between 18 and 25 years old, a phase that is often referred to as “late adolescence” (Veroude, Jolles, Croiset, & Krabbendam, 2013). In late adolescence, working memory was predictive for learning performance, but this relation was not significant after controlling for general cognitive ability (Rohde & Thompson, 2007). However, a recent review suggests that working memory and EFs uniquely contribute to learning performance, even when general cognitive ability is taken into account (Titz & Karbach, 2014). This means that, despite the fact that working memory has a strong relation with the EF updating, EFs are unique predictors for learning performance, regardless of general cognitive ability. A study investigating self‐reported ratings of EFs showed that these adolescents who had better self‐reported EFs attained more course credits (Baars, Nije Bijvank, Tonnaer, & Jolles, 2015), another strong indicator for the importance of EFs in learning. Other studies using self‐reported ratings of EFs in learners in late adolescence showed that deficits in EFs were negatively predictive for learning performance (Knouse et al., 2014) and self‐control contributed to objective and subjective measures of learning performance, irrespective of cognitive ability (Stadler, Aust, Becker, Niepel, & Greiff, 2016). This all emphasizes the importance of EFs for learning performance, also in older students.
Processing speed was also considered in the present study because an aging population was investigated. Processing speed is important because many cognitive processes depend on processing speed (Albinet, Boucard, Bouquet, & Audiffren, 2012). Taking this into account gives more interpretable information on the unique contribution of the EFs measured. Second, as aging causes cognitive processes to decline, the decline in processing speed can influence learning performance. Although the effect of aging is independent and larger for processing speed than for EFs (Albinet et al., 2012), it can show insight in cognitive age‐related decline in learning performance.
The aim of the present study was to investigate whether single EF tests can serve as predictors for learning performance in mainly young and middle‐aged adult students participating in formal university‐level distance education. The EF tests measured shifting and updating. Processing speed was taken into account as cognitive aging plays a role in this population. Based on the research findings presented above, it was hypothesized that
- shifting performance is positively predictive for learning performance;
- updating performance is positively predictive for learning performance; and
- processing speed is positively predictive for learning performance.
METHODS
Design
The data in this study come from the Adult Learning Open University Determinants (ALOUD) study. This observational study was carried out among students of the Open University of the Netherlands (OUNL), an institute providing formal university‐level distance education to adults. The ALOUD study is an investigation of different psychological and biological factors possibly affecting cognitive performance and/or learning performance in adult students participating in distance education (Neroni, Gijselaers, Kirschner, & de Groot, 2015). Previous publications on the ALOUD study can be found here (Gijselaers, Barberà, Kirschner, & de Groot, 2016; Gijselaers, Kirschner, & De Groot, 2016; Gijselaers, Kirschner, & de Groot, 2015; Gijselaers, Kirschner, Verboon, & de Groot, 2016). Data from the biological part of this project, including the data for the present study, are available and stored permanently on DANS EASY, a sustainable platform for archiving research data (Gijselaers, 2015).
Participants
In the ALOUD study, throughout 1 year (6 August 2012–5 August 2013), all new OUNL students who signed up for one or more regular bachelor or master level course(s) were invited to participate. At that time, students could register and start throughout the entire year as the education was modular and self‐paced, open to everyone (≥18 years), and the curriculum was not fixed. The OUNL delivers primarily online education.
The approached population size was 4,945 students; 57.5% of those approached responded (N = 2,842) and 41.3% of those approached fully participated (N = 2,040). From the sample of students that fully participated, the majority studied part‐time as most students had a full or part‐time paid job (85.2%). Most students either lived alone (20.4%), with a partner (27.6%), or with a partner and children (34.3%). A smaller portion (17.7%) had other living situations (i.e., alone with children or with (grand)parents). The age of participants ranged from 18 to 80, with the largest part (56.9%) being between 26 and 45 years old. These participants are similar to the general population of students who normally study at the OUNL (Moerkerke, 2014).
Attrition rates in this population are high; more than 50% of the responders in the investigated population did not complete any course after 14 months, and many of them reporting not having started studying. As the goal was to predict learning performance, including students without any value on the learning performance measures because they did not study could confound possible relations. However, excluding all students without a value on the learning performance measures (i.e., those who had not received a grade for the course followed) is not desirable as they may have studied, but without successfully finishing a course. To make a valid data selection, an official examination attempt was used as a proxy of having studied. This way, the students who had purchased a course but who never attempted to officially finish it or who did not intend to attain course credits (i.e., who bought the course purely out of interest) could be excluded. The information on exam attempts was provided by the exam registration office.
The ALOUD study was approved by the OUNL ethical assessment committee. Each participant signed a digital informed consent form, explicating the use of the personal data gathered, voluntary participation, possibility to withdraw at any time, and finally giving their permission to use the data for the described goals. Participants had to click a check‐box to agree with the terms mentioned; a mandatory action to start the survey.
Procedures
Two to three weeks after registration at the OUNL, students received an invitation to participate in the ALOUD study. A survey was administered online to provide information regarding psychological, biological, and background variables using LimeSurvey®, version 1.92+ (LimeSurvey Project Team/Carsten Schmitz, 2012). Cognitive performance tests were located at the end of the survey. Full participation lasted 45–60 min on average and it was possible to stop at any time and continue later, allowing participants freedom in their participation by spreading the time burden. Participants who fully participated could win (5% chance) a gift voucher of €20. Over 14 months, learning performance measures were derived from objectively measured performance data from the exam registration office. The time period of 14 months was chosen because this was the standard subscription period when registering for a course. More information can be found in the data paper of the ALOUD study (Neroni et al., 2015).
Materials
Dependent Measures
As the education at the OUNL is modular and self‐paced, an operationalization for learning performance such as the common grade point average was not possible. Therefore, learning performance was operationalized as two separate measures: study progress and academic performance. Study progress was operationalized as the number of successfully completed study modules in 14 months (i.e., the standard subscription period when registering for a course). A course at the OUNL consists of one or more modules. One module is equal to 4.3 European Credits (EC) in the European Credit Transfer System (ECTS). The nominal study load for one module is approximately 120 hr. Academic performance was calculated for each course per student separately. A mean score of all obtained examination grades within a course represented academic performance for that particular course. A grade is a score between 1 and 10, with 10 being the best possible score. Both learning performance measures were derived from objectively measured learning performance data provided by the exam registration office of the OUNL. The assessments of most courses measured in this study were timed computerized exams which students had to perform at one of the 21 study centers of the Open University located throughout the Netherlands and Belgium.
Independent Measures
Cognitive performance was measured via an online digital cognitive test battery collected after the survey. Three tests were administered in the following order: (1) the Trail Making Test (TMT; Army Individual Test Battery, 1944); (2) the Substitution Test (ST), which resembles the symbol digit modalities test (A. Smith, 1991); (3) and the N‐back task (NBT; Lezak, Howieson, & Loring, 2004). In total, the tests took an average of 15 min to complete, including the time spent on instructions. Training sessions were included to make the participant familiar with the tests. As a pointing device was needed for the TMT and the ST, participants were instructed to only execute the tests when they had a mouse connected as an external pointing device. No other devices were indicated as permitted pointing devices (e.g., trackballs, track points, touch pads).
The TMT consisted of an A and a B part, which each consisted of one training session and one test session, leading to four parts in total. The A part involved clicking randomly placed numbers as quickly as possible in the correct order (i.e., 1, 2, 3, etc.), and the B part involved clicking randomly placed numbers and letters as quickly as possible in the correct order in a shifting mode (i.e., 1, A, 2, B, 3, C, etc.). Both parts were preceded by an instruction and a practice session. For both the A and B part the practice session consisted of 7 items, while the test session consisted of 25 items. After every session, feedback was given on the time on task and performance. The TMT resulted in a measure for the EF shifting, namely by subtracting the A part from the B part (i.e., B to A, in seconds). This provides a relatively pure indicator of task‐switching ability that minimizes for working memory and visuoperceptual demands (Sánchez‐Cubillo et al., 2009).
The ST consisted of two parts, namely one training session and one test session. The participants had to match a symbol shown to them with the correct number from a key on the top of the page. The numbers one to nine were shown on the bottom of the page in a 3 × 3 design. After clicking any number, the next symbol came up. Participants were instructed to substitute as many items possible in 90 s. The total number of items was unrestricted. After every session, feedback was given on the performance. The outcome measure in the ST was the number of items correctly substituted, which is a measure of processing speed. This ST mainly measures perceptual processing, visual search, and involves a motor component (e.g., Sánchez‐Cubillo et al., 2009; Shum, McFarland, & Bain, 1990).
The NBT consisted of four parts: three training sessions and one test session. The participants performed a 2‐back task with 60 items, in which they had to indicate whether a number shown to them was identical to a number shown two trials earlier. Participants had to place their left index finger on the letter A and their right index finger on the letter L of their keyboard. A was [Yes] and L was [No] in answer to indicating whether the number presented was identical to the number shown two trials earlier. For the first two numbers, participants were instructed to hit No (i.e., the letter L) as no previous trials were present. In the left and right lower corner, the meaning of the letters (i.e., A = Yes and L = No, respectively) was depicted. For all three tests, participants were instructed to work as accurately and quickly as possible. After every session, feedback was given on the performance. In the NBT, the number of correctly remembered items is a measure for working memory and the EF updating. Updating tasks, such as the NBT, measure general working memory processes as well as unique substitution processes which are independent of working memory (Ecker, Lewandowsky, Oberauer, & Chee, 2010; Wilhelm, Hildebrandt, & Oberauer, 2013).
Covariates
The list below provides information on the covariates, how and why they were measured, where they originated from and how they were calculated, where relevant.
- Number of modules: Each course consisted of one or more modules. One module is equal to 4.3 EC (i.e., 120 hr of studying). Therefore, the duration of courses could be different, depending on the number of modules. This could result in differences in academic performance.
- Age: Memory performance declines with increasing age (Grady & Craik, 2000), possibly hampering learning efficiency and also possibly confounding relations as it is associated with cognitive performance (Albinet et al., 2012). Age was measured using reported date of birth and was calculated in years (i.e., with two decimal values).
- Sex: Sex differences in intelligence (e.g., in the fields of memory, reasoning, science) have been found which could influence study progress. Also, there are intellectual domains where males are reported to exceed females (e.g., spatial reasoning) and vice versa (e.g., verbal fluency) (Halpern, 1997).
- Nationality: Education is not “culture-free” and as such nonnative Dutch people could have more difficulties with the more cultural elements in the educational system. Further, almost all courses at the OUNL are in Dutch. Participants were asked whether they were Dutch. If not, participants could enter their nationality.
- Native language: Nonnative Dutch speakers could have difficulties with the language of study. Participants were asked whether their native language was Dutch. If not, they could enter their native language. These manually entered data were inspected and if necessary recoded (e.g., some participants entered a regional Dutch dialect as their native language, which was recoded into Dutch).
- Body mass index: An increase in obesity is associated with a decrease in cognitive performance (Burkhalter & Hillman, 2011). Body mass index (BMI) was computed from self‐reported weight (in kg) and height (in cm) by the following equation: BMI = weight / height2 (kg/m2).
- Level of education: Level of education has been found to be a significant predictor of academic success for adult DE students (Bernt & Bugbee, 1993). It was measured as an eight-level ordinal variable following de Bie (1987) which is typical in Dutch research as these levels correspond with education levels in the Netherlands. These levels were (1) primary general education, (2) lower vocational education, (3) secondary general education, (4) secondary vocational education, (5) secondary higher education, (6) higher vocational education, (7) higher general education / university education, and (8) postgraduate / post‐university education. These eight categories were dummy coded into low (i.e., 1, 2, 3, 4), high (i.e., 5 and 6), and university level (i.e., 7 and 8), with low as reference category.
- Computer behavior: Students used an electronic learning environment which could be a disadvantage when one is neither computer literate nor fluent. The behavior towards using a computer was measured via a self‐developed 4‐point scale (ranging from fully disagree (1) to fully agree (4)), questionnaire mapping attitude (5 items), confidence (3 items), and skills (5 items). Due to the way the questions were phrased, six items were reversed. The minimum score was 13 and the maximum score was 52. A higher score indicated being better at using a computer and feeling more confident and positive towards using the computer.
- Alcohol consumption: Alcohol has been found to influence study progress (Singleton & Wolfson, 2009). Consumption was measured as the number of standard glasses of alcohol on work days and free days and was calculated to a weekly total.
- Life satisfaction: The Satisfaction with Life Scale (Diener, Emmons, Larsen, & Griffin, 1985) was used as more satisfaction has been found to be synergistic with better learning (Seligman, Ernst, Gillham, Reivich, & Linkins, 2009).
Analyses
Pre‐processing and the analyses were executed in SPSS (version 22; SPSS Inc., Chicago, IL, USA). A p‐value below .05 was considered to be significant. Before the analyses, all covariates were evaluated on the criterion of confounding. A confounder is a variable that threatens a study's internal validity because it is associated with one or more predictors and the dependent variable. Because a variable that is unrelated to either the dependent variable or the independent variable cannot distort the identified effect sizes, potential confounders were only included in the analyses as covariates if preceding bivariate tests indicated that they were associated with both one or more predictors and the dependent variable with at least a small effect size (e.g. r ≥ .1).
Outliers on the variables of interest (i.e., independent variables) were excluded before analyses following visual inspection. A covariate model was built including all relevant covariates (see the previous paragraph) yielding model A. Model B was built by adding shifting, updating, and processing speed to model A. A chi‐square model comparison was executed to evaluate whether model B had a significant better prediction over model A.
Though both analyses had the same model‐building procedure, the analyses were different. The study progress outcome measure revealed a negative binomial distribution. The positive skew and the variance‐to‐mean ratio being higher than 1 (i.e., 2.54) indicated overdispersion. Therefore, these data were analyzed using a generalized multiple linear regression with a negative binomial distribution (i.e., the GENLIN function in SPSS). The academic performance outcome measure was analyzed using a mixed model regression (i.e., the MIXED function in SPSS) as students were free in the number of courses they enrolled in and could start their study at any given moment, leading to a personal study path for each individual student. For this reason, a mixed model was used with student as a level and to account for the fact that grades were dependent within students. Each model was first built without taking the hierarchical structure of the data into account. Then random intercepts were added and a chi‐square model comparison was executed to evaluate whether adding the random intercepts added to the model. If this was the case, random slopes were added and a chi‐square model comparison was executed to evaluate whether adding the random slopes added to the model (Field, 2009).
RESULTS
Data Set Compilation
The original data set contained 2,842 cases. Participants were excluded if they: (1) did not complete the survey and the cognitive tests (1,228 cases); (2) made a remark at the end of the survey that led to exclusion (e.g., “I was distracted while doing the tests”; 85 cases); (3) performed below chance level on the NBT (45 cases); (4) had outliers as mentioned in the methods section (8 cases); and (5) did not attempt an exam (625 cases). All exclusions led to the analyses reported below with 851 students included.
Results
The descriptives for interval variables are depicted in Table 1. The descriptives for dichotomous variables are depicted in Table 2. Tables A1 and A2 in the appendix provide the results of the bivariate tests between the covariates and both outcome measures. These results led to the decision which covariates were included in model A in both of the analyses reported below.
| Variable | Mean | SD | Range |
|---|---|---|---|
| Study progress (successfully completed modules in 14 months) | 2.80 | 2.67 | 0–20 |
| Academic performance (average grade)a | 6.44 | 1.83 | 1–10 |
| Age (years) | 35.94 | 11.38 | 18–75 |
| Body mass index (kg/m2) | 23.69 | 3.71 | 15.4–50.1 |
| Computer behavior (scale score) | 42.99 | 5.68 | 16–52 |
| Total weekly alcohol consumption (standard glasses) | 3.53 | 5.71 | 0–70 |
| Life satisfaction (scale score) | 25.42 | 5.33 | 5–35 |
| Educational level (ordinal)b | 5.88 | 1.38 | 1–8 |
| Shifting performance (test score) | 19.69 | 12.44 | 0.0–104.3 |
| Updating performance (test score) | 55.81 | 5.30 | 32–60 |
| Processing speed (test score) | 50.71 | 7.97 | 28–86 |
- a Academic performance is a multilevel measure; for this table the average grade was determined per student over which the sample descriptives were calculated.
- b Educational level is measured on an eight‐level scale, ranging from low general education to post‐higher education.
| Variable | Number | Percentage |
|---|---|---|
| Sex | ||
| Male (0) | 322 | 37.8 |
| Female (1) | 529 | 62.2 |
| Nationality | ||
| Dutch (1) | 753 | 88.5 |
| Non‐Dutch (0) | 98 | 11.5 |
| Native language | ||
| Dutch (1) | 808 | 94.9 |
| Non‐Dutch (0) | 43 | 5.1 |
Table 3 displays the multiple linear regression results for the prediction of study progress. None of the covariates proved to be a possible confounder as no significant correlation between any of the covariates and study progress reached the minimal effect size of 0.1. Therefore model A is not applicable as there are no covariates to take into account (see Table A1 and A2). Model B was better than the null model (i.e., the intercept‐only model) as indicated by the chi‐square measure reported in the table. This means that the cognitive performance measures predicted study progress. Model B revealed that only updating performance was related to study progress. The effect size of the relation is small as an increase of one SD on updating performance is related to an increase of .091 standard deviations in the study progress measure. This means that one SD higher on the updating performance measure is related with 1.04 EC (.091 × 2.67 × 4.3 EC's per module) higher study progress.
| Predictor variable | β (standardized) | Significance (p‐value) |
|---|---|---|
| Model B (χ2 = 10.862, df = 3, p = 0.012) | ||
| Shifting performance | −.001 | .986 |
| Updating performance | .091 | .007 |
| Processing speed | .027 | .430 |
Table 4 displays the multiple linear regression results for the prediction of academic performance. Age, educational level, and native language proved to be possible confounders and were included as covariates in the analyses (see Table A1 and A2). The relation between all predictors and academic performance showed variance in intercepts across participants. The slopes did not vary across participants, leading to a mixed model with a random intercept estimated for each student. Model A contained all relevant covariates. The cognitive measures were added to form model B. The difference in chi‐square between the models was investigated to evaluate whether model B predicted academic performance better than model A. Model B was better than model A (χ2 (3, N = 851) = 21.371, p < .001). Thus, cognitive performance measures add to the prediction of academic performance. Model B revealed that both updating performance and processing speed were related to academic performance. The effect size of the relation is small as an increase of one SD on updating performance or processing speed is related to an increase of .177 or .176 standard deviations, respectively, in the academic performance measure. This means that one SD higher on both updating performance and processing speed is related to 0.32 (.177 × 1.83 and .176 × 1.83) higher academic performance.
| Predictor variable | β (standardized) | Significance (p‐value) |
|---|---|---|
| Model A (χ2 = 7,167.189, df = 6) | ||
| Age | .138 | .007 |
| Educational level | .243 | <.001 |
| Native languagea | 1.219 | <.001 |
| Model B (χ2 = 7,145.818, df = 9) | ||
| Age | .255 | <.001 |
| Educational level | .229 | <.001 |
| Native languagea | 1.143 | <.001 |
| Shifting performance | .066 | .232 |
| Updating performance | .177 | .001 |
| Processing speed | .176 | .010 |
- a These dichotomous variables were not standardized as this does not enhance interpretation.
DISCUSSION
The aim of the present study was to investigate whether performance on single EF tests for shifting and updating predicted learning performance in terms of study progress and academic performance in mainly young and middle‐aged adults. The shifting test was not predictive for learning performance. The updating test, on the other hand, predicted both learning performance measures. Performance on the updating test had a small effect size on study progress; students with a performance one SD higher had around 1 EC more study progress. Performance on the updating test had a small, but a twice as large, effect size on academic performance; students with a performance one SD higher had a 0.32 higher score on academic performance. Last, processing speed was taken into account as a measure of cognitive aging and was equally related to academic performance as compared to updating.
Shifting performance was hypothesized to be predictive for learning performance but was not. This is new, as there is no research where the relation between shifting and general learning performance measures is investigated, either in children or adults. Based on the common EF proposal of Miyake and colleagues (Miyake et al., 2000; Miyake & Friedman, 2012), one could argue that some common EF variance would be present in the shifting test used in the present study, as all three EFs share variance which lead to the common EF. If so, this still did not lead here to a contribution in predicting learning performance. This is possibly because inhibition is the largest contributor to common EF and shifting‐specific variation tends to show opposing correlations compared with common EF (Miyake & Friedman, 2012). This might make the global shifting task used in the present study—in which both common EF variance and shifting‐specific variance is captured—not associated with learning performance. Therefore, the question remains as to whether shifting is truly unrelated to learning performance. What is clear at this point is that a single task such as the shifting task used in the present study is not useful to predict learning performance.
Updating performance was hypothesized to be predictive for learning performance and also proved to be so for both learning performance measures. This is in line with previous research in children (cf., Best et al., 2011; Jacob & Parkinson, 2015) and adolescents (Titz & Karbach, 2014). This concerns research on general learning performance measures (Best et al., 2011), such as those used in the present study, as well as specific outcomes such as reading and math (Jacob & Parkinson, 2015). Performance on the updating test had an effect size twice as large for academic achievement than for study progress. Academic achievement is a measure which is more fine‐grained compared with study progress as it involves the average grades of courses within students, taking into account the dependence of grades within students. This might explain the larger effect size for updating performance when compared with study progress. Additionally, higher updating performance—meaning better control of the working memory—could be determinative for the final grade as more questions on an exam can be answered better within a certain time frame. Important to consider, however, is the fact that updating is strongly related to working memory (E. E. Smith & Jonides, 1999) and that working memory in its turn is strongly related to intelligence (Rohde & Thompson, 2007). This means that it is not possible to clarify whether the relationships are due to general intelligence or the EF updating. Nevertheless, the single updating task in this study proved to be a predictor for learning performance in adults, albeit with a small effect size.
Last, processing speed was taken into account as a measure for cognitive aging, which is likely to be present in this population. Processing speed was hypothesized to be predictive for learning performance and proved to be a relevant predictor for academic performance, but not for study progress. This might stem from the fact mentioned earlier: academic performance is more fine‐grained compared to study progress. More specifically, when taking an exam, it is very likely that processing speed determines the speed with which questions can be answered and thus how many questions can be answered within a certain time frame and hence is determinative for the final grade. This might explain why processing speed is predictive for academic performance and not for study progress. Although beyond the scope of the present study, it might be important to take processing speed into account as its effect size was comparable to updating, meaning both measures equally and uniquely contribute to the prediction of learning performance.
Finally, in accordance with the unity/diversity framework of common EF (i.e., unity) and EF‐specific variation (i.e., diversity) (Miyake & Friedman, 2012), it is not possible to clearly demonstrate how EFs are related to learning performance. This is mainly because it is very difficult or not even possible to extract the pure EF component of a cognitive test because multiple EFs can be involved in one task and EFs share variance via the common EF. For future research it is recommended to simultaneously measure inhibition, shifting, and updating and, in addition, take into account a complex measure that clearly involves all EFs to cover common EF.
The strengths of this study are multiple. First, the results were derived from a large data set which increases the power of the analyses and decreases the risk of Type I error. Second, more importantly, this type of study has never been done before in such a large age range capturing both the entire young and middle‐aged adulthood. Last, a number of possible confounders have been checked and controlled for. The study also had several limitations. First, not all EFs were measured and also intelligence was not controlled for. Second, it was observational, meaning no causal inferences can be drawn from the data, despite the fact that all hypotheses were theory‐driven. Third, the participants executed the cognitive tests on their computers, most probably at home or at their workplace. They were instructed to conduct the tests in a well‐rested active state and were able to execute the tests at a later time, meaning it was not necessary to do the tests directly after the survey, which lasted 45 min. Nevertheless, participants might have done the tests directly after the survey when they possibly were fatigued. Fourth, participants could be distracted by something or someone in their surroundings (e.g., partner, child, pet, media such as television or radio, computer applications such as Facebook®, and so forth) when executing the tests. Last, in terms of scientific content, the most imperative limitation is that inhibition was not measured in this study as it plays a vital role in EFs and specifically in common EF. Inhibition was not measured, as it was practically not possible at the time the data were collected to implement a suitable test.
CONCLUSION
The results presented here clearly show that the performance on a single updating test is predictive for learning performance in young and middle‐aged adults. Performance on a single shifting test proved not to be predictive for learning performance. Although inhibition was not measured in the present study, it is recommended that this EF is always measured when reporting on EF. From these results it can be concluded that a single updating test has predictive value for learning performance over a longer period of time. However, since the effect size is rather small, the single test on its own does not serve as a proper selection tool for determining whether a student will be successful or not.
APPENDIX
| N = 851 | Dependent variables | Independent variables | |||
|---|---|---|---|---|---|
| Covariates | Study progress (successfully completed modules in 14 months) | Academic performance (average grade) | Shifting performance (test score) | Updating performance (test score) | Processing speed (test score) |
| Age (in years) | .035 | .132** | .156** | −.599** | −.159** |
| Body mass index (kg/m2) | −.093** | −.004 | N/A | N/A | N/A |
| Computer behavior (scale score) | −.052 | .039* | N/A | N/A | N/A |
| Weekly alcohol consumption (standard glasses) | .059* | .116** | .050* | −.020 | .008 |
| Life satisfaction (scale score) | .094** | .062** | N/A | N/A | N/A |
| Educational level | .018 | .131** | −.028 | −.157** | −.024 |
| Number of modules | N/A1 | −.055** | N/A | N/A | N/A |
- Notes. All reported correlations are non‐parametric correlations (i.e., Spearman's rho), because distributions were not normal. N/A: not applicable; only significant correlations with a minimal effect size of 0.1 on the dependent variable are considered possible confounders and are subsequently tested in relation to the independent variables. The covariates printed in italic are possible confounders for academic performance. No covariates proved to be confounding for Study progress as no significant correlation reached the minimal effect size of 0.1. 1: Number of modules is only relevant for the Academic performance variable, not for the Study progress variable.
- * p‐value < .05.
- ** p‐value < .01.
| N = 851 | Dependent variables | Independent variables | |||
|---|---|---|---|---|---|
| Covariates | Study progress (successfully completed modules in 14 months) | Academic performance (average grade) | Shifting performance (test score) | Updating performance (test score) | Processing speed (test score) |
| Sex | −.019 | −.048* | N/A | N/A | N/A |
| Nationality | −.004 | −.005 | N/A | N/A | N/A |
| Native language | .082* | .137*** | −.154*** | .054* | .282** |
- Notes. All reported relations are effect sizes of non‐parametric t‐tests (i.e., Mann–Whitney), as the dependent variables were not normally distributed. N/A: not applicable; only significant correlations with a minimal effect size of 0.1 on the dependent variable are considered possible confounders and are subsequently tested in relation to the independent variables. n.s.: not significant. The covariate printed in italic is a possible confounder for academic performance. No covariates proved to be confounding for Study progress as no significant correlation reached the minimal effect size of 0.1.
- * p‐value < .05.
- ** p‐value < .01.
- *** p‐value < .001.
