- Top of page
Medical Education 2011: 45: 192–199
Context Testing increases memory of a topic studied more than additional study or training. The mechanisms by which this occurs are not clearly understood. Testing can be stressful and studies suggest that the stress hormone cortisol has modulating effects on memory, predominantly in men. The aim of this study was to investigate whether cardiopulmonary resuscitation (CPR) skills testing induce a cortisol increase, whether the cortisol increase enhances retention of CPR skills, and how this relates to gender.
Methods We randomised a convenience sample of medical students attending a mandatory course to one intervention and one control group. Students received a 4-hour course on CPR skills. During the final half-hour of the intervention course, participants were tested in CPR scenarios, whereas the control group underwent additional training. We assessed learning outcomes 2 weeks later by rating student performance in a CPR scenario using a checklist and a single blinded assessor. We measured salivary cortisol pre-course, half an hour before the end of the course and post-course, and compared learning outcomes and cortisol responses between groups and genders.
Results In total, 146 of 202 (72%) students completed the study. We found a significant difference in learning outcome between the intervention and control groups for both genders (mean ± standard deviation, 5.0 ± 3.5; p = 0.006). We found a significant effect of increase in cortisol on learning outcome in men. The correlation between learning outcome and cortisol increase was medium to large for men (r = 0.38), but not for women (r = − 0.05).
Conclusions Cardiopulmonary resuscitation skills testing induces a rise in cortisol in men, which is related to the better retention of skills in men. Cortisol modulates test-enhanced learning in men.
- Top of page
Test-enhanced learning relates to the fact that testing increases memory of a topic studied more than additional study or training. This phenomenon pertains to both knowledge1 and skills2 learning. Although test-enhanced learning is a well-studied and robust phenomenon, the mechanism behind it is not clearly understood.
Roediger and Karpicke suggest that complex mechanisms known from psychonomics, such as transfer-appropriate processing and elaborate retrieval, are possible explanations. Transfer-appropriate processing suggests that if the testing situation and thus processing are sufficiently similar to a later transfer situation, retrieval is facilitated. Elaborate retrieval suggests that the harder it is to retrieve a specific element from memory in the intervention situation, the easier it is to retrieve in an assessment or transfer situation later.1 However, it is possible that these mechanisms do not entirely explain test-enhanced learning. In our earlier studies of the testing effect on skills learning,2,3 we assumed that the acute stress that pertains in the testing situation could be a factor of importance.
A well-known physiological response to acute stress is the release of cortisol from the adrenal cortex. The literature on the effect of cortisol on learning is extensive, in both animal models and human studies. Researchers in neurobiology, learning and cognition describe the effect of the stress hormone cortisol on memory.4–6 An extensive meta-analysis by Dickerson and Kemeny concluded that the social-evaluative threat and lack of control experienced by the testee when being tested will cause medium to large increases in cortisol.7 Therefore, it is possible that an elicited cortisol response might contribute to test-enhanced learning.
However, the effects of increased cortisol levels can be both positive and negative in terms of learning.8,9 The positive effect of stress-level cortisol is that it facilitates memory consolidation, the process by which a memory is stabilised after its initial acquisition. The negative effect of stress-level cortisol is impairment of memory retrieval10 (i.e. the ability to access stored memories). It has been suggested that, as a result of cortisol, the brain is in learning mode and thus retrieval is impaired.11,12
Testing has been demonstrated to enhance learning outcomes1,2 and to increase cortisol levels.13 However, until now any possible link between test-enhanced learning and cortisol response has not been studied. Hence, the aim of this study was to explore the connection between test-enhanced learning and cortisol response.
It is well known that there are differences between men and women with regard to the effects of cortisol and that men are more susceptible to the effects of cortisol on memory and learning.4–6,14 Given that men and women may have different responses to stress and thus to cortisol levels, we also aimed to investigate differences according to gender.
This study was set in the context of a cardiopulmonary resuscitation (CPR) skills training course for medical students. Our research questions were:
- Top of page
Our results demonstrate a positive effect of testing on learning outcome. In men this effect was in part related to an increase in cortisol.
A wealth of studies have demonstrated test-enhanced learning1 and some have shown the effect of cortisol on learning.20 However, to our knowledge this study is the first to combine these two concepts to explore whether an increase in cortisol might explain test-enhanced learning. Our results show that there may be such a connection. The following discussion relates firstly to the results pertaining to test-enhanced learning, secondly, to the effect of testing on cortisol and, thirdly, to the connection between test-enhanced learning and the increase in cortisol. Finally, we address the practical implications of the findings of this study.
The testing effect
The demonstration of test-enhanced learning in this study is in accordance with our earlier studies on skills learning2,3 and findings on knowledge learning.1 However, other factors, such as intention to learn, preparation for a retention test and observational learning, may have influenced the results. Although students in the intervention group knew they were about to be tested at the end of the course, we doubt that this influenced their intention to learn and hence increased their learning outcome. According to Crocker and Dickenson, there is no difference in learning outcome between intentional and unintentional learning if the participants are not allowed to rehearse in the retention period.21 The tight course schedule and standardised format were similar in both groups and participants in the intervention group did not have an opportunity to engage in self-directed training. Moreover, none of the participants reported having rehearsed in the retention period. Hence we do not find that intention to learn and subsequent rehearsal threaten the results.
Observational learning is known to be an important part of skills learning.22 However, in this study the amounts of observational learning taking place in the intervention and control groups were probably equal as the last half-hour in both groups was dedicated to basically the same scenarios and the same modus of feedback. The only difference was that the control group was undergoing training, whereas students in the intervention group were being tested individually, without being told the results. Hence we do not find observational learning to threaten the results.
Test-induced cortisol elevation
Diurnal variations and considerable interpersonal variations in SCLs represent a threat to any analysis of SCL measurements. A wide range of factors such as time of day, female menstrual cycle, use of oral contraceptives and hormone replacement therapy, pregnancy, lactation and breast-feeding, smoking, alcohol consumption, dietary energy supply and possibly caffeine may affect salivary cortisol responses.13,16 It was not feasible to standardise the design of the current study according to all these factors. However, we anticipated that these possible biases would be evenly distributed between the groups following the randomisation procedure and chose the deltaSCL measured over a narrow time span (20–40 minutes) as outcome variable. Given the great variability in SCL, we expected some results to be measurement errors and therefore removed outliers that were ≥ 3 SDs from the mean. However, post hoc analysis demonstrated that the ancova model obtained was robust with regard to outliers. Saliva samples were analysed in multiple runs and thus inter-test variability (< 10% according to Rigshospitalet’s validation report) may have had some impact on the results. However, interassay variability of 10% must be considered acceptable in comparison with other saliva cortisol studies.23–25 Hence, although we acknowledge the possibility of measurement errors, we feel confident in our results regarding the effect of testing on cortisol.
Referring to neurobiological studies on psychosocial stress and acute cortisol response, we theorised that the test situation used in our study would induce an acute cortisol response comparable with that provoked by a standardised psychosocial test (e.g. the mental challenge component of the Trier Social Stress Test [TSST]).16 We found a significant cortisol response to the testing situation, but only in men, which is in accordance with the conclusions proposed in a major review by Kirschbaum and Hellhammer.16 However, the test-induced elevation of cortisol in our study was on average equivalent to only 10% of the elevations mentioned in the review article.16 This may be explained by the fact that the test situation was of no consequence to our participants and perhaps not as stressful as the highly standardised TSST.
Test-enhanced learning, cortisol and gender
Our results indicate the presence of a connection between learning outcome and acute cortisol elevation in men, but not in women. A thorough review by Kudielka et al.13 showed that healthy males elicit a stronger and more uniform cortisol response to stress, whereas the response in females depends on menstrual cycle and use of oral contraceptives.
However, when we examined the extensive literature on test-enhanced learning, we found no studies on differences between genders in learning outcome.1 It seems unlikely that such a gender difference has passed unnoticed through decades of research in test-enhanced learning. With regard to the current study, we can only state that the literature supports our results indicating that test-enhanced learning in males is modulated by cortisol to some extent. However, whether test-enhanced learning is a male phenomenon must be explored further, either by re-examining older studies or by future research.
Test-enhanced learning has been recommended for inclusion in instructional strategies.1 This may be particularly relevant for simulation-based skills training. Medical simulation facilities are expensive and the retention of CPR skills is known to be poor.26 Therefore, whatever can be done to make teaching in simulation facilities more effective should be implemented in order to save money and lives. Against this background, we have earlier argued that tests should be implemented in CPR courses.3 However, the gender-related results of this study disturb this picture. Medical universities all over the world have male : female student ratios of approximately 40 : 60.27 We consider that it is vital to report differences between genders in future studies of test-enhanced learning. A possible gender difference may undermine the educational value of the testing effect and this issue requires further investigation.
Implications for research in medical education
Issues related to test-enhanced learning have important implications for research designs. When administering both an immediate post-test and a later retention test to the same group, researchers should be aware that the immediate post-test will interfere with performance on the later retention test and thus a true learning outcome will be difficult to estimate from this design.1,2 Moreover, an immediate post-test does not necessarily correlate to long-term learning outcome.1 Finally, any cortisol elevation during an intervention or control situation may obscure the true results by impairing retrieval for about an hour in most males and some women.16 Hence, when choosing a research design, it is important to observe that testing may influence learning and therefore may bias the results.
The participants in this study were medical students. The sample was relatively homogeneous with regard to age and educational level. Consequently, the findings do not necessarily apply to other groups at higher levels of training, different age groups or other groups of medical professionals.
The convenience sample of 146 of a population of 202 medical students at exactly the same educational level can be considered a weakness, but was necessary for feasibility reasons. We would have preferred to include more than 80% of the population, but, as we prioritised the testing of all participants within 24 hours and at 2 weeks after the course to avoid any time-related bias, we had to leave out 56 participants who were unable or unwilling to participate in follow-up.
A simulation scenario in CPR skills learning is an activity that potentially affects the emotional state of participants. The effects of stress and cortisol on memory mainly pertain to elements that carry emotional valence.28,29 There is probably an emotional valence in CPR training and hence we caution against the generalisation of our findings to other skills or pure knowledge learning. To our knowledge, we are currently the only group reporting on the testing effect in skills learning and, until our results are reproduced elsewhere, our findings suffer the disadvantage of having no direct comparison.
Contributors: CBK contributed to the study conception and design, the acquisition, analysis and interpretation of data, and to securing funding for the study. MLJ contributed to the study conception and design, and to the acquisition and interpretation of data. CR contributed to the study conception and design, the analysis and interpretation of data, and to securing funding for the study. All authors contributed to the drafting and revision of the article and approved the final manuscript for publication.