The application of the challenge point framework in medical education

Introduction

‘People need adversity, setbacks, and perhaps even trauma to reach the highest level of strength, fulfilment, and personal development.’ This quotation is from Jon Haidt’s book, Happiness Hypothesis.¹ It may seem odd that an article about optimising learning and performance begins with a quote about happiness, but this notion of producing success out of adversity is the very foundation of a learning model known as the challenge point framework (CPF).² In short, this model states that the cognitive and/or physical systems must be appropriately challenged for optimal learning to take place. Critical in this model is the shift in focus from short-term practice performance to long-term learning. In fact, one of the interesting research findings in motor and cognitive learning is that depressed practice performance can lead to enhanced test performance. More to the point, the CPF has been shown to significantly increase learning such that, in test situations, individuals who learned with the framework consistently outperform those trained under traditional methods. These findings have been validated with skills such as surgery, golf, complex timing tasks and driving.^3,4 Additionally, several unique populations have been used to test the parameters of the model. These populations have included children, older adults, health professionals, patients with Parkinson’s disease, rehabilitation patients and adult control subjects.^5–7 These studies have routinely found positive benefits to using the framework described below.

In this article, we will overview research that demonstrates the paradoxical finding that practice performance does not necessarily reflect how much learning is taking place. This is followed by a detailed explanation of the CPF. The framework then leads into specific methods of practice or teaching for efficient learning. Finally, we discuss specific examples of how the methods can be used in classroom and clinical training settings.

Background and definitions

Classically, learning is defined as a relatively permanent change in performance resulting from practice or experience. One aspect of this definition, which is particularly important in the medical community, is that the relative permanence, or strength, of the learning should be demonstrated on a consistent basis including during situations of high stress. Indeed, these are the situations that truly test what has and has not been sufficiently learned. This suggests that learning is not demonstrated by practice performance but at some later time, usually by formal or informal assessments. In fact, one of the persistent and perplexing issues in any learning environment is the learning–performance paradox. This paradox, simply stated, is that you cannot determine how much or how well someone is learning simply by viewing practice performance. This translates to the idea that the way a student performs in the classroom does not predict how well the student will perform during test situations. In fact, it has been repeatedly demonstrated that certain practice methods artificially inflate practice performance and retard retention performance (learning).⁴ Other practice methods have been shown to hinder practice performance and to enhance retention, and yet others have been shown to hinder practice and retention performance. In essence, there appears to be a paradoxical relationship between practice and retention performance. Under certain circumstances they relate to one another in a direct manner (if one is good, the other is good) and in other circumstances they relate to one another in a reciprocal manner (if one is good, the other is bad). Therefore, one would presume that a rather complex formula between practice and learning is necessary to describe the relationship. Fortunately, this relationship can be modelled fairly simply.

The CPF for motor learning

In 2004 Guadagnoli and Lee² published a paper describing a model of practice and learning. The intention of the model was to describe the relationship between practice performance and learning, and to make predictions about how an individual’s learning will be affected by certain practice protocols. Since then, more than 100 papers testing the model and its predictions have been published, and these studies have routinely found positive benefits to using the framework described below.

The challenge point model starts with a simple graphic of the relationship between practice performance and task difficulty (Fig. 1). For the current paper, task difficulty is defined as the physical or cognitive challenge posed by a motor problem. This challenge can result from an individual’s perception (e.g. psychological factors) of task difficulty or the mechanical constraints (e.g. degrees of freedom) of the task. The solid line in Fig. 1 shows that as task difficulty increases, practice performance decreases; that is, as the motor problem becomes more difficult, immediate performance deteriorates. The point that practice performance decreases as task difficulty increases may not be dramatic, but it is important. The decrease in practice performance is the result of the task becoming more challenging and, we will argue, it is precisely this challenge that leads to improved long-term performance. A theme that will recur several times in this paper is that an appropriate degree of challenge will create some degree of failure. This appropriate failure during practice will yield success when it really counts during times of stress.

The dotted line in Fig. 1 extends the relationship between performance and task difficulty to include learning (i.e. retained skill). The learning curve demonstrates that as task difficulty increases, so does learning, at least for the first half of the curve. This increased learning continues until the optimal challenge point is reached. At this point, the learner is being optimally challenged and efficient learning can occur. Therefore, as task difficulty increases to the optimal challenge point, practice performance decreases. At the same time, learning is enhanced: that is, as challenge is increased, immediate performance is negatively affected, but long-term performance is enhanced. However, if the challenge is increased beyond the optimal challenge point, both practice performance and learning begin to suffer. In this latter case, the system is overtaxed and is likely to result in a negative cycle of stress. Indeed, evidence in neuroscience may provide insight into why various levels of challenge affect learning in this manner.

During exposure to challenging situations, one of the earliest signals of stress is the presence of the hormone corticotropin-releasing factor (CRF). This hormone is released in the hippocampus, the brain structure central to learning and memory.⁸ It has been hypothesised that CRF impacts our ability to remember performance information.^9,10 The nature of this impact is highly dependent on the difficulty of the practice situation. Specifically, under moderately stressful learning situations, increased levels of CRF have been found to increase skill learning.¹¹ What is or is not moderately stressful (and, hence, where the optimal challenge point lies) is largely dependent on the individual performing the skill.

For example, in a series of experiments using a feedback paradigm, Guadagnoli et al.¹² found that as a learner improves, optimal learning occurs when feedback is delayed rather than given immediately. Inexperienced individuals performed better during both practice and retention if they were given feedback more immediately. As subjects improved, the more immediate feedback continued to produce better performance during practice, but produced poorer retention performance; that is, longer delays in feedback yielded superior learning for the experienced individual, but inferior learning for the novice. Therefore, it was concluded that two factors were primarily responsible for determining what constitutes medium or appropriate challenge: performer expertise, and task complexity.

Because the optimal amount of challenge is learner-specific, it is not static. When an individual is in the early stages of learning, information should be presented in smaller units to allow efficient processing. Large units would overwhelm the learner. However, when a performer is in a later stage of learning, the cognitive system’s ability to group information improves and thus the learner can more efficiently handle a more demanding practice protocol.¹³ The overall efficiency of processing information is dependent on the difficulty of the task relative to the skill of the individual completing the task.

Consider Fig. 2. The figure shows the original curves seen in Fig. 1, along with a second set of curves. The original curves demonstrate the practice-learning curves of someone in the advanced stages of learning. The new set of curves demonstrates someone in the early stages of learning. The shapes of the curves are similar, but the optimal challenge point varies depending on expertise. For example, a simple incision may be of relatively low difficulty for the expert surgeon but of high difficulty for the first-year medical student. As such, practice protocols should be designed to maintain an appropriate relative task difficulty and this means that the practice should change as the learner changes. For example, for a novice performer learning a new skill, it is appropriate to present feedback after every trial or at least every few trials so that the learner can learn to compare performance with feedback. However, as learners get better at the task at hand, they should receive feedback less often, thus increasing the task difficulty by making experienced learners work out the details of the feedback for themselves. Again, as the learner changes, so should the practice protocol.

Feedback is not the only method by which task difficulty can be manipulated. Instructors might also manipulate the contextual interference of practice. Contextual interference is a term used to describe the interference that results from practising a variety of tasks within the context of a single practice situation.¹⁴ A low degree of contextual interference can be established by having the performer practise only one task within a block of trials (i.e. blocked practice). For example, attempting the same surgical suture time after time would constitute a low degree of contextual interference. A high degree of contextual interference can be established by having the learner practise several tasks in a random order (i.e. random practice). For example, attempting to perform incisions, sutures and clamps in random order would constitute high contextual interference relative to practising the same task on a repetitive basis. A series of studies using a blocked versus random manipulation demonstrated that novice subjects who practised under a blocked protocol performed better than novice subjects who practised under a random protocol.⁴ However, the opposite was true for experienced subjects. Experienced subjects who practised under a random protocol performed better than experienced subjects who practised under a blocked protocol. Presumably, blocked practice provided the most appropriate challenge for novice performers and random practice provided the most appropriate challenge for experienced performers. From these results, it was concluded that it is desirable to decrease extraneous challenge for performers in the early stages of learning. However, as the performer becomes more proficient, more challenge is beneficial. This study clearly demonstrated that efficient learning is based on challenging the performer appropriately, and this means implementing different practice strategies for different levels of ability.

Again, considering these behavioural observations in the light of recent neurophysiological findings makes the reason for increased learning for experts in random practice situations even clearer. As an individual learns a motor skill, different areas of the brain are preferentially recruited for performance of that skill.^15,16 In the expert performer, remodelling of the motor cortex has created a stable neural circuit in which the performance of the skill takes place.¹⁷ In the random practice situation, the stability of the performance circuit is believed to be interfered with, essentially redirecting the expert performer’s attention to task performance. This attention, in turn, enables the performer to recruit the areas of the brain used in initial skill acquisition for skill remodelling.¹⁸ In essence, random practice both increases the stress response in the expert performer to optimal levels by creating an appropriate release of CRF in the memory areas in the brain, and enables re-activation of the areas involved in initial task learning. As noted, contextual interference is not the only way to create appropriately stressful practice situations.

One final factor to consider (consequence) is the psychological, as opposed to the physical, aspect of the task. In this case, consequence is defined as the cost or benefit of successfully or unsuccessfully completing the task. In the operating theatre, the consequence of failure is grave. It would be inappropriate to put a day 1 medical student under that type of challenge. However, it would be equally irresponsible not to put a Year 4 student under significant consequence in his training. Just like the other factors discussed, the degree of consequence should increase as the learner’s proficiency increases.

The general point is that the learner needs to be challenged at an appropriate level for learning to occur. Under-challenging the learner will create a situation of good practice performance and poor ‘game time’ performance. Over-challenging will result in poor practice and poor ‘game time’ performance. The appropriate level of challenge will create some degree of failure in practice, but will be likely to yield long-term success. Table 1 summarises ways in which task difficulty can be increased to facilitate training for long-term success. Again, it is important to note that these manipulations should match the performer’s ability. That is, in an early stage of learning, the system is inefficient in grouping multiple task elements. Therefore, the information should be presented in more appropriate units to enable efficient processing. Situations that include the provision of more immediate feedback or less practice variability may provide information in units that are more appropriate for the novice.

Applications of the CPF

Guadagnoli and Lee² made several very specific predictions related to practice efficiencies and challenge point. Many of these predictions have been tested and, as a result, can be applied to skills such as surgical training.

The first set of predictions is based on contextual interference. As a reminder, contextual interference is a term used to describe the interference that results from practising a variety of tasks within the context of a single practice situation.¹⁴ A high degree of contextual interference can be established by practising several tasks in a random order (i.e. random practice).

Guadagnoli and Lee² predicted that for relatively simple tasks, blocked practice would be beneficial for practice performance, but random practice would be best for actually learning the skill. This prediction has been borne out in previous studies of motor learning⁴ and can be applied to surgical techniques.¹⁹ Specifically, for relatively simple or repetitive tasks, random practice will challenge the learner during practice, but will serve the learner best for learning.

A second prediction related to contextual interference stated that for relatively inexperienced individuals, blocked practice would be beneficial for practice performance and learning the skill. Random practice, by contrast, might overwhelm the individual. However, as the learner progresses in ability, the practice schedule should become more randomised to challenge the learner. As this happens, the appropriately increased challenge will enhance learning. For example, attempting to perform incisions, sutures and clamps in random order would constitute high contextual interference relative to practising the same task on a repetitive basis. The high contextual interference would serve the learner best in solidifying the skills in an automatic fashion.

The relationship between the individual and appropriate task challenge is summarised in Table 2.

Another prediction derived from the CPF is based on the amount of feedback given to the learner. Guadagnoli and Lee² predicted that for relatively simple tasks, or for highly skilled individuals, a high frequency of feedback would be beneficial for practice performance, but would in fact retard long-term retention.²⁰ Specifically, in relatively simple or repetitive tasks, learners tend to rely on a high frequency of feedback rather than on processing task-related information themselves. As such, individuals are learning to manipulate performance based on feedback rather than learning the skill itself.

A second prediction related to feedback stated that for relatively inexperienced individuals, or for tasks of high complexity, a relatively high frequency of feedback (every two or three trials) would be beneficial for both practice performance and the learning of the skill. Withholding feedback under these circumstances might deprive the individual of the information he or she needs to refine the skill.

As with contextual interference, as the learner progresses in ability, the practice amount of feedback should decrease. As this happens, the appropriately increased challenge will enhance learning. In short, the challenge should match the learner’s ability and therefore as ability changes, so should the specific challenge.

The relationship between the individual and appropriate task challenge is summarised in Table 3.

Guadagnoli and Lee² made specific predictions regarding only contextual interference and feedback frequency. However, it is logical that the CPF is applicable to several other factors of practice and learning. These factors will be briefly discussed.

Environment of practice

At the level of individual skills, the CPF would suggest that in addition to changing the practice and feedback schedule as the learner changes, the environment of training (e.g. surgical skills laboratories, simulators, standardised patients) should be manipulated for the learning of the cognitive and technical aspects of procedural tasks.^21,22 For example, the initial learning environment should consist of a controlled practice area, thus minimising outside distractions.^23,24 This allows the learner to focus on the salient features of the task. As the learner progresses in skill, the environment should become more dynamic until it eventually simulates a real-world medical setting.

Consequence of practice

Clearly, the physical and cognitive aspects of skill acquisition are important. However, we would be remiss if we did not discuss the psychological aspect of skill performance. The way a learner performs in training does not predict how well the student will perform in a real-world situation. Just as the practice conditions and environment are manipulated, the level of stress under which an individual learns should change as the learner’s expertise changes. For complex tasks or unskilled individuals, outside stress and consequence should be kept to a minimum. However, as the learner progresses in skill, the level of stress imposed and the consequence of his or her actions should increase until they eventually simulate those of a real-world medical setting.²⁵ Indeed, practice conditions, environment and consequence manipulations can be summarised under a rubric declaring that as the learner changes, so too should practice such that the individual is consistently challenged in order to optimise learning rather than to optimise practice success.

Conclusions

The CPF discussed in this paper is based on the notion of generating success out of planned adversity. The model states that the cognitive and physical systems must be appropriately challenged for optimal learning to take place. It is emphasised that for the most efficient and stress-resistant learning to occur, there must be a shift in focus from short-term practice performance to long-term learning. At its heart, the CPF is consistent with the competency-based progression recommended by American Surgical Association Blue Ribbon Committees, whereby increases in challenge are based on level of performance rather than time-on-task. This suggests that short-term stress and failure should be promoted to enable long-term success.

We opened this article by quoting Jon Haidt: ‘People need adversity, setbacks, and perhaps even trauma to reach the highest level of strength, fulfilment, and personal development.’¹ It is our hope that if this notion of adversity and the CPF are applied in medical education and training, their application might enhance the speed with which professionals learn and improve performance in critical situations.

Contributors:  MG conceived the primary version of the challenge point framework (CPF) and drafted the first half of this paper reviewing the CPF, thus providing the critically important intellectual content. M-PM and AD made substantial contributions to the design of the CPF application as it is described in the second half of the paper and drafted the section on application. All authors approved the final manuscript for publication.

Acknowledgements:  none.

Conflicts of interest:  none.

Ethical approval:  not applicable.