SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

Medical Education 2012: 46: 49–57

Context  In response to historical trends in expectations of doctors, the goals of medical education are increasingly framed in terms of global competencies. The language of these competencies has tended to adopt a prescriptive, rather than descriptive, approach. However, despite widespread agreement on the importance of competency-based education and more than two decades of study, this effort has not generated a dependable set of assessment tools.

Discussion  Because models of competency are legislated, rather than shaped by scholarly consideration of empirical data, it is unlikely that such models directly reflect actual human behaviour. Efforts to measure clinical behaviours could benefit from increased clarity in three related conceptual areas. Firstly, the language of educational constructs should be framed in terms of data-based hypotheses, rather than in terms of intuitively plausible abilities. Secondly, these constructs should be specified in terms of the situations to which they are relevant, rather than as global personal characteristics. Finally, the resources required to measure these constructs should be rigorously established because a common resource-based metric would allow for rational selection of assessment methods. Specific methods to establish each of these objectives are discussed.

Conclusions  The political process of negotiating educational objectives should not be confused with the scientific work of establishing coherent and interpretable patterns of behaviour. Although the two activities can complement one another, each has its own distinct methods and style of discourse. It is thus critical to maintain boundaries between these two approaches to defining professional performance.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

Throughout the 20th century, formal assessment of medical trainees was based almost entirely on written tests of knowledge. A passing score on a multiple-choice examination, in combination with the successful completion of a training programme, was all that was required to demonstrate readiness for clinical practice. The past 20 years, however, have witnessed a widespread consensus that mere mastery of factual knowledge is not sufficient to ensure clinical competence. It would now be difficult to find a medical educator who does not agree with the spirit of the Miller pyramid,1 which sees ‘knowing’ not as a final destination, but rather as a first step on the long journey towards ‘doing’. The various higher-order ‘doing’ skills (e.g. professionalism, cultural competence, clinical reasoning, etc.) have come to be known as ‘competencies’.

More than two decades of work, however, have not produced a widespread consensus on how to define these competencies, let alone on how to assess them. The recent 100th anniversary of the Flexner Report, for instance, occasioned the publication of several articles calling for a modern Flexnerian clarity about goals and assessment in medical education.2–5 Others argue that competencies have always been with us as an organic part of medical practice, and that attempts to measure these complex attributes only trivialises them.6,7 Leaders of the competency movement have acknowledged that medical educators remain ‘nihilistic’ in their response to competency-based assessment.8

Why has it been so difficult to crystallise a general consensus about ‘doing’ into a set of specific, well-accepted and measurable competencies? In this article, I argue that a core problem is that discussions of competencies are typically framed in prescriptive and organisational terms. This approach confuses what is desirable with what is observable. By contrast with current prescriptive models of competency, a descriptive approach would conceptualise assessment in the language of naturalistic and measurable patterns of behaviour. Such an approach to assessment would promote evidence-based consideration of empirical data as an alternative to the current methods of values-based discourse.

I begin by examining the historical and social context of the competencies movement. I then describe how the resulting prescriptive approach to competencies can only generate impracticable approaches to assessment. I describe some elements of a descriptive, naturalistic approach that could lead to greater clarity about what to measure in trainees and practitioners, and how to interpret the results of such assessments. I then discuss the relationship between political leadership and the science of assessment.

Medical education in the early 20th century

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

It would be difficult to find a reference as universally recognised, yet so seldom actually read, as the Flexner Report.9 The ‘report’ is in fact a book-length compendium of data, anecdote and rhetoric, presented in the orotund and encyclopaedic style of the time. The modern reader, accustomed to tabbing through executive summaries for crisply bulleted goals and objectives, will search the report’s hundreds of detailed pages in vain for a clearly articulated ‘vision’ of the outcomes of medical education. It is clearly the product of a different time and culture than our own.

University-based Flexnerian medical training is nonetheless still with us, but the original values and mental habits of its originators can now be reconstructed only by an act of historical imagination. Although the value of selfless devotion to scientific principles is implicit in the Flexner Report, there are only passing hints of higher-order attributes such as rational judgement, self-sacrifice and diligence. There is absolutely no mention of how one would assess these qualities. To the extent that such ‘moral virtues’ could be taught at all, they may perhaps have been seen as a natural result of rigorous scientific training. Given the assessment tools of the time, such attributes would have been viewed as unmeasurable in any event. Experienced faculty staff were trusted to recognise good students when they saw them.

It would scarcely have occurred to patients in the early 20th century to question the value of a scientific university-based medical education. Indeed, to anyone with living memory of the questionable characters who could obtain a medical licence in the pre-Flexnerian era, this new breed of doctors must have seemed positively heroic. Here is a passage from Sidney Kingsley’s 1933 Pulitzer Prize-winning drama Men in White,10 which expresses the sentiments of the time. A patient (Hudson) and an attending doctor (Hochberg) are discussing a medical resident who is about to begin a 5-year apprenticeship:

Hudson: ‘God, they make a slave of that boy. And he doesn’t even get a dime! I can’t see it.’

Hochberg: ‘He’s not here for the money! He’s here to learn. The harder he works, the more he learns. If he wanted to make money he wouldn’t have chosen medicine in the first place. You know, when he comes with me [for his 5-year apprenticeship], his pay is only going to be $20 per week, but there’s a chance to work. The man who’s there with me now works from 16 to 18 hours a day. He even has a cot rigged up in one of the laboratories where he sleeps sometimes.’

Hudson: ‘For $20 a week?’

Hochberg: ‘Yes, yes… George is a fine boy with great promise. The next 5 years are crucial years in that boy’s life. They’re going to tell whether he becomes an important man or not.’

In the first several decades of the last century, doctors were seen as having committed to an almost monastic life of science and of service. Upon completion of rigorous scientific study, they undertook yet more years of arduous clinical training. This latter phase was conducted almost entirely by hand, so to speak, and clinical wisdom was imparted directly from great teachers in the privileged and ancient relationship between master and apprentice. Only a select few had the intellect and dedication to complete those difficult years of training. Grateful patients could hardly have asked for more.

The modern era of competencies

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

Speculations about the dangers of a biomedically oriented education and unregulated postgraduate training began to appear in the 1960s. Critiques arose both from within the medical profession,11,12 as well as from outside observers.13 The heroic standards of historical texts such as Men in White10 gave way to increasingly sinister modern depictions of doctors in the popular media. Generally speaking, these decades saw a growing concern about the dehumanising consequences of biomedical training.

Some medical educators responded by providing home-grown experiences devoted to communication skills, medical humanities, cultural competence and the like. For a time, training institutions were free to navigate their own courses through these historical currents. Some schools were progressive, whereas others re-committed to traditional educational methods. During this period, medical schools were free to brand themselves according to their unique institutional priorities, such as those defined by the biopsychosocial model, rigorous clinical training, signature research experiences or a special humanistic orientation.

The most recent stage in this historical trend (beginning circa 1995) has seen the involvement of regulatory bodies, which are increasingly mandating universal teaching and assessment of various general competencies. Because such competencies are cast in global and overarching terms, they have proven challenging to define and to measure. The language of such competencies is typically derived by a consensus process involving large numbers of national experts, stakeholders and interested parties.14,15

Perhaps the most contentious aspect of this approach has been the mandate for every programme, irrespective of its unique culture and traditions, to demonstrate that its trainees have attained these universal competencies. The resulting top-down requirements threaten to intrude upon historically protected relationships among master clinicians and apprentices. Recent critiques predict that the liberal and learned practice of medicine will soon be overrun by simplistic checklists representing unproven managerial mandates.6 Competency-based assessment has been accused of discouraging critical thinking7 and of paradoxically trivialising the humanistic dimensions of care.16 Even supporters of the approach acknowledge widespread opposition to the idea of measurable competencies.8,17

At least for now, however, predictions about both the harms and benefits of competency assessment remain largely unrealised, as accreditation bodies have yet to mandate specific assessment technologies. In part, the lack of universal assessment tools may be attributed to resistance among faculty staff.17 There is also concern that the competencies have been defined by a political process, rather than an empirical one. As such, there is no guarantee that any measurement procedure can reflect the competencies.18 Theoretical attempts to map specific behaviours onto general competencies generally find that most current assessments are plausibly related to most hypothesised competencies.19–22 Thus, it is unclear what the competencies add to existing practices of assessment. For all these reasons, the competency approach does not yet seem poised to instigate a Flexnerian reorientation of medical education.

Taxonomies of competence

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

Accrediting agencies have responded by continuing to negotiate more specific and measurable behaviours from the language of general competencies. The Canadian CanMEDS approach defines seven superordinate ‘Roles’, which contain 28 more specific ‘Key Competencies’, which themselves are further elaborated in 126 subordinate ‘Enabling Competencies’.14 In the USA, the Accreditation Council for Graduate Medical Education (ACGME) is developing the concept of clinical milestones, based on its six ‘General Competencies’. These are categories of clinical skills in which trainees should demonstrate increasing mastery and which may someday dictate the rate at which they progress through training.8 A related approach envisions ‘entrustable professional activities’ (EPAs), which are specific behaviours that a competent practitioner should perform with an acceptable degree of skill and consistency.23

These subcompetencies – key competencies, enabling competencies, milestones, EPAs, and the like – represent the beginnings of classification schemes that would link abstract competencies to directly observable behaviours, via a series of intermediate taxonomic subcategories. A recent set of ACGME milestones for paediatrics trainees24 provides some indication of the level of abstraction of this first tier of subcompetencies. Here is a sample of three of the 17 milestones:

  •  assess, diagnose and manage common childhood injuries and refer those needing advanced treatment;
  •  provide primary longitudinal care for well and chronically ill children of all ages, and
  •  screen for normal and abnormal behaviour and development and manage appropriately or refer those needing subspecialty care.

Each of these complex sentences contains a collection of unrelated verbs, nouns and adjectives. If parsed into simpler declarative sentences, these milestones would represent a great many more abstract abilities than are implied by a simple count of the bullet points. Thus, this new tier of subcompetencies does not immediately suggest specific behaviours or settings for assessment and evaluation.

If efforts were to continue in this vein, these subcompetencies would need to be further subdivided and specified, although it is not intuitive to anticipate what direction this might take. Perhaps future discussions among stakeholders will result in classifications that subdivide these into organ systems, and from there into specific diagnoses, or perhaps subpopulations of patients. Another approach might account for normal variation among patients presenting with the same diagnostic entity. Whatever form these negotiated subcompetencies may take, it is likely that they will cut across several superordinate taxonomic categories, resulting in numerous theoretical pathways between abstract competencies and specific clinical behaviours. There will be considerable challenges in translating these socially constructed ideas of competence into reliable and specific assessments. In the end, creating the requisite level of detail for each of these subcompetencies may prove overwhelming and impracticable. As ten Cate et al.25 recently acknowledged, ‘One basis for criticism [of competency-based education] is a tendency to describe general competencies in exhaustive detail, leading to bulky, fragmented documents that lose practical value for education as they become less and less connected with the real world.’

Competencies and the hypostatisation error

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

Competencies represent a negotiation of language based on expert opinion, experience and consensus. As these socially constructed ideas enter daily discourse, they may come to be mistaken for objective phenomena.26,27 The hope of developing ‘valid’ tests of competencies rests on the assumption that such competencies have an independent existence rooted in something other than collective imagination. If the competencies do not in fact exist as naturally occurring regularities in people’s actual behaviours, then there is little hope that they will ever be measured in a reliable way. In the current environment, however, when attempts at measurement of competencies yield ambiguous results, speculation as to why this is so tends to focus on the presumed inadequacy of the measurement tools, or human error, rather than the possibility that the hypothesised competency may not exist at all.

For instance, to many observers in the past several decades, daily experience has insisted that there must be a durable transcendent quality called ‘professionalism’, which underlies a range of clinical behaviours. Repeated attempts to quantify this socially constructed idea of ‘professionalism’, however, have yielded empirical factors that are murky to interpret and do not strongly support any particular model of the construct.28 This problem becomes even more challenging for de novo constructs, such as ‘systems-based practice’, which have even less intuitive definition. We have previously reported that there is no evidence that currently available tools reflect the ACGME’s general competencies.18 To date, however, organisational enthusiasm for assessing ‘professionalism’ or, indeed, for assessing competencies in general has not been appreciably diminished by a general lack of confirmatory empirical data.

A belief in the natural existence of abstract competencies may be supported by the knowledge that other similar-sounding abstract psychological constructs, such as ‘extraversion’, ‘neuroticism’ or ‘formal operations’, can indeed be meaningfully measured. Although competency-based constructs such as ‘professionalism’ and ‘communication skills’ may appear to partake of a similar language, the crucial difference is that established psychological constructs are much more likely to reveal themselves in measurement, by virtue of repeated testing and refining of their underlying theoretical assumptions. The theories behind such constructs have been shown to be coherent from a measurement point of view.

However, because there is currently no similar evidentiary base for socially negotiated educational competencies, it remains possible that they are entirely imaginary and hence unmeasurable. Social consensus provides no guarantee in this regard. It is now widely recognised, for instance, that once largely accepted medical and scientific notions such as ‘the ether’, ‘the humours’ or ‘phlogiston’ were never measurable, for the simple reason that they were never there in the first place. It would be whimsical to speak of ‘valid’ measures for these constructs. Indeed, it was a process of systematic measurement that revealed these ideas to be incoherent in terms of measurement.

Belief in the measurable reality of such imaginary constructs represents the logical error known as ‘hypostatisation’, which is the act of erroneously ascribing material existence to an abstract idea. As yet, the competencies have not been shown to exist from a measurement point of view, and thus may be subject to hypostatisation error. Here is a recent passage from Borsboom et al.,29 who succinctly explicate this problem with validity for psychological tests:

‘The argument to be presented is exceedingly simple: so simple, in fact, that it articulates an account of validity that may seem almost trivial. It is as follows. If something does not exist, then one cannot measure it. If it exists but does not causally produce variations in the outcomes of the measurement procedure, then one is either measuring nothing at all or something different altogether.’29

Current efforts to measure the competencies may in fact be measuring ‘something different altogether’: we now know, for instance, that earlier attempts to measure ‘black bile’ in fact represented efforts to quantify a jumble of unrelated and (at the time) poorly understood physiological processes.

In order to derive measurement constructs that do exist from a measurement point of view, a necessary first step would be to abandon the project to ‘assess the competencies’ in favour of empirical studies to define measurable factors that do underlie human performance in the clinical setting. The rest of this article describes several problems that would need to be anticipated by such a project.

Methodological issues in the assessment of clinical behaviour

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

Appropriate specification of constructs

A useful case study in appropriate specification of educational measurement constructs is provided by Flynn.30 In his analysis of data from nearly a century of empirical intelligence testing, he found that worldwide scores on tests of context-free problem solving have improved consistently over the past century and at a faster rate than scores on tests of underlying knowledge. These findings question the value of intuitive curriculum-based constructs of intelligence, such as ‘verbal’ versus ‘mathematical’ versus ‘spatial’ intelligence. They also cast doubt on a ‘general’ intelligence factor. The data are in fact more consistent with a continuum of tasks that can be organised by the degree to which they reflect the non-intuitive constructs of ‘fluid intelligence’ versus ‘crystallised intelligence’. The resulting model of intelligence leads to a number of interesting and testable hypotheses about the ways that social and historical forces interact with human ability. It is unlikely that any amount of expert-based consensus-building about the meaning of ‘intelligence’ would have led to this data-based model of human intellectual potential. These data also lead to non-intuitive research questions that would not necessarily have suggested themselves from simple introspection about why some people appear smarter than others.

Beginning with the current ideas of competencies, empirical study could help to focus the definition of naturally occurring domains of performance in the medical setting. Any data-based model of assessment must also account for assessor's mental models of trainees' behavior, as well as any systematic biases in judgment. Thus, in addition to standard psychometric testing, a number of exploratory methodologies could be employed to explore the structure of expert judgement. There is a role, for instance, for methodologically rigorous qualitative studies to explore the dimensions of competence that faculty members do discriminate in the context of specific clinical settings.31 For example, it has been reported that faculty observers can be more reliable judges of performance if the categories of measurement are more congruent with their underlying model of clinical performance than with that of consensus-based competencies.32,33

There may also be a role for less conventional exploratory techniques, such as multi-dimensional scaling, which converts ratings of similarities between objects (e.g. skills, situations, behaviours, etc.) into multi-dimensional maps that place like objects together and less similar objects farther apart.34–36 This not only yields clusters of similar objects, but also interpretable performance dimensions.

Situational factors in human performance

Trainees’ performance on tests of general constructs is known to be highly case-dependent.1,37 For novices, performance in any particular clinical scenario may largely reflect construct-irrelevant factors, such as the amount of prior experience that a trainee happens to have had with a particular disease entity. Because any single clinical assessment may reveal little about a trainee’s underlying global abilities, trainees must be observed over a large number of situations in order to distinguish construct-related ‘signal’ from construct-irrelevant ‘noise’. For this reason, the measurement of global competencies in trainees is highly resource-intensive.

In general, there has been little empirical work to define the structural dimensions along which clinical situations differ from one another. Currently, the selection of a set of clinical scenarios for assessment is generally based on the personal judgement of the assessor or sometimes on ‘blueprinting’ exercises,38 rather than on a data-based understanding of the situational factors that affect clinical performance. Without such an understanding, it is difficult to know both whether multiple different assessments are truly comparable and the degree to which performance represents a stable characteristic of the examinee rather than unique elements in a particular testing situation.

Although case-dependency may become less of a concern at higher levels of training, even experts are known to disagree with one another about the ‘best’ responses in ambiguous clinical scenarios,39 which are presumably those that most readily differentiate experts from novices. Thus, another important project would be to classify the situational elements that contribute to clinical ambiguity.

In response to these problems, some commentators have suggested that ‘competence’ should be differentiated from ‘competency’ so that the former construct is used to denote a stable personality trait and the latter to describe behavioural expectations in a specific context.23 In general, the language of competencies, milestones and EPAs represents something of a hybrid, and generally describes competency as a general ability constrained within a set of clinical circumstances (although still at a very high level of abstraction, such as in ‘common childhood injuries’ or ‘well and chronically ill children of all ages’). The CanMEDS framework approaches this problem by mapping competencies onto roles.14

From a measurement point of view, though, it is far simpler to measure situations and people in isolation, rather than to untangle their interactions. In order to avoid this temptation, it is worth recalling the 20-year debate among research psychologists, from the 1960s through the 1980s, about the role of traits versus situations in determining people’s behaviours.40 On one side were personality theorists, who could demonstrate that people display predictable behaviours and interactional styles across situations. On the other were behaviourists, who could condition behaviour that was situational, contingent and malleable. The data that emerged from this controversy suggest that the initial terms of the debate were poorly specified and that neither extreme represented the nuances of actual human behaviour. The ‘debate’ is now only of historical interest – the partisans have abandoned their respective ramparts as neither territory turned out to be worth defending. It will likely prove similarly unfruitful to define competency purely in terms of global personal characteristics on the one hand, or as a checklist of situation-specific behaviours on the other.

Descriptions of ‘competencies’ should thus specify the range of examinees and situations to which they apply. If they are specified in overly dispositional terms, or over too wide a range of unrelated situations, they will contribute to only a small fraction of the variance of any particular performance.41 Global constructs such as ‘professionalism’, ‘communications skills’ and ‘cultural competency’ appear to be cast at this impracticable level of generality. Thus, rather than speaking broadly of general competencies and milestones, it will be more practical to explore the dimensions of variability across a carefully specified set of tasks and to study how people with different levels of ability interact with different attributes of tasks.42

There is a place for systematic methodologies for mapping behaviours onto situations, such as the professional performance situation model (PPSM).43 Perhaps because it requires sustained time and structure, the method has never achieved widespread use. As McGaghie44 observed in 1980:

‘Use of the PPSM is obviously time- and labour-intensive. However, the fact that the method encourages evaluators to systematically identify the dimensions and inter-relationships of relevant professional content makes the PPSM a unique contribution to the field of competence assessment. Its logic is compelling, and the use of rule-governed procedures gives added reassurance that, at the very least, orderly domain descriptions will result.’

Rational decisions about assessment resources

Any assessment based upon clinical performance will need to have sufficient reliability to ensure that the scores are generalisable to the universe of competency-based situations to which the assessment is meant to apply. Assessments of general clinical skills require large amounts of testing time and resources in order to achieve adequate reliability.45 Even under tightly controlled circumstances, a single observation by a single observer lacks acceptable reliability, particularly for high-stakes decisions such as whether to allow a trainee to take on more responsibilities. Thus, any specific clinical performance will need to be measured over several occasions or by several independent raters. Given the constraints imposed by finite time and resources, it will not be possible to reliably measure any more than a minute fraction of all the behaviours and scenarios that might result from a socially constructed competency taxonomy. Indeed, the various competency-based projects have not begun to develop a method for prioritising the many different assessments that might be performed to evaluate trainees’ competence.

The relative sizes of task-irrelevant influences on examinees’ scores can be quantified by generalisability theory, which is becoming increasingly common in studies of educational testing.46 In fact, several recent studies have found that such task-irrelevant effects are responsible for a considerable amount of the variability in trainees’ scores.47–49 Assuming that constructs have been adequately specified, generalisability studies and decision analyses could be considered as best practice for assessing new evaluation methodologies as these would allow for direct and meaningful comparisons among the costs and benefits of various assessment strategies. The use of a common metric would allow such data to be combined across studies into cost–outcome analyses. By including the anticipated values of measurement of various competency-based constructs, such analysis could allow for the modelling of the most cost-effective testing strategies. Although such analyses have become very widespread in health services research, studies of the resource trade-offs in competency testing are still uncommon.50 The development of cost–benefit analyses methodology would be an entirely new area for theory and research in educational assessment.

The role of organisational leadership

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

There are socially compelling reasons for educating clinicians in specific skills and for assessing the ability of the trainees and clinicians to whom patients entrust their health and welfare. Organisational leaders are charged with responding to these societal and historical demands. Clear organisational priorities are necessary to coordinate the educational activities of the large numbers of people involved in these tasks. By contrast, actual measurable behaviour of trainees and practitioners is subject to a range of influences that cannot easily be anticipated based on opinion or experience. Over-rehearsed patterns of individual behaviour are probably affected by local changes in the individual’s social environment more than by larger organisational and societal dynamics.

The work of discovering and describing such influences and patterns could be carried out in tandem with political efforts to move medical education to a more competency-based approach. These parallel political and scientific efforts, however, represent different spheres of intellectual activity and it is crucial that they are not confused. Organisational priorities and policies, to be effective, must be negotiated by a social process. Models of behaviour, to be accurate, must be studied, refuted and refined by a scientific process of observation and measurement. The two activities can inform one another as long as it is acknowledged that the politics of competence respond to a different set of priorities and rules than does the science of assessment. A clear understanding of the roles of politics and science in medical education is essential if appropriate boundaries between these two complementary activities are to be maintained.

Conclusions

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References

For nearly the first 100 years of modern medical education, the formal assessment of medical trainees was limited to tests of knowledge. Although educators of that era were undoubtedly sensitive to variations in the behaviour of trainees in the clinical setting, they did not then have access to an official vocabulary with which to describe these aspects of performance. Such language has been invented in the last few decades. This language was negotiated largely in response to long-term societal trends and expectations, rather than to accommodate specific data about patterns of human behaviour. There is no guarantee that the resulting competencies will be measurable. Further efforts to refine these socially constructed competencies will not necessarily render them any more measurable. Another approach would involve abandoning the unwarranted certainty that the competencies can be measured in favour of an evidence-based project of assessing patterns of human performance in the clinical setting. Such efforts would set the stage for a credible science of tracking the effects of curricular reform over time.

Acknowledgements:  none.

Funding:  none.

Conflicts of interest:  none.

Ethical approval:  not applicable.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Medical education in the early 20th century
  5. The modern era of competencies
  6. Taxonomies of competence
  7. Competencies and the hypostatisation error
  8. Methodological issues in the assessment of clinical behaviour
  9. The role of organisational leadership
  10. Conclusions
  11. References