SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  8. Appendix

Medical Education 2012: 46:604–612

Objectives  Feedback in workplace-based clinical settings often relies on expert trainers’ judgements of directly observed trainee performance. There is ample literature on effective feedback, but in practice trainees in workplace-based training are not regularly observed. We aimed to examine external conditions that impact feedback in observational workplace-based assessment (WBA).

Methods  Interviews were conducted and the resulting data analysed using a qualitative, phenomenological approach. Between October 2009 and January 2010, we interviewed 22 postgraduate general practice trainees at two institutions in the Netherlands. Three researchers analysed the transcripts of the interviews.

Results  A three-step scheme emerged from the data. Feedback as part of WBA is of greater benefit to trainees if: (i) observation and feedback are planned by the trainee and trainer; (ii) the content and delivery of the feedback are adequate, and (iii) the trainee uses the feedback to guide his or her learning by linking it to learning goals. Negative emotions reported by almost all trainees in relation to observation and feedback led to different responses. Some trainees avoided observation, whereas others overcame their apprehension and actively sought observation and feedback. Active trainers were able to help trainees overcome their fears. Four types of trainer–trainee pairs were distinguished according to their engagement in observation and feedback. External requirements set by training institutions may stimulate inactive trainers and trainees.

Conclusions  In line with the literature, our results emphasise the importance of the content of feedback and the way it is provided, as well as the importance of its incorporation in trainees’ learning. Moreover, we highlight the step before the actual feedback itself. The way arrangements for feedback are made appears to be important to feedback in formative WBA. Finally, we outline several factors that influence the success or failure of feedback but precede the process of observation and feedback.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  8. Appendix

Feedback on trainees’ performance in workplace-based clinical settings generally reflects the judgement of experts who have observed the performance.1 The intended effect of this feedback is to help trainees learn and improve their performance and is based on the assumption that feedback creates awareness of shortcomings and thereby motivates learners to improve or change.2 This study focuses on formal feedback that is based on the observation of a trainee conducting a consultation with a single patient. Although frequent observations also provide information about performance for the overall assessment of trainees, we focus in this study on the formative aspect of feedback, whereby trainees purposefully invite and receive feedback in order to learn and improve from it. Many articles have reported on when and how feedback is likely to be most effective,1–4 but there are strong concerns that trainees only infrequently receive feedback based on direct observation of a patient encounter.5–7 We will first present an overview of the literature that is relevant to the scope of the study and will then examine practice in the clinical setting.

Most of the studies on feedback on performance have focused upon factors that influence the use of feedback and strategies for delivering it.3,4 In a review, Shute showed that the actual use of feedback in different settings depends on motivation (the trainee needs the feedback), opportunity (feedback is given in time to be used by the trainee) and means (the trainee is able and willing to use the feedback).3 Shute also provides comprehensive lists of guidelines to enhance learning.3 Some of the implications of the guidelines are that feedback should be given in response to a problem or task, may prevent or correct errors and misconceptions, should preferably be written and should promote a specific learning goal. It should also be objective, which means that effective feedback will allow for a comparison of performance with an established standard of performance. In a review of the literature on the impact of feedback on learning and achievement, Hattie and Timperley4 reported that effective feedback should focus on the task, process and (self-)regulation. Feedback relating to the personal level is rarely effective because trainees are very strongly focused on avoiding risk and failure. In their model of feedback, Hattie and Timperley also emphasised goal orientation exemplified by three questions that a trainee should be able to answer if feedback is to be effective: Where am I going? How am I going? Where do I go next?4 The model gives a good overview of how feedback might enhance learning. Both reviews also showed that the type and delivery of feedback influence its effectiveness and that it may even have a negative impact.3,4

A review of feedback within the medical domain, by Archer,8 indicated that for feedback to be relevant and effective, it should be specific and not exclusively trainer-driven. It should be part of a two-way process and trainees should be given the opportunity to reflect on their actions. In Archer’s model of effective feedback, feedback is not a series of unrelated events, but a sequential process linked to personal goals that includes self-monitoring (reflection on action) that is supported by external feedback.8 In a study of multi-source feedback for practising doctors, Sargeant et al.9 found that acceptance and usage of feedback by individual doctors depended on the nature of the feedback. Positive feedback appeared to be easily assimilated by recipients, whereas negative feedback was first appraised for its credibility based on its process (feedback based on observed performance), source, content and specificity, as well as on its congruence with feedback from other sources. Watling and Lingard also concluded that participants’ perceptions of an evaluation process profoundly affected the usefulness of the evaluation and the extent to which it achieved its goal.1 Similarly, Eva et al.10 found that the self-perceptions of recipients of feedback, relating to their confidence, experience and fear of not being sufficiently knowledgeable, impacted on their interpretation and uptake of feedback.

According to Norcini and Burch,5 there are indications that trainees are observed only rarely and that faculty staff play a critical role in the successful implementation of formative assessment.5,6 By contrast, in their review, Miller and Archer2 pointed to evidence that formative feedback in the workplace is highly appreciated by users, who (subjectively) reported positive educational impact. How is this positive subjective response of users reconciled with Norcini and Burch’s claim5 that observation and feedback happen only too rarely?

In summary, with respect to formal feedback based on the observation of a trainee during a consultation with a single patient, we know that formative feedback in workplace-based clinical settings relies on the judgement of expert trainers and that the literature has provided models, lists and guidelines indicating when feedback is likely to be effective and how it is best provided. However, there seems to be a discrepancy between the evidence in the literature and what we see in day-to-day practice, in which the application of the evidence with respect to feedback falls short of what we would expect. To clarify this issue, we conducted a qualitative study in which we explored the lived experiences and perceptions of individual trainees by addressing the following research questions:

1 How is feedback, based on observation of a trainee performing a consultation with a single patient, carried out in the workplace-based clinical setting?

2 Which aspects influence the feedback process in the workplace-based clinical setting?

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  8. Appendix

Because we wanted to be receptive to all experiences and perspectives, we conducted semi-structured individual interviews to collect data. We conducted the study in the clinical setting of postgraduate training for general practice.

Context

From the eight departments of general practice in the Netherlands that offer postgraduate training, we selected two for inclusion in the study. General practice trainees learn by working in one general practice for a prolonged period under the supervision of a general practitioner (GP) who works in the practice. Trainees work mostly independently, but can ask their trainer for help and advice. There are no external rules to guide observation of trainee performance. Both the trainer and trainee can make arrangements for the observation of a consultation with feedback by planning a consultation at which they are both present or by recording a consultation and planning a meeting for observation and feedback sometime afterwards.

The 3-year postgraduate training programme for general practice consists of 2 years in general practice (years 1 and 3) and 1 year (year 2) of rotations in hospitals and other medical institutions. The eight training institutions deliver a national programme in ways that are broadly similar but leave room for local interpretation. National summative examinations include knowledge tests and work-based communication video-based assessment. Formative assessments are organised locally. We conducted the study in two departments of general practice which differ in their organisation of and recommendations to trainees in relation to (formative) feedback based on observation in daily practice. We selected two departments to substantiate our findings. The Nijmegen programme recommends the direct observation of performance combined with a mini-clinical exercise (mini-CEX) instrument that covers the competencies of medical expertise, communication and professionalism. The instrument allows for written narrative reflections and feedback based on the Pendleton rules,11 with reflection on ‘what went well’, feedback on ‘what went well’, reflection on ‘what could have been done better’, feedback on ‘what could have been done better’ and the planning of further activities. Trainers and trainees are advised to conduct one feedback session using the mini-CEX instrument per week and to conduct a minimum of three sessions every 3 months. The results can be used in trainees’ 3-monthly progress interviews and thus the feedback is not strictly formative because when it is used frequently, it gives the trainer input for summative purposes. Compliance with these recommendations is not monitored. In Maastricht, videotaped observation combined with a feedback discussion is recommended. The daily learning meetings of trainer and trainee can be used to watch and discuss a videotape of a patient encounter. This discussion does not have to be supported by a tool, but the format of the national communication video-based assessment can be used if desired. No minimum has been set for the number of observations. The recommendations of the institutions are directive, but trainers and trainees are free to organise observations differently. Although the institutions’ recommendations for observation differ, the feedback sessions are of comparable intensity and both generate substantial narrative feedback. Both departments see the primary aim of feedback as supporting the learning of trainees, although it can also be used as input for portfolios and to inform trainers about a trainee’s competence for the purposes of progress decisions.

Participants

Because we wanted to explore trainees’ lived experiences with regard to observation and feedback on performance in general practice, we interviewed 11 first-year and 11 third-year general practice trainees in the period between October 2009 and January 2010. E-mails were sent to general practice trainees in years 1 and 3 at both Nijmegen and Maastricht to invite them to participate. Participation was voluntary. Of the 27 trainees who responded, we selected a purposeful sample of 22 trainees, based on institution, gender and year of education. Because of this sampling and for practical reasons, we scheduled and conducted all 22 interviews. We envisaged that we would conduct more interviews if we had not reached a point of data saturation after 22 interviews. Participants received a gift coupon to the value of €20.00 and were assured that all data would be used anonymously and confidentially. Participants gave informed consent, which was confirmed on the audiotape during the interview. The study was approved by the ethics review board of the Dutch Association for Medical Education (Nederlandse Vereniging voor Medische Onderwijs [NVMO]).

Design

The interviews lasted 20–30 minutes and were all conducted by the first author (EAMP). Based on the literature and the results of two pilot interviews (conducted in July and September 2009), we designed an interview guide that covered the topics of: practical organisation; frequency; receiving and accepting feedback, and the relationship of observations to other assessment formats included in the portfolio. The interview guide is shown in Appendix 1.

Data analysis

Because of the exploratory nature of the study and our focus on trainees’ experiences, we used a qualitative approach related to phenomenology, which is not an empirical analytical science, but a philosophy looking for the meaning of phenomena.12,13 Our study draws on the assumptions of what has come to be known as ‘new’ or ‘American’ phenomenology, which can also include interpretation of data. The aim is to describe participants’‘lived experiences within the context’ in order to find a general meaning.13

We audio-recorded and literally transcribed the interviews in a manner that safeguarded the anonymity of participants and their trainers. Firstly, all interviews were read by EAMP to gain an overall impression of the content and possible themes. Then, one interview was read and themed by three researchers (EAMP, HGAM, AWMK). They discussed the emergent themes in order to ascertain that they were drawing the same concepts from the transcripts and to establish how they might formulate codes. An example of a theme is: ‘routine in observation’. Possible codes associated with this theme are: ‘Wednesday morning’ and ‘Thursday afternoon’. Five transcripts were coded by EAMP and HGAM, and five other transcripts were coded by EAMP and AWMK. This resulted in codes and themes for 11 interview transcripts. atlas.ti (Scientific Software Development GmbH, Berlin, Germany) was used to manage the codes and themes. The discussion about the themes and codes from the first 11 interviews and the analysis of the remaining 11 interviews were considered to provide validation of the analysis. The second set of 11 transcripts was coded mainly by EAMP, but, to ensure consistency in the coding, three interviews in this set were coded by EAMP, HGAM and AWMK, EAMP and HGAM, and EAMP and AWMK, respectively. Saturation was reached after 13 interviews had been coded (no new themes emerged), but all 22 interviews were analysed and the remaining nine interviews were used to confirm saturation. Data from all 22 interviews were consistent with the results.

The interviews were conducted and transcribed in Dutch. The quotations in the Results section are literal translations of the original statements.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  8. Appendix

Three steps appeared to be necessary for feedback to have a beneficial effect during single-encounter assessments. The first step concerned arrangements for observation and feedback made by trainer and trainee together. The second step related to the content and delivery of the feedback. The third step concerned the incorporation of the feedback in the learning process and required the trainee to accept the feedback, reflect on it in relation to his or her learning goals, and use it to plan some kind of action to pursue these learning goals.

We combined these three steps into a scheme (Fig. 1). We will discuss the three steps consecutively and then answer the second research question by showing which aspects influenced the extent to which the steps of the scheme were actually used. Finally, we will discuss the effects of the different approaches recommended by the two institutions.

image

Figure 1.  Steps in the process of obtaining useful feedback

Download figure to PowerPoint

Step 1: organisation of observation and feedback

The frequency of observation of consultations varied considerably and ranged from twice weekly to not at all. Remarkably, trainees who reported a high observation rate also reported having made agreements with their trainers on clearly defined training routines at the beginning of the training year:

‘We have an arrangement that we just, say during consultation hours, that she observes me every Thursday: my last two consultations on Thursday afternoons.’ (Trainee 16)

Trainers and trainees who had set clear routines for observation also scheduled time for feedback immediately following direct observation or in the same week in the case of videotape observation. These routines appeared to promote the effective completion of the first step.

Trainees spontaneously mentioned observation during home visits, and weekend duty or night shifts. These occasions created natural moments for observation because trainers and trainees were generally working together. Trainees remarked, however, that the content of these encounters and, consequently, of the feedback differed from that of ‘regular practice’. This meant that these natural moments for observation could not replace observation and feedback on ‘regular consultations’.

Step 2: content and delivery of feedback

The analysis showed that most of the feedback related to communication. Even with the use of the mini-CEX instrument, which specifically invited feedback on three competencies (medical expertise, communication and professionalism), communication was the predominant topic of feedback.

There was considerable variety in the delivery of feedback. At one end of the spectrum were very intensive sessions in which specific feedback was given and trainees reflected on their performance and considered further action to improve it. One trainee described this as:

‘We frequently stop it [the video]. And then we watch… mostly he first asks me what I think of it, and what I think I could have done differently, or should have done differently, or could have done differently […]. And the effect is that we frequently role-play in between.’ (Trainee 4)

At the other end of the spectrum lay superficial, non-specific feedback in which no attention was paid to reflection or further action. One trainee reported:

‘My GP-trainer […] said twice: “Yes […] I have nothing to do right now’; two or three times: “I can sit in with you.” And at the end of the consultation: “Rather fine, I would not have done it differently, seems accurate.”’ (Trainee 14)

Step 3: incorporation of feedback into the learning process

Some trainees proceeded from the first and second steps to the incorporation of the feedback into their learning. These trainees were aware of the relationship of feedback with their learning goals, their portfolio and the results of other assessments:

‘We used it because in my portfolio, due to, yeah to find out how you explore the “request for help”, and we did that with a mini-CEX.’ (Trainee 20)

Trainees used observation and feedback to judge their progress with regard to learning goals formulated earlier, and they also used feedback to formulate new learning goals. When a trainee had completed all three steps, the cycle was able to start again when observation was planned to elicit further feedback on the same learning goals or on newly formulated goals. This process is represented by the large arrow pointing from Step 3 back to Step 1 in Fig. 1.

Factors influencing the process of receiving feedback

Not all trainees used all three steps: some failed to take the first step or to proceed to the second or third steps. To answer our second research question, we investigated the factors influencing this process.

Attitudes towards observation and feedback

The analysis of the interviews showed that trainees’ reactions to observation were primarily emotional. They talked spontaneously about feeling apprehensive about observation, saying that they felt they behaved differently when they were being observed. They also said they did not like being observed. Both videotaped and direct observation were considered to interfere with normal practice:

‘It is intuitive, if someone watches you, I get nervous. It is not that my GP-trainer is doing, or not doing, something, but generally…’ (Trainee 10)

Although almost all trainees mentioned these emotions, two distinct patterns of responding to them emerged. Some trainees wanted to be observed and actively sought feedback, despite their fears. They acknowledged that they could benefit from feedback and that it could help them to improve their performance. Other trainees, however, allowed their fears to prevail and were reluctant to make arrangements for observation and feedback. As observation and feedback were not subject to external control (other than being strongly recommended), these trainees were able to avoid them.

Avoidance behaviour could be counteracted by an active GP-trainer. Based on the data from the interviews, two groups of trainers were identified: one set of trainers took initiatives to ensure that observation and feedback took place, whereas the other set refrained from observation and feedback:

‘My trainer said: “[…] shouldn’t we do something like a mini-CEX?” I said: “Yes.” And then he said: “With my last trainee we used the walk-in surgery...”’ (Trainee 1)

‘I think it [observation] is useful, but also scary, I think that’s the reason I am not inclined to mention it to my trainer. But if you just do it, then it gets easier […] and then it is very helpful.’ (Trainee 1)

‘I think it is just laziness that I don’t do it. And I notice that my GP-trainer does not ask for it either. And of course that is a little childish [of me].’ (Trainee 5)

Most trainers who actively observed their trainees also invited their trainees to observe them in order to provide a learning experience for the trainee. Based on the attitudes of trainees and trainers towards observation, four groups of trainee–trainer pairs were distinguished (Table 1).

Table 1.    Types of trainer–trainee pairs based on responses to negative emotions relating to feedback
 GP-trainer shows active behaviour in relation to giving feedbackGP-trainer does not show active behaviour in relation to giving feedback
  1. GP = general practitioner

Trainee shows active feedback-seeking behaviourFrequent observation and feedbackGP-trainer does not take the initiative in giving feedback When the trainee asks for feedback, it may or may not be provided
Trainee does not show active feedback-seeking behaviourTrainee does not seek observation and feedback, but they take place because the trainer takes actionObservation and feedback take place only if they are externally required (by the department)
Recommendations from the training institution

In addition to trainee and trainer attitudes, the recommendations of the training institution influenced the occurrence of observation and feedback. The institutional recommendations (direct observation using the mini-CEX format at least three times in 3 months [Nijmegen]; videotaped observation with a feedback session [Maastricht]) influenced the feedback process in various ways. Some trainees reported that they were observed because the institution required it, whereas other trainees said they would organise more observations if a minimum number was required and closely monitored by the institution. By contrast, trainer–trainee pairs who showed a positive attitude towards observation and feedback regarded the imposition of strict requirements as excessive regulation:

‘I think it depends on the relationship with the trainer. If things are going well, he [the trainer] knows how you work and how you are doing. But if things aren’t going well, then you can use it and say: Hey, we need to do this [the mini-CEX].’ (Trainee 15)

Differences among trainees

We interviewed trainees from two different institutions, each of which recommended a different approach to formative feedback on observed consultations. Nevertheless, in practice, trainees in both institutions used both direct and videotaped observation. The mini-CEX instrument was used only in Nijmegen, where videotaped observation was also used. In Maastricht, video-based assessment is standard, but some trainees were also observed directly. The main reasons for selecting one of the observation methods referred to the preferences of the trainee or trainer and the method’s practical feasibility in the general practice. We found no striking differences between Nijmegen and Maastricht. Trainees differed in their appreciation of the two methods, but selected the method they preferred. Appreciation depended on the degree of apprehension of the trainee and the personality of the GP-trainer. No differences emerged between trainees of different genders or years of education. Trainees in each category (institution, gender, year of education) showed no differences in whether they completed the three steps or failed to take the first, second or third step.

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  8. Appendix

Although the literature on feedback has mainly focused on the content and delivery of feedback3,4,9 and on the recommendation that feedback should promote a learning goal3,4,8 (the second and third steps of our scheme), our results underscore the importance of the step before the actual provision of feedback in the workplace setting. Deliberate planning of observation and feedback appears to be essential. The feedback literature explains how feedback can be effective, based on the assumption that feedback takes place, but Norcini and Burch5 and the present results show that this assumption may be rather unrealistic.

Although the second step in our scheme has been described earlier, our findings confirm that it is an important aspect of the feedback process and that its occurrence should not be taken for granted; some trainees take the first step of the scheme, but receive no effective feedback at all or not in an appropriate manner. With respect to the third step, Archer8 stated that feedback should not comprise a series of unrelated events, but should be incorporated into the overall learning process by relating it to learning goals and plans for improvement. In line with this, Hattie and Timperley4 highlighted the need to resolve the question of ‘where to next’ in order to make feedback effective. In addition, Shute3 showed that formative feedback should promote an orientation based on learning goals. We think that feedback based on the observation of a consultation in formative workplace-based assessment (WBA) should not be an isolated event, but should represent the starting point of a continuing learning process. It can also be used to reflect on an ongoing learning goal.

Although feedback is a concept with many facets, the three-step scheme we propose can be useful in pinpointing exactly where things go wrong for trainees and thereby may prevent failure. Our second research question indicated our wish to further examine factors influencing the successful completion of the steps.

Apprehension about being observed and receiving feedback proved to have a powerful negative effect on feedback. The role of fear has also been described in the literature.14 We found, however, that apprehension can be overcome when trainees are motivated to actively seek feedback. Teunissen et al.15 found a similar pattern, showing that trainees are not merely passive recipients of feedback, but differ in their ways of seeking or avoiding it. Our results showed that not only trainee motivation, but also the role of the trainer can help to overcome avoidance patterns induced by negative emotions. Future research should further examine this area, particularly in light of the substantial added value of trainer-led initiatives shown in this study. A trainer who actively promotes observation and feedback may be able to counteract trainees’ avoidance behaviour with regard to feedback.

External regulation by the institutions also influenced the feedback process. The literature shows, however, that simply introducing a tool does not suffice to ensure good feedback that ties in with personal goals16 (E.A.M. Pelgrim, A.W.M. Kramer, H.G.A. Mokkink and C.P.M. van der Vleuten, unpublished data, 2011).

Our findings show that it remains important for trainers and trainees to actively pursue observation and feedback. Nevertheless, it may be advisable for institutions to enforce recommendations more strictly, given that our results indicate that it is quite easy for trainees and trainers not to comply with institutional recommendations at present. This suggests that the implementation of requirements for observation and feedback is a prerequisite for the provision of feedback in formative WBA. Possible ways to enhance implementation might include the provision of instructions and training for trainees and trainers, stipulating a mandatory frequency of observation and feedback, and conducting a quality review.

The factors described here underscore the complex relationships between the feedback receiver, the feedback giver, the training institution and probably other environmental influences as well. Our explorative qualitative study has resulted in a framework of important elements that should be taken into account with regard to feedback on observed consultations in daily practice, but we do not claim that it offers comprehensive coverage of all possible influencing factors. More research is required to further investigate variables, the impact of variables and their interactions. In addition, we focused on the formative purpose of observation and feedback, but this is not as absolute as outlined. The process is not strictly formative because when it is used frequently, the outcomes can be used by trainers as input for summative purposes.

The prominence of communication as a topic of feedback may be attributed to the strong emphasis placed on communication skills in general practice training. Another explanation may be that summative performance assessment of communication skills during general practice training in the Netherlands is based on videotaped observations and several trainees use this method to obtain formative feedback.

There are some limitations to our study. The fact that participants were self-selected may have introduced bias. We do not know if participating trainees differed from non-participating trainees in their ways of seeking and organising observation and feedback. However, our results show wide variations in trainee experiences and ideas about observation and feedback. Furthermore, the transferability of the present findings to other medical specialties and work-based settings may be limited because the study was confined to general practice trainees.

An important finding, which can be interpreted as contributing to the theory on feedback, concerns the strong impact of organisation and arrangements made for observations and feedback. Our results show that this crucial first step in the feedback process in formative WBA is influenced by the individual characteristics of both trainer and trainee, as well as by external regulations imposed by the training institution. We therefore recommend that institutions set a mandatory minimum number of observation and feedback sessions to be completed, supported by a mini-CEX instrument to direct feedback content and help trainees to connect the feedback with their learning goals. Moreover, institutions should train their trainers to make better use of observation as a teaching aid. Training could be structured around the three steps identified in this study. We also recommend further research into trainer behaviour in the provision of feedback in order to investigate how trainers perceive their role in the feedback process and how trainees and trainers influence one another’s feedback-giving and feedback-seeking behaviours.

Contributors:  EAMP is the principle author of the article and contributed to the study design, interviewing, analysis of interviews, interpretation of interview data, and the writing and revision of the article. AWMK contributed to the study design, the first pilot interview and the revision of interview questions; AWMK also observed one interview and contributed to the analysis of interviews, the interpretation of interview data and to the revision of the article. HGAM contributed to the study design, the formulation of interview questions, analysis of the interviews, interpretation of data and the revision of the article. CPMvdV contributed to the study design, the interpretation of interview data, and the revision of the article. All authors approved the final manuscript for submission.

Acknowledgements:  the authors thank Elke Butterbrod, Greetje Sagasser, Department of Primary Care and Community Care, Radboud University Nijmegen Medical Centre and Mereke Gorsira, Department of Educational Development and Research, Faculty of Health, Medicine, and Life Sciences, Maastricht University, for their help in preparing this manuscript, and all participating trainees for the time they invested in the study and the stories they related.

Funding:  SBOH Stichting Beroepsopleiding Huisarts (The Foundation of Postgraduate Training in General Practice, The Netherlands).

Conflicts of interest:  none.

Ethical approval:  this project was approved by the ethics review board of the Dutch Society for Medical Education (Nederlandse Vereniging voor Medische Onderwijs [NVMO]).

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  8. Appendix

Appendix

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. References
  8. Appendix

Appendix 1

Items included in the semi-structured interviews.

  • 1
     How many times does your trainer observe you (with or without a mini-CEX instrument)?
    • Is this direct observation or video-based observation?

    • In which situations does your trainer observe you?

    • Who takes the initiative?

    • Further exploration of the observation context for this specific trainee.

  • And if not, why not? Exploration of the reasons for the absence of observation.

  • 2
     Do you receive feedback after these observations?
    • When (do you receive feedback)?

    • What is said?

    • How does your trainer give feedback?

    • Is there an opportunity for you to reflect on the feedback?

    • What do you do with the feedback afterwards?

    • Give a recent example of the feedback process in your training practice.

    • Further exploration of the feedback context for this specific trainee.

    • And if not, why not? Exploration of the reasons for the absence of feedback.