Gaps in Procedural Experience and Competency in Medical School Graduates
Presented at the Association of American Medical Colleges Annual Meeting, Washington, DC, November, 2007.
CoI: The author reports no conflict of financial interest.
Address for correspondence and reprints: Susan B. Promes, MD, e-mail: email@example.com.
Objectives: The goal of undergraduate medical education is to prepare medical students for residency training. Active learning approaches remain important elements of the curriculum. Active learning of technical procedures in medical schools is particularly important, because residency training time is increasingly at a premium because of changes in the Accreditation Council for Graduate Medical Education duty hour rules. Better preparation in medical school could result in higher levels of confidence in conducting procedures earlier in graduate medical education training. The hypothesis of this study was that more procedural training opportunities in medical school are associated with higher first-year resident self-reported competency with common medical procedures at the beginning of residency training.
Methods: A survey was developed to assess self-reported experience and competency with common medical procedures. The survey was administered to incoming first-year residents at three U.S. training sites. Data regarding experience, competency, and methods of medical school procedure training were collected. Overall satisfaction and confidence with procedural education were also assessed.
Results: There were 256 respondents to the procedures survey. Forty-four percent self-reported that they were marginally or not adequately prepared to perform common procedures. Incoming first-year residents reported the most procedural experience with suturing, Foley catheter placement, venipuncture, and vaginal delivery. The least experience was reported with thoracentesis, central venous access, and splinting. Most first-year residents had not provided basic life support, and more than one-third had not performed cardiopulmonary resuscitation (CPR). Participation in a targeted procedures course during medical school and increasing the number of procedures performed as a medical student were significantly associated with self-assessed competency at the beginning of residency training.
Conclusions: Recent medical school graduates report lack of self-confidence in their ability to perform common procedures upon entering residency training. Implementation of a medical school procedure course to increase exposure to procedures may address this challenge.
In 1996, the American Association of Medical Colleges (AAMC) adopted the Medical Schools Objective Project (MSOP), an initiative whose goal was “to develop a consensus within the medical education community on the attributes that medical students should possess at the time of graduation.”1 Included in the MSOP was the recommendation that students have the ability to perform routine technical procedures, including venipuncture, inserting an intravenous catheter, arterial puncture, thoracentesis, lumbar puncture, inserting a nasogastric tube, inserting a Foley catheter, and suturing lacerations.1 Although some data about student skill level with these procedures have been collected, a more-comprehensive set of information across multiple medical schools is needed.2–5
Coupled with the MSOP recommendations has been a movement toward competency-based education in graduate6 and undergraduate medical education. Additionally, changes in graduate medical education in the decade after the MSOP, including introduction of the six core competencies and duty-hour regulations, have affected the time that residents have to teach students about clinical care, including the performance of routine clinical procedures. These changes have placed residency training time at a premium. Greater educational efficiency could be achieved for first-year residents and their preceptors if the first-year residents received better preparation in medical school for performing procedures. It is hypothesized here that greater experience with procedures in medical school would be associated with higher levels of confidence in conducting procedures earlier in graduate medical education training.
Study Design and Population
Physicians entering their first postgraduate year of training at three teaching hospitals in the southeastern region of the United States were asked to complete an anonymous survey during orientation about their procedural skills experience and perceived competency for a number of common procedures. The Institutional Review Board at each of the institutions approved this study.
Survey Content and Administration
First-year residents were asked to identify how many attempts they had made at performing various common medical procedures and select their own personal level of competency for each of the procedures. Competency was based on the Dreyfus module of knowledge development that articulates five progressive stages of learning, beginning with novice and moving to advanced beginner, competent, proficient, and expert.7 This model has been applied to medicine, with the novice correlating with a freshman medical student, competent with a resident physician, and expert with a mid-career specialist.6 In the survey sent to first-year residents, options and definitions for level of competency included novice, advanced novice, and competent. Competent was defined as having the basic knowledge, technical skill, and understanding of anatomic, physiologic, and procedural issues to perform the task independently. Advanced novice was defined as a person with limited exposure to the skill set required to successfully complete the given task. An advanced novice would feel uncomfortable performing the skill without supervision. Novice was defined as little or no exposure to the skill set, requiring extensive instruction and supervision to perform the given task adequately. Thus, a novice individual could not safely perform the given skill without supervision. In addition to the experience and competency questions, first-year residents were asked whether they had participated in a procedures course as part of medical school training or whether they did an emergency medicine (EM) or intensive care unit (ICU) clinical rotation (because both types of rotations tend to be procedure-heavy). Study participants were also asked to rate their satisfaction with their procedural skills education and experience in medical school, in addition to rating globally how adequately prepared they felt to perform basic procedures as they began their postgraduate training. The procedural skills assessed in this study are listed in Table 1.
Procedural Skills Assessed
|Intravenous catheter insertion*|
|Basic life support – airway foreign body management|
|Basic life support – cardiopulmonary resuscitation|
|Basic life support – bag-valve-mask ventilation|
|Central venous catheter insertion|
|Foley catheter insertion*|
|Nasogastric tube insertion*|
Basic characteristics of survey respondents were compiled, and the number of procedures performed was organized according to categories (0, 1–5, >5). Hicks has defined more than five attempts as necessary for procedural comfort.8 Logistic regression analysis was used to determine overall procedure competency based on medical school coursework. The independent variable was defined as whether an individual had completed procedures course, an EM rotation, or an ICU rotation as part of their medical school curriculum. The dependent variable was defined as whether the survey respondents felt themselves to be competent (yes or no) to perform basic medical procedures. Chi-square analyses were used to determine the effect of procedure volume and self-assessed competency for each of the medical procedures, as well as the relationship between coursework (EM rotation, ICU rotation, procedures course) and the total number of procedures performed. Significance was set at p < 0.05. All analyses were conducted using SAS statistical software, Version 9.1 (SAS Institute, Inc., Cary NC).
There were 256 respondents to the procedures survey at the three institutions, representing a 77% response rate. The basic characteristics of survey respondents are listed in Table 2. The vast majority of respondents were recent graduates who were beginning categorical residency programs in a variety of medical specialties.
Description of Survey Respondents (N = 256)
|U.S. medical school||248 (97.3)|
|Year of medical school graduation|
| 2006||240 (93.8)|
| 2005||6 (2.3)|
| Before 2005||8 (3.9)|
| Emergency medicine||23 (9.0)|
| Internal medicine||89 (34.8)|
| Surgery and surgical subspecialties||47 (18.4)|
| Pediatrics||34 (13.3)|
| Others||57 (22.3)|
|Resident status (categorical vs. transitional or preliminary)||234 (93.6)|
The procedural experience of the survey respondents is highlighted in Table 3. The incoming first-year residents reported the most procedural experience with suturing, Foley catheter placement, venipuncture, and vaginal delivery. The majority of first-year residents reported that they had performed all of the other procedures five or fewer times. The least experience was reported with thoracentesis, central venous access, and splinting. Most had not performed basic life support, and more than one-third had not performed cardiopulmonary resuscitation (CPR). For each of the medical procedures queried in the survey, a greater number of procedures performed was associated with higher level of self-assessed competency (p < 0.001 for all procedures).
Number of Procedures Performed
|Central venous access||47||45||8|
|Basic life support||64||28||8|
Active medical school programs directed specifically toward procedures (procedure courses, EM rotations, and ICU rotations) were associated with greater self-assessed global adequacy to perform basic procedures (Table 4). First-year residents who completed a dedicated procedures course in medical school (24% in the current study) were significantly more likely to report competency in performing basic medical procedures. Those who took a procedures course were more likely to report adequacy in performing procedures (odds ratio = 2.5, 95% confidence interval = 1.32 to 4.78). First-year residents in this study who completed an EM (68%) or ICU (63%) rotation in medical school tended to report procedural competency.
Overall Competency by Medical School Rotation, Medical School Procedures Course and Self-assessed Adequacy to Perform Basic Procedures (Not or Marginally Adequate vs. Some or to a Great Extent)
Additionally, first-year residents who took a medical school procedures course were significantly more likely to have performed larger numbers of central lines and venipuncture attempts (p < 0.05 for both). Those who took an EM rotation in medical school were significantly more likely to have performed larger numbers of splinting attempts (p < 0.05), whereas those who had taken an ICU rotation were significantly more likely to have performed larger numbers of arterial blood gas draws and central lines (p < 0.05).
This study suggests that first-year residents who took a procedure course during their undergraduate medical education were significantly more likely to feel prepared to perform procedures as incoming residents than those who did not take a procedures course. These findings are consistent with those reported by Coberly and Goldenhar, who found that fewer than 50% of students performed common procedures during the 2-month acting internship (or “subinternship”) rotation in internal medicine.3 Sub-internship rotations are those most likely to provide opportunities for students to perform procedures, given their relatively advanced level of supervised patient care responsibility. Furthermore, the Coberly and Goldenhar findings are of particular interest, because the sub-internship requirement at their institution is twice as long as the requirement at most other medical schools.
Our data reveal consistency in the kinds of procedures performed based on common medical school fourth-year electives. Unique procedural skills were obtained from each of the different experiences in medical school, and there was little overlap between rotations. Although medical students were more likely to have greater levels of experience with specific procedures on the EM and ICU rotations, they did not report significantly higher global levels of self-perceived procedural adequacy than first-year residents. This suggests that a multidisciplinary procedures course in medical school may be necessary to ensure medical student exposure to a broad array of common clinical procedures.
The literature provides additional support for the need for development of such a course. The method by which medical schools teach procedures is varied; the modeling of behavior and teaching by upper level residents is probably the most common method, but given that resident training time is at a premium because of the Accreditation Council for Graduate Medical Education duty hour regulations, this teaching time may be at risk. One study demonstrated that 78% of schools offer no formal skills training other than an introduction to venous phlebotomy.9 The value of a dedicated course to teach routine procedures has been documented; students who took a course in emergency procedures were found to have higher procedural skill scores in clinical rotations.10
A recent survey of medical school deans found that there was strong agreement among deans that graduating medical students should be proficient in performing routine procedures, yet most schools do not rigorously teach or evaluate these procedures.4 This survey also asserted that more than two-thirds of graduating students were thought to be proficient in venipuncture, intravenous placement, arterial puncture, placement of nasogastric tube, placement of Foley catheter, suturing lacerations, and CPR.4 However, a survey of medical students conducted in 2006 demonstrated that, at the end of their third year, exposure to routine (33% had not performed arterial puncture, 24% had not performed phlebotomy, 37% had not performed nasogastric tube insertion, 72% had not performed CPR) and more-complex (88% had not performed thoracentesis, 65% had not performed lumbar puncture) procedures was limited.2
Our study indicates that self-assessed competency is significantly correlated with experience, yet there was great variability in the number of procedures performed depending on fourth-year medical school experiences. A focused curriculum to prepare students and ensure adequate numbers of procedure attempts would be one possible solution for preparing students to function confidently from the beginning of their residency training. Students in the study who completed a procedures course tended to perform greater numbers of procedures.
This study suggests an imperative to incorporate a standardized procedures-training curriculum into undergraduate medical education. Next steps include the development of such a curriculum with clear articulation of what should be included in such a course. Operational definitions of competence should be developed for common procedures, and a study of the minimum number of attempts required to approach competence must be conducted. Most importantly, standardized strategies for assessing student competence should be developed and consistently implemented. In addition, residency programs may want to consider assessing the ability of all incoming residents to perform common basic procedures and instituting educational programs to teach basic procedural skills during orientation that are important to their functioning as a house officer.
A recent editorial reinforces the need for the American Board of Internal Medicine’s requirement that interns demonstrate cognitive and manual competency in performing basic procedural skills.11 The authors assert that “the profession now needs to redesign procedural training across the continuum of medical education and a lifetime of practice.” Such training should begin in undergraduate medical education with the expectation that full competence is achieved in graduate medical education and then maintained through continuing medical education.
This was a survey-based study, and for that reason, it has inherent limitations, including that it was based on self-reported data instead of objective data. Incoming first-year residents were asked to recall the number of times they had performed various procedures during medical school. It is possible that their responses were not completely accurate. They were also asked to self-assess their competency level using the Dreyfus model terminology. Although definitions for each of the terms (novice, advanced novice, competent) were supplied to entering first-year residents to assist them in categorizing their competency level, a lack of familiarity with the terms could have had an effect on their responses. Finally, there was no objective confirmation of self-reported competency with an actual skills assessment.
Recent medical school graduates report poor self-confidence in their ability to perform common procedures upon entering residency training. Implementation of a medical school procedure course to systematically increase exposure to procedures may address this challenge. Additionally, medical schools and residency programs might consider formally assessing procedural competency to ensure that incoming residents have mastered basic procedural skills and are prepared for residency training.