Evaluating Medical Students' Skills in Obtaining Informed Consent for HIV Testing
Address correspondence and requests for reprints to Dr. Roberts: Department of Psychiatry, University of New Mexico Health Sciences Center, 2400 Tucker NE, Albuquerque, NM 87131-5326 (e-mail: email@example.com).
OBJECTIVE: To evaluate fourth-year medical students' abilities to obtain informed consent or refusal for HIV testing through a performance-based evaluation method.
DESIGN: Student competence was assessed in a standardized patient interaction in which the student obtained informed consent or refusal for HIV testing. A previously validated 16-item checklist was completed by the standardized patient. A subset was independently reviewed and scored by a faculty member to calculate interrater reliability for this report. Student feedback on the assessment was elicited.
SETTING: School of Medicine at the University of New Mexico.
PATIENTS/PARTICIPANTS: All senior medical students in the class of 2000 were included.
INTERVENTIONS: A 10-minute standardized patient interaction was administered within the context of a formal comprehensive performance assessment.
MEASUREMENTS and MAIN RESULTS: Seventy-nine students participated, and most (96%) demonstrated competence on the station. For the 15 specific items, the mean score was 25.5 out of 30 possible points (range, 13 to 30; SD, 3.5) on the checklist. A strong positive correlation (rs = .79) was found between the total score on the 15 Likert-scaled items and the score in response to the global item, “I would return to this clinician” (mean, 3.5; SD, 1.0). Scores given by the standardized patients and the faculty rater were well correlated. The station was generally well received by students, many of whom were stimulated to pursue further learning.
CONCLUSIONS: This method of assessing medical students' abilities to obtain informed consent or refusal for HIV testing can be translated to a variety of clinical settings. Such efforts may help in demonstrating competence in performing key ethics skills and may help ensure ethically sound clinical care for people at risk for HIV infection.
Professional values find expression in the everyday practices of clinicians. Certain practices, such as obtaining informed consent, are especially important to the provision of astute, ethically sound modern patient care.1 These activities are essential to clinical competence—skills that must be learned, and should be assessed, in the process of medical education.2–8 Obtaining informed consent for HIV testing, in particular, is a routine professional practice relevant to nearly all medical specialties and disciplines. At the end of 2000, it was estimated that worldwide 36.1 million people were living with HIV/AIDS and in 1996, 24.6 million HIV tests were performed during a 12-month period in the United States.9,10 It is increasingly recognized, however, that individuals who seek HIV testing are subject to stigma and potential discrimination.11,12 For these reasons, consent for HIV testing must be sought with sensitivity, respect, and attentiveness to the complex psychosocial, cultural, and legal issues faced by patients.
In this report, we describe an educational assessment practice at our institution in which students' abilities to obtain informed consent or refusal for an HIV test are evaluated. This assessment occurs in the context of a standardized patient exercise during a comprehensive performance-based examination required of all fourth-year medical students.13 In its content and methodology, this assessment represents an innovative effort to ensure the ethical skill of physicians-in-training. For this reason, it has value as a clinical and ethical competence measure, with initial empirical validation data. In addition, this approach may be translated to other settings in which people at risk for HIV infection receive care, helping to verify that optimal ethical as well as clinical practices are implemented in the treatment of patients at risk for this stigmatizing disorder.
Ethical Safeguards for Students
Recent work has called attention to the need to address ethical considerations related to medical student involvement in educational scholarship and research.14 At the University of New Mexico School of Medicine, students are informed that confidentially encoded data on their school performance may, from time to time, be used in educational research reports. Students are invited to consent to this use of data, or to decline to participate, without consequence to their academic status. Rigorous confidentiality safeguards are in place to protect students' privacy. In addition, this specific project was reviewed and approved by the Human Research Review Committee at our institution.
Medical students at our institution are formally assessed with respect to professional attitudes, values, and ethics in the context of 3 comprehensive performance examinations (Student Progress Assessment or SPA exams) occurring over the 4 years of medical school.13 Each SPA Exam is a 2- to 4-day pass/fail test encompassing 5 competence areas: 1) clinical skills; 2) communication skills; 3) critical reasoning and integration of knowledge; 4) professional attitudes, values, and ethics; and 5) self-assessment.
The aim of the first SPA examination, occurring during the first year of medical school, is strictly formative—students receive written and verbal feedback from faculty and standardized patients on their performance in each competence area. Considerable attention is given to the ways in which each student may improve his or her knowledge base and skills. Most students perform very well, but some require intensive study, support, and supervision before they are sufficiently prepared to participate in subsequent SPA examinations. The second and third SPA examinations occurring during the second and fourth years of medical school involve “higher stakes,” i.e., students may not move forward in the curriculum if they have not successfully passed every competence area tested in the SPA exam.
Specific content of the SPA examinations is focused on case material and skills in routine clinical care, e.g., evaluating pain, “dizzy spells,” or insomnia as a chief complaint, performing a full physical examination, gathering the patient's history, writing clinical notes, and formulating hypotheses and treatment approaches. The clinical, ethical, and psychosocial issues involved in seeking consent for an HIV test—the evaluation domain described in this report—are covered directly and indirectly in multiple settings, for example, during the “Perspectives in Medicine” small group discussion course that occurs longitudinally during years 1 through 3, during the Infectious Disease Block early in the second year (“Phase I”), during the orientation to the hospital in the middle of the second year, and during the Internal Medicine and Family and Community Medicine clerkships (“Phase II”) during the third year of medical school.
The SPA examinations utilize a number of testing methods, such as standardized patient interactions with linked written exercises, analyses of ethical issues and communication elements in trigger videotapes, responses to essay and modified essay questions, self-assessment exercises, “take home” assignments that entail information gathering and critical reasoning skills, interpretation of laboratory test results and x-rays, and standardized multiple choice tests.13 All standardized patient interactions are videotaped, and some stations focusing primarily on communication skill evaluation have faculty observers as well.
The HIV Consent Station
The HIV consent station was 1 of 4 standardized patient interactions in the SPA-3 examination for fourth-year students (class of 2000). It involved a 10-minute follow-up interaction with a standardized patient, Ms. Karen Lasco, a young woman who had “presented” with lower abdominal pain. The HIV consent station was linked to a station with the same patient testing both clinical skills and communication skills. The standardized patient case was based on a real patient's story and was written in 1998 by Dr. Dorothy Barbo of the Department of Obstetrics and Gynecology at our institution. The case and student performance criteria were then reviewed and revised by the multidisciplinary faculty and staff on the SPA team.
Instructions to the Students. After completing this 25-minute station focusing on evaluation of Ms. Lasco's medical problem, students were instructed to return to the patient's room for the follow-up 10-minute station. The instructions given to the students for the 2 standardized patient interactions are detailed in Appendix A. This interaction was focused on obtaining consent and on addressing the confidentiality issues raised by the standardized patient in connection with a recommendation that the patient undergo an HIV test. Students were instructed to obtain consent or refusal for HIV testing whether or not they considered the possibility of HIV infection or took a sexual history in the initial interview. They were not penalized in this follow-up station for leaving out these 2 tasks in the first station.
Evaluative Measures in the Testing Situation. Student performance on the HIV consent station was assessed through a 16-item checklist completed by the standardized patient at the end of the interview. The items were generated from the literature on informed consent and confidentiality in the context of HIV, and had been refined on the basis of the authors' experience with a previous informed consent station on obtaining consent for cardiac catheterization (see Table 1).15 Items related to respectfulness, relatedness, and quality of communication during the encounter (Items 1–3, 9–11), the reasons for and procedure of the HIV test (Items 4–5), the risks and benefits of and alternatives to the test (Items 6–8), the confidentiality issues (including documentation and access) related to the HIV test (Items 12–14), and the absence of coercion in consenting to or refusing the test (Item 15) were included in the checklist. The first 15 of the 16 items were encoded as “yes,”“sort of,” and “no” (2 points, 1 point, and 0 points, respectively) for the statistical analyses. In addition, 1 global assessment item was included: “I would return to this clinician” (Item 16). The global item was assessed on a 5-point Likert scale (1 = strongly disagree; 5 = strongly agree).
Table 1. Items Assessed by Standardized Patients in the HIV Informed Consent Station
| “The student …”|
| 1. Spoke with me in a respectful manner.||1.88||0.33|
| 2. Attempted to establish rapport with me.||1.72||0.52|
| 3. Used language I could easily understand.||1.91||0.33|
| 4. Explained the reason(s) for the HIV test.||1.95||0.22|
| 5. Explained the procedure of the HIV test to me.||1.48||0.83|
| 6. Explained or explored the psychosocial risks of the HIV test to me.||1.02||0.77|
| 7. Explained the benefits of the HIV test to me.||1.80||0.45|
| 8. Explained alternatives to performing the HIV test at this time to me.||1.53||0.68|
| 9. Tried to make sure that I understood the information given to me about the HIV test.||1.93||0.26|
| 10. Listened carefully to any questions I had about the HIV test.||1.93||0.30|
| 11. Seemed to understand how I felt about the issues we discussed.||1.77||0.53|
| 12. Addressed my concerns about confidentiality.||1.54||0.72|
| 13. Explained how the HIV test results would be documented in my records.||1.56||0.65|
| 14. Explained who would have access to my HIV test results.||1.60||0.57|
| 15. Tried to make sure that I did not feel pressured to have the HIV test.||1.86||0.34|
|Global assessment item†|
| “I would return to this clinician.”||3.52||0.96|
Standardized Patient Training. The standardized patients underwent extensive training in preparation for both of the student interactions. Standardized patients were instructed to express apprehension and embarrassment regarding the HIV test. They were also to indicate skepticism about the need for the test based on having few and only heterosexual partners. The standardized patients were to convey significant concern about confidentiality, especially in light of experience as a secretary in an insurance office. They were also to express fear that “someone might find out about the test” and question the patient's ability to be a good parent to her daughter. The standardized patients were instructed, ultimately, to refuse the HIV test that was being recommended by the student in the interaction.
Every item of the checklist was reviewed in detail with the standardized patients. Specific examples of biological risks of the HIV test (e.g., bleeding, bruising, pain with the needle stick), of potential benefits of the test (e.g., the clinical advantages of excluding the possibility of HIV seropositivity), and of psychosocial risks (e.g., personal feelings of shame, documentation of the result of the test in the medical record with possible implications for future insurability, employment, or stigmatization, the fact that false-positive results could also result in these forms of discrimination, etc.) were reviewed carefully with the standardized patients so that they would recognize the elements being assessed. Tremendous attention was given to subtle aspects of the interaction with students, such as identifying and documenting an internal sense of feeling pressured or “gently” coerced by the student into having the HIV test, so that the complex domain of voluntarism could be evaluated. The principal trainer (LWR) and the standardized patients role played the situation repeatedly to enhance consistency and ensure greater reliability in scoring the checklist.
Feedback from Students. Students' views of the SPA-3 examination and its exercises were obtained through written evaluations that included both closed- and open-ended items.
Frequency data based on the standardized patients' scoring of the students were compiled. Correlation coefficients were calculated to compare the performance of students on the HIV consent station with their performance on other elements of the SPA examination. Comparisons were also made on the basis of gender and minority status; for statistical analyses, based on representation among physicians in New Mexico, medical students who described themselves as Native American, African American, and Chicano/Mexican American/Hispanic were defined as underrepresented minorities. Interrater reliability data were obtained by comparing the standardized patient scoring with scoring of a subset of students (n = 29) performed by a senior faculty member with communications expertise (TM) who later observed videotapes of the standardized patient interactions.
Seventy-nine fourth-year students (41 women, 38 men; 54% white, non-Hispanic; 28% Chicano, Mexican-American or Hispanic; 7% Asian or Pacific Islander; 5% Native American; 1% African American; 4% unreported) participated in the required SPA-3 examination.
Student Performance Measures
Data for the standardized patient station are listed in Table 1. Overall, students did well, with 96.2% (76 of 79 students) demonstrating competence according to criteria set in advance of the test. For the 15 specific items, the mean score was 25.5 out of 30 possible points (range, 13 to 30; SD, 3.5) on the checklist. A strong positive correlation (rs = .79) was found between the total score on the 15 Likert-scaled items and the score in response to the global item, “I would return to this clinician” (mean = 3.5; SD, 1.0).
Pearson correlation coefficients were also calculated to compare the performance of students on the 15 specific items for the HIV consent station with their performance on other elements of the SPA-3 examination: (a) the score on a second Professional Attitudes, Values, and Ethics (PAVE) station with written exercises focused on a standardized patient case in which a side issue of suspected elder abuse was introduced (rs = .08); (b) a clinical skills checklist (rs = −.05) and a communication skills checklist (rs = .56) from the initial 25-minute interaction with Ms. Lasco; (c) the total score for all clinical skills checklists (rs = .15) for the 4 standardized patient interactions included in the SPA-3 exam; (d) the average of the communication skills checklists for the 4 standardized patient interactions (rs = .54); and (e) the overall SPA score related to both Clinical Skills and Critical Integration of Knowledge (rs = .17).
Based on the independent ratings performed by the standardized patient and one of the authors (TM) who reviewed the videotapes of 29 student–patient interactions, an intraclass correlation of 0.69 (P < .01) was obtained for the cumulative score of the 15 items. An intraclass correlation of 0.53 (P < .01) was obtained for the global item, “I would return to this clinician.”
Evaluation of the HIV Consent Station and the SPA Examination by Students
On a 5-point scale (1 = not well prepared; 5 = very well prepared), students (n = 73; 92%) were asked how well prepared they felt for the HIV consent station. The students offered the full range of responses with a mean score of 3.2 (SD, 1.3). Students responded similarly to an analogous question about the PAVE written exercise station about suspected elder abuse with a mean score of 3.8 (SD, 0.9).
The HIV consent station generated a high number of spontaneous narrative comments in the anonymous written evaluation of the SPA-3 examination. One student noted, “I was dismayed with myself that I was unaware of how confidentiality of HIV (or ANY) results was maintained … it's still unclear to me and will have to be a ‘learning issue.’” Another student wrote, “The HIV station was hard for me. I don't know current New Mexico law about testing/results/counseling very well. My own experience with a needle stick in San Francisco in 1992 while drawing blood at the VA there…was scary. I got tested with no regard to the insurance/result discrimination issue. This is not a clear-cut issue even today.” A third student wrote, “I was quite surprised that I was unable to get consent from this patient! And I also felt at a loss as to what the real limitations of confidentiality are. I had learned that my job as a physician is to reassure about the existence of confidentiality. But, the reality/truth about the case … . makes me wonder if in providing reassurance to my patients, I would be lying to them!!” Finally, a more skeptical comment was offered by a fourth student who said that, “the cases and actors, especially for the Karen Lasco case, were too contrived. My training doesn't involve these ‘unreal’ situations.”
When assessed about whether the overall SPA-3 examination asked the student to integrate knowledge, skills, and abilities in a manner consistent with other training experiences in medical school (1 = strongly disagree; 5 = strongly agree), students offered a mean response of 3.6 (SD, 1.1). On the same scale, students indicated mild endorsement of the statements: “The level of complexity presented in SPA-3 was appropriate for my current level of medical training” (mean = 3.9; SD, 0.7) and “I would like feedback on my overall SPA-3 performance” (mean = 4.3; SD, 0.9). Also on the same scale, students offered neutral responses to the statements, “The SPA-3 examination helped me feel competent about my learning as a medical student” (mean = 3.0; SD, 1.2); “I felt my SPA-3 performance accurately reflected my abilities” (mean = 2.7; SD, 1.1); and “SPA-3 comes at an appropriate time in my medical training” (mean = 3.0; SD, 1.2).
In addition, when asked whether SPA “redundantly” examined the knowledge and skills tested in other parts of the curriculum, students' mean responses were 3.2 (SD, 1.1) and 3.5 (SD, 1.1). Finally, when asked about the level of stress associated with the SPA-3 examination, students responded that it was less stressful than the students' most challenging Phase I block (mean = 2.5; SD, 1.2), Phase II or clerkship rotation (mean = 2.5; SD, 1.3), and Phase III or sub-internship rotation (mean = 2.5; SD, 1.4).
While it is widely accepted that physicians should exhibit professionalism and skill in performing ethically sensitive tasks, little consensus exists on how to teach and assess these qualities during medical training.5,16–19 In this report, we have described 1 approach that has been developed, tested, and adopted at our medical school for evaluating the skills of senior medical students in obtaining informed consent or refusal from a standardized patient for whom an HIV test has been recommended in a simulated clinical situation. Criteria for evaluating individual students' capacities for performing the consent task encompassed domains such as respectfulness and sensitivity of the physician-in-training, rapport and relatedness in the clinical interaction, discussion of relevant information including risks, benefits, and alternatives, and responsiveness and accurate information around confidentiality concerns. In this discussion, we outline 3 implications of our findings for medical education and describe limitations of this work.
Implications for Medical Education
Three key considerations emerge from the student performance data in this project. First, despite having little formal or systematic preparation for the specific exercise, our medical students performed well on this test. Nearly all (96%) of the students met or surpassed the competence threshold set by faculty before the examination, with an average score of 25.5 out of possible 30. The only problematic item in the checklist pertained to the students' skill in exploring the psychosocial risks associated with the HIV test. The scores given by standardized patients correlated well with those given by a senior faculty member with expertise in communication skills and ethics, suggesting that the strong performance represents a reliable finding. The overall performance of students also positively correlated with the single global item “I would return to this clinician.”
Second, the overall pattern of findings suggests that ethics and professionalism may exist as a distinct domain of clinical competence. Performance on the HIV consent station, for example, correlated moderately with communication skill assessments but did not correlate strongly with scores achieved in other content and skill testing areas on the SPA examination. This is consistent with the observation that the HIV consent station required students to establish a sense of rapport and relatedness with the patient, but also tested their ability to explain medical and psychosocial information about HIV, to minimize the experience of pressure and coercion in the interaction, and to address ethical and psychological concerns about the risks of the diagnostic test. Prislin et al. and Hodges et al. similarly have discovered a relative lack of intercase correlation across communication and “professionalism” scores in standardized patient assessments.20,21 These findings suggest that this station assessed a distinct domain of ability that is closely linked, but not reducible, to communication skills. Epstein and Hundert recently defined ethical skill and professionalism as “habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served.”22 By its nature, it appears to be an integrative and synthetic capacity with developmental, cognitive, attitudinal, and behavioral components.5,13,22,23 The validity of this new construct, which has been embraced on intuitive and theoretical grounds, requires further evidence-based study.16,24
A third implication relates to the positive role of evaluation in medical education. The HIV consent station certainly stimulated discussion among both students and faculty at our institution, and it is our impression that “putting ethics on the test” through a standardized patient interaction greatly increases the importance of this competence domain in our curriculum. In their written comments, students certainly affirmed the educational value of the HIV consent station and its relevance to their clinical training. After the examination, many students indicated the need for more intensive training around the content areas of HIV, informed consent, confidentiality, and ethics generally. This is consistent with the findings of an earlier survey conducted at the University of New Mexico and the University of Chicago in which 92% of the medical students expressed willingness to care for people living with HIV infection and many requested more curricular attention to this topic area.25 Finally, there is a need for rigorous, reliable performance-based assessments pertaining to ethical practice standards throughout training so that students receive consistent and realistic feedback regarding their knowledge and skills. Such formative comments are critical in trainee development, and they may foster a sense of self-awareness and commitment to continuing education that together will help ensure their future competence.
The psychometric properties of performance-based assessment have been the subject of considerable debate in the medical education literature.26–28 Several factors in the construction of objective structured examinations may help ensure the validity of these tests. These include the use of several faculty to craft checklists, the use of effective and systematic procedures for training standardized patients, the use of the Objective Structured Clinical Examination in conjunction with other evaluation methods, and increasing the number and length of stations, all measures adopted in SPA examinations and the HIV consent exericse.29–34
Even with these efforts, however, our study is open to the central criticism of all performance-based assessments of clinical skills: do students actually behave in real situations as they do with standardized patients?35,36 In the empirical literature, performance-based assessments have been shown in a wide variety of studies to correlate with grades on precertification examinations and clinical evaluations, to discriminate between different levels of skill and training, and to measure dimensions of clinical competence and knowledge that other traditional evaluations may not assess.29,37,38 A prospective study was conducted using standardized patients to evaluate the quality of care for common medical conditions provided by 20 outpatient physicians. In this study, standardized patients were able to accurately assess physicians' quality of care as verified through several measures, and in only 3% of the cases were the physicians able to detect the standardized patients.39 These encouraging findings suggest the importance of continued work in this area of education.
For these and related reasons, we believe that performance-based examinations in medical education offer a “snapshot”—a picture that has value but can capture only a portion of the true or potential capacities of future clinicians.40–42 It appears that triangulation of qualitative and quantitative data, as done in this study, can help improve the interpretation of the results of a performance test.43,44 Nevertheless, we concur with Singer et al., Epstein and Hundert, and others who argue that such assessments should not be used as the sole measure of a student's ethical abilities.22,42,44 Evaluative information pertaining to ethics skills should be integrated with a thorough, multimodal assessment strategy. Viewed from this perspective, standardized patient exercises, such as the HIV consent station described here, may be useful in placing a student on a continuum of developing abilities. The question then becomes not so much whether standardized patient exercises correlate with “real world” performance but whether they can be refined as a heuristic means of improving trainees' abilities in ethics and professionalism.31,43,45 With all of these limitations in mind, we offer the HIV consent station as a starting point for further work in this competence domain.
In conclusion, 3 considerations warrant emphasis in this report.43,44 First, we have characterized our early experience in assessing the informed consent skill of physicians in training. We sought to provide sufficient detail that this educational method could be adapted and translated to a variety of clinical training settings. This approach presupposes that professionalism and ethics skill are abilities that may be developed and systematically examined during medical education. Empirical validation of this kind of work is critical because performance-based assessments are increasingly being incorporated into the evaluative methods of medical schools, residency programs, and specialty and licensing boards. Our data further contribute to the set of evidence in support of ethics and professionalism as a distinct domain for evaluation.
Second, it is increasingly recognized that physicians-in-training must develop the knowledge and skill required to perform HIV risk assessment and counseling.46 The few studies of medical students in this area indicate that specific training related to HIV testing and counseling improves students' self-awareness, enhances their competence in performing HIV testing according to usual standards of care, and possibly influences their choice of specialty.47 These same studies emphasize the value of developing more and more innovative educational approaches to help prepare future physicians for the challenges associated with caring for people with HIV and other stigmatizing infectious diseases.47–49 This report of our HIV consent station is 1 such effort.
Finally, the senior medical students who participated in this evaluation of their clinical ethics skills performed extremely well overall. In light of the overwhelming concern about the ethics skills, personal values, and professional behavior of medical students and residents—a concern that echoes far and wide in the medical literature—our data suggest that there is reason to have faith in the next generation of physicians. It appears that they may well be capable of upholding the shared traditions, standards, and ideals of the profession of medicine.
The authors offer sincere thanks to Drs. Dorothy Barbo, Mike Murnik, Richard Rubin, and Teddy Warner, and Ms. Jan Mines, Ms. Lane Lucas, Ms. Eva Quintana, Ms. Cheri Koinis, and Mr. Khanh Nguyen for their contributions to this work.
Dr. Roberts was supported by a Career Development grant (no. 1K02AI01738-03) from the National Institute of Mental Health.