- Top of page
- Statistical Analysis
Cardiac physical examination is a declining art,1–5 even though it is a sensitive, specific, and cost-effective tool for screening for cardiac disorders.6–8 At Harbor-UCLA, where we have archived cardiac examination (CE) findings for >10 years with video,9 we found that 25 of 254 patients recorded had new physical findings on bedside recording that materially changed the diagnosis and treatment plan (unpublished data). For example, diminished femoral arterial pulses were noted in a young patient with aortic regurgitation, indicating the presence of previously unsuspected aortic coarctation.
Ideally, CE involves integration of patient history with inspection, palpation, auscultation, and, when appropriate, maneuvers to bring out pathophysiological findings.10–14 Although objective assessment of competence in patient examination remains a challenge,15,16 there is a growing body of literature documenting an alarming lack of competency in current medical trainees and practicing physicians,1,10,17–19 despite the fact that auscultatory training in internal medicine programs has been increasing.20
Classroom testing with a validated multimedia test of CE had been reported in a previous study, but only 7 cardiologists (3 faculty level, 4 private practice) had participated.1 To enlist more academic cardiologists, we provided custom CD-ROMs that could be installed at academic centers that would upload test scores to a central secure database via the Internet. To ensure that remote testing was statistically similar to classroom testing, mean scores from each training level obtained by both methods of testing were compared.
- Top of page
- Statistical Analysis
From August 6, 2004 to October 15, 2008, we received 527 test scores with demographic data from 19 medical centers. From these 527 participants, 7 had specified a training level of “other” and were not used in this analysis. Likewise, 37 of the 860 classroom test scores collected in a previous study between July 10, 2000 and January 5, 2004 were marked as “other” and were not used.
Table 1 provides the mean values, standard deviations, and 95% confidence intervals for each training level group for both remote and classroom testing. The sample size of remotely tested groups over classroom testing was 57 vs 3 for cardiology faculty, 10 vs 0 for cardiology volunteer clinical faculty, 4 vs 4 for private-practice cardiologists, and 150 vs 85 for cardiology fellows. Students, residents, and noncardiology faculty were also tested remotely. Figure 2 plots mean CE test scores by training level. With remote testing, mean scores were highest for cardiology volunteer clinical faculty (86.3), cardiology faculty (82.0), cardiology fellows (77.3), and cardiologists in private practice (77.0). Mean scores for medical students (63.7), internal medicine (62.7) and family medicine (59.2) residents, and noncardiology faculty (67.3) were ≥10 points lower than scores for cardiologists and fellows. The difference between cardiology and others was P < 0.05.
Figure 2. Mean CE test scores by training level. Classroom testing was conducted by the authors with a computer that projected animations and patient videos onto a screen, and sound transmission into headphones or stethophones, with answers recorded on paper. Remote testing used personal computers to administer the test individually; participants entered their answers directly into the computer. Abbreviations: CE, cardiac examination; IM, internal medicine; FM, family medicine.
Download figure to PowerPoint
Table 1. CE Test Scores From Remote and Classroom Testing
|Groups Tested||N||Mean||SD||95% CI |
|Remote testing (N = 520)|| || || || |
| Cardiology faculty||57||82.0||12.4||3.3|
| Cardiology volunteer faculty||10||86.3||11.2||8|
| Cardiology fellows||150||77.3||13||2.1|
| Medical students||78||63.7||15.4||3.5|
| IM residents||95||62.7||15.7||3.2|
| FM residents||107||59.2||16.7||3.2|
| Noncardiology faculty||19||67.3||18.2||8.8|
| Noncardiology private practice||NT||NT||NT||NT|
| Cardiology private practice||4||77.0||4.2||6.8|
|Classroom testing (N = 823)|| || || || |
| Cardiology faculty||3||80.0||7.6||18.75|
| Cardiology volunteer faculty||NT||NT||NT||NT|
| Cardiology fellows||85||71.8||16.4||3.6|
| Medical students||318||56.8||15||1.7|
| IM residents||226||61.5||14.2||1.9|
| FM residents||63||56.5||13.6||3.5|
| Noncardiology faculty||59||58.3||14.2||3.7|
| Noncardiology private practice||65||55.8||16.8||4.2|
| Cardiology private practice||4||66.3||15.6||33.5|
Comparison of the 2 studies (classroom vs remote) for the different training levels via 2-factor ANOVA revealed that the P value for the interaction between training level and classroom vs remote testing was 0.28, indicating that there was no significant difference in test scores obtained for each training level by either test method. Therefore, we combined the 2 sets of results for a common overall assessment of training level differences. Comparison of differences in test scores by training level had a P value < 0.0001, whereas differences by classroom vs remote testing had a P value < 0.01. The partial eta squared (η2, or the relative contribution of each factor to the observed difference in scores) was heavily weighted toward training level (partial η2 = 0.161) over that of remote vs classroom testing (0.016).
We did not test noncardiology private-practice physicians in the new remote testing study, and we did not test cardiology volunteer faculty in the old classroom testing study, and were therefore unable to measure directly how these groups may have differed from the corresponding other study. We assume that the study differences for these 2 groups would be similar to the differences observed in the other training levels that were tested in both studies.
Based on the results of the 2-factor ANOVA, we had reasonable confidence that remote and classroom testing could be combined to make comparisons across all training levels, from medical students to cardiology faculty. Table 2 shows results from Student Newman-Keuls comparisons of mean CE test scores confirmed the observed difference in scores between cardiology faculty, cardiology volunteer faculty, cardiologists in private practice, and cardiology fellows vs every other training level (P < 0.05).
Table 2. Stratification of Test Scores Into Distinctively Similar Groups (N = 1343)
| || ||Subset for α = 0.05|
|Student Newman-Keuls||N||Group 1||Group 2||Group 3||Group 4 |
|Cardiology volunteer faculty||10||86.3|| || || |
|Cardiology faculty||60||81.9||81.9|| || |
|Cardiology fellows||235|| ||75.3||75.3|| |
|Cardiology private practice||8|| || ||71.6|| |
|IM residents||321|| || || ||61.9|
|Noncardiology faculty||78|| || || ||60.5|
|Medical students||396|| || || ||58.2|
|FM residents||170|| || || ||58.2|
|Noncardiology private practice||65|| || || ||55.8|
To measure the effect of years in practice on CE test scores, 91 faculty and volunteer faculty cardiologists and internists for whom years since completion of training were available were grouped into 5-year increments. The results are plotted in Figure 3. The highest scores were obtained by the most senior faculty members, with scores decreasing linearly the more recently the physician was trained (P < 0.004).
- Top of page
- Statistical Analysis
Remote and confidential testing of full-time academic faculty cardiologists provided access to a previously untested population cohort that performed (as would be expected) better than any other group of examinees with the exception of cardiologists on the voluntary teaching faculty. Remote testing appeared to be a viable alternative to classroom testing by installing the CE test on computers at teaching centers where cardiologists were able to take the test and upload their answers confidentially.
When the number of years after completing training are compared with test scores (Figure 3), we observed generally greater CE test scores with increasing seniority. There are 2 potential interpretations. The more comforting explanation is that performance improves with years of experience. A less comforting explanation is that older physicians benefited from superior teachers at the time of their medical school and residency training, and that later experience had little impact on CE skills. Our earlier report of classroom testing showed no improvement in CE test scores after the second year of medical school; in fact, internal medicine faculty tested no better than third-year medical students.1 For this reason, we believe it is less likely that the increase in CE test scores is simply due to years of experience. On the contrary, we believe that competence in bedside CE requires: (1) preparation, (2) supervised exposure to patients with cardiac findings, and (3) patient exposure with critical reinforcement.
Figure 3. Linear regression of CE test scores achieved by faculty members. Mean CE test scores for cardiologists and internists are plotted against 5-year increments since completion of their training. Scores decrease linearly the more recently the physician was trained. Abbreviations: CE, cardiac examination.
Download figure to PowerPoint
Preparation in CE begins with acquisition of a basic knowledge of cardiac physiology and pathophysiology in the first 3 years of medical school. Supervised exposure to patients with cardiac findings is principally confined to the third year in most medical schools,14 but it can be enhanced by taking fourth-year electives offering supervised exposure by competent mentors. “Critical reinforcement” implies a commitment to confirming or refuting one's bedside diagnostic impressions by critical review and correlation with available imaging and/or hemodynamic studies performed on that patient. Cardiology fellows indeed benefit from exposure to patients with a wider spectrum of cardiac findings than trainees would be expected to encounter in family and internal medicine. Moreover, fellows augment this natural advantage in patient population by ready access to special studies that provide the necessary critical reinforcement to improve their examination skills. We propose that this mode of exposure is the means by which cardiology fellows obtained better CE test scores, and why cardiologists continue to improve over time.
On the other hand, patient exposure without critical reinforcement seems to be the norm for the average medical resident, explaining their lack of advancement in scores despite clinical encounters with hundreds of patients. Some form of critical reinforcement is needed for all trainees to improve their cardiac examination skills.
The lack of improvement in scores achieved by full-time faculty members in internal medicine may also represent the absence of critical reinforcement during their own training and therefore suspension of any further advancement in this complex skill. Unfortunately, internists have become largely responsible for teaching CE to medical students and residents.23 Cardiac examination is unlikely to improve at any training level without first improving the skill levels of the faculty who teach it.
Compounding the problem of critical reinforcement is the substantial drop in available teaching material: teaching hospitals no longer have a ready population of patients with heart valve disease from which to learn. Although schools and training programs continue to teach CE on the inpatient services, patients with heart valve disease are now almost exclusively processed as outpatients, where they are followed, treatment plans are made, and preoperative assessments are done. When these patients do arrive at the hospital for same-day surgery, if they are not immediately wheeled into the surgical suite, they often spend very little time in their hospital room while being transported elsewhere for tests, minimizing time available for obtaining a cardiac history and bedside examination, and preventing access on teaching rounds. Further, cardiac patients are often festooned with electrocardiographic electrode patches strategically placed over the precordium, inhibiting stethoscopic access to desired listening areas; in addition, ambient beeps emanating from monitors, infusion pumps, and ventilators add to a cacophony that inhibits contemplative auscultation.
The consequence of cursory CEs that miss important findings is delay in proper diagnosis and appropriate treatment. These “objective findings” in the medical record can and do affect subsequent workup and therapy. It is especially true when the initial workup in a patient's chart contains the statement “S1, S2, no MRG” (first heart sound; second heart sound; no murmurs, rubs, or gallops). This terse entry too often becomes a “rubber stamp” entry on all subsequent history and physical entries in the chart. Although we have documented 10% of patients in our collection as having materially different cardiac findings, it is likely that this error rate underestimates the true magnitude of the problem. Our patients have come from a teaching hospital that has emphasized CE competency in its trainees and attending physicians; other hospitals may have even higher error rates. Further, we do not know the false-negative rate, where patients have important cardiac findings that remain undetected, because we do not routinely screen patients with normal findings.
Fortunately, convenient “on demand” access to patients with important CE findings is possible by the use of patient surrogates. Several approaches that have been used are: (1) audiotapes and compact disks, (2) manikins, and (3) multimedia VPEs. Audio-only programs are disadvantaged by the inability to use visible or palpable timing signals to distinguish systole from diastole, or to discern invaluable ancillary diagnostic information inherent in the appearance of the patient or the quality of the carotid, jugular, and precordial pulses. Manikins (ie, artificial patients) have also been employed, but are costly, space consuming, and require skilled maintenance for optimal performance. As simulations, manikins are the result of editorial choices on what to leave in, and what to leave out, of the training experience. At best, they present a particular lesion as an archetype (sacrificing the spectrum of presentations a disease may have in nature). At worst, manikins may introduce errors based on an incomplete understanding of the disease.24 In addition, the simulated sounds and murmurs are never very realistic and probably do no more than help the learner with timing.
Some form of objective, reproducible, and realistic testing is necessary if CE skills are to improve. The traditional objective structured clinical examination, which employs standardized patients (healthy individuals trained to deliver a medical history and symptoms), cannot adequately portray abnormal heart sounds and murmurs with altered carotid and jugular venous pulsations. Some medical schools have attempted to supplement these stations with audio recordings of heart sounds, but the lack of a timing reference (either a carotid, jugular, or apical pulsation) severely limits their utility. In contrast, the National Board of Medical Examiners has incorporated actual heart sounds and animated virtual patients into its US Medical Licensing Examination. We believe that this form of testing, with synchronized pulsations that have realistic contours that match the pathology portrayed, is superior to using standardized patients. For residents, the dearth of an inpatient population with bedside cardiac findings means that what patient encounters they do have must be supplemented with VPEs. For any training level, a minimum of 5 hours of practice appears to be necessary to create any significant improvement in CE skills,10 and 12 hours of practice is probably necessary for the skills to improve without further intervention.14
Computer technology has advanced rapidly over the past decade, making multimedia programs reliable, reproducible, and affordable, and capable of being transmitted or downloaded over the Internet. The same technology used for CE testing can also be used for CE training. These programs can be studied by individuals or small groups, often a few steps away from the patient's bedside. The immediacy of critical reinforcement can make the acquisition of CE skills less dependent on a dwindling population of master clinicians, and more widely available to physicians who need them.
Remote testing of previously tested training groups (cardiology volunteer faculty, cardiology fellows, residents, students, and noncardiology faculty) revealed slightly higher mean CE test scores over classroom testing, although this difference was generally not significant. For medical students, however, mean scores from remote testing were significantly higher (63.7 vs 56.8, P < 0.05). One explanation is that remote administration allowed students more time to contemplate their answers, because they could complete the test at their own pace, as opposed to classroom administration of the test, which requires that all participants complete answers at the same pace. A more likely explanation is that the medical students tested remotely were a special population of students with slightly greater interest or skill in cardiology; 60 of 78 students tested remotely were in their fourth year, presumably during their elective cardiology clerkship. Noncardiology faculty tested remotely did so at the behest of a cardiologist, whereas the majority of noncardiology faculty members tested in a classroom setting were attendees of a heart sounds workshop at an annual meeting of the American College of Physicians. Therefore, we believe that the observed differences in remote and classroom testing were probably the result of mild sample bias. For both testing methods, some selection bias was probably at work, because people who were more confident in their skill would be more likely to volunteer to take the test, whereas those less confident would be more likely to avoid it. Therefore, the actual skill levels for each training level are probably worse than reported here.