Systematic Bias Introduced by the Informed Consent Process in a Diagnostic Research Study


  • Presented at the 2007 American College of Emergency Physicians Research Forum, New Orleans, LA, October 2006.
    This work was supported through a collaborative research agreement with Biosite, Inc., and funded in part by NIH/NHLBI Grant R42HL074415 to JAK.

Jeffrey A. Kline, MD; e-mail:


Objectives:  To determine population characteristics, outcomes, and reasons for unsuccessful enrollment among potential study subjects approached for written, informed consent in a minimal-risk emergency department (ED) study. The authors hypothesized that the prevalence of venous thromboembolism (VTE) would be lower among study participants and that medical acuity and refusal to provide a blood sample would be the most common reasons for nonparticipation.

Methods:  The authors requested prospective, written, informed consent for a blood sample and follow-up from consecutive ED patients undergoing evaluation for pulmonary embolism (PE) and recorded spontaneously stated reasons for refusal. VTE was diagnosed or excluded using a combination of D-dimer testing and selective computed tomography (CT) angiography of the chest with venography of the lower extremities. The primary outcome was defined by the number of CT scans positive for VTE among ED patients evaluated for PE.

Results:  Over 16 weeks, 260 of 287 (91%, 95% confidence interval [CI] = 87 to 94%) eligible patients were approached and consent was obtained from 183 patients (64%, 95% CI = 58% to 69%). The prevalence of VTE was 6% among participants and 13% among nonparticipants (95% CI [of the difference] = 1% to 15%). The proportions of African Americans, uninsured, and Medicaid patients were significantly higher among nonparticipants. No significant differences were found in the proportions of nonparticipants who disliked or distrusted research or desired financial reimbursement, compared to those not enrolled due to medical acuity or refused a blood sample.

Conclusions:  These data implicate the written, informed consent process as a significant source of bias on estimated disease prevalence.

The informed consent process has been suggested as a source of systematic bias in diagnostic and clinical registry or cohort studies conducted in a variety of settings. The Canadian Stroke Network demonstrated that the in-hospital mortality rate for the 50% of eligible patients who declined enrollment was approximately three times that of those who consented to participation in the registry. The cost associated with the informed consent process was estimated to account for 25% of the total $2 million (Canadian) budget for the registry.1 Al-Shahi and colleagues reported a 40% nonparticipation rate for a registry of adult patients with brain arteriovenous malformations, and nonparticipants had a significantly higher rate of severe outcomes compared with patients who provided written consent for study participation.2 The Privacy Rule of the Health Insurance Portability and Accountability Act (HIPAA) may have exacerbated the problem by increasing the paper burden associated with the informed consent process.3–5 However, the magnitude of effect of this potentially important source of bias has not been defined in research conducted in the emergency care setting.

We sought to measure the population characteristics and outcomes of two groups of self-referred patients to an urban, academic emergency department (ED): patients who were willing and able to provide written, informed consent for a minimal-risk, noninterventional research study of pulmonary embolism (PE) diagnosis, compared with patients who declined enrollment or could not be successfully enrolled in the same study. We hypothesized that the prevalence of venous thromboembolism (VTE) would be higher among patients who were not enrolled and that a significantly higher proportion of potential study subjects would not be successfully enrolled due to medical acuity or refusal to provide a plasma sample when compared with other spontaneously stated reasons for refusal.


Study Design

This study analyzes data collected from one center during the conduct of a prospective, two-center, industry-sponsored study of potential biomarkers of PE. This study was an externally monitored and Food and Drug Administration (FDA)-regulated study requiring written, informed consent. The informed consent document indicated that patient participation would require a case report form, phlebotomy for a 10-mL specimen of venous blood, and telephone and medical record follow-up at 45 and 90 days. Participants did not incur any additional study-related costs and did not receive monetary compensation for their participation. A complete copy of the consent form used for this study is available as a Data Supplement. This study was approved by our institutional review board. For this study, we defined a positive outcome as a contrast-enhanced computed tomography (CT) angiography of the chest with lower extremity venography demonstrating the presence of a VTE (PE and/or deep venous thrombosis) performed and interpreted using previously described criteria.6

Study Setting and Population

The study was conducted in the ED at Carolinas Medical Center in Charlotte, North Carolina. This is an urban, teaching hospital staffed by board-certified emergency physicians (EPs) 24 hours a day and has an Accreditation Council of Graduate Medical Education (ACGME)-accredited residency in emergency medicine. At this institution, physicians use D-dimer screening in low- and moderate-risk patients and selective pulmonary vascular imaging in high-risk patients and patients with a positive D-dimer. Chest CT imaging for PE is available at all times and images are interpreted within 3 hours by a board-certified radiologist.6 More than 99% of patients are diagnosed with PE on the basis of a CT imaging study; less than 1% of ED patients evaluated for PE at this institution undergo other means of pulmonary vascular imaging such as ventilation–perfusion scanning or formal pulmonary angiography.

Consecutive ED patients evaluated for PE from August 10, 2005, through October 9, 2005, were approached for written informed consent for participation in this study prior to the results of diagnostic tests. The inclusion criteria required that an EP place an order for a D-dimer or CT of the chest to rule out PE. We obtained partial waiver of authorization to screen ED charts and order-entry systems to survey for D-dimer testing or CT ordered for the purpose of evaluating a patient with suspected PE. Exclusion criteria included age less than 18 years, chronic hemodialysis, cardiac catheterization within 28 days, active heart failure, and current anticoagulation for known PE. Patients meeting exclusion criteria were screened but not approached for enrollment.

The informed consent process was guided by qualified research coordinators, 16 hours a day, 6 days per week. Coordinators also supervised undergraduate research students, who assisted in screening patients. Students were able to obtain written informed consent, but only in the presence of a study coordinator. Study coordinators and their student assistants did not participate in patient care and are not authors of any study-related manuscripts. Both students and coordinators completed Good Clinical Practice training consistent with the tripartite-harmonized guidelines.7 Consistent with the Standards for Reporting of Diagnostic Accuracy (STARD) initiative, we required that > 85% of eligible patients be approached to meet our definition of “consecutive.”8 Patients were informed that their participation was voluntary, that all information would be deidentified prior to analysis, that neither they nor their physicians would have access to the results of experimental marker measurements, and that they would not be financially compensated for their participation (see Data Supplement).

Study Protocol

Eligible patients were approached for written informed consent prior to the availability of the results of diagnostic testing. After obtaining consent, venous blood samples were collected concurrently with blood draws performed for usual care, when possible. Patients were only approached for a separate venipuncture when these measures were unsuccessful. Study participants were followed for a total of 90 days for the development of VTE. The details of the method of follow-up have been previously described.9


The total number (denominator) of eligible patients was determined by a review of administrative databases that recorded D-dimer and CT orders (McKesson Inc., San Francisco, CA). The overall prevalence of VTE was defined as the number of CT studies interpreted as positive for VTE (PE and/or deep venous thrombosis) in eligible ED patients during the study enrollment period. These databases were also used to determine the total number of eligible patients and to determine the number of patients missed by the screening and enrollment process. Coordinators recorded the proportion of potential study subjects not approached due to medical acuity on a screening log. Medical acuity precluding consent was defined as a condition that would preclude legally effective informed consent as defined in Federal Register 21 CRF50.24. Coordinators also recorded reasons for refusal as spontaneously stated by potential study subjects on the screening log. Coordinators did not prompt potential study subjects to give a reason for refusal. Following a refusal for participation, the coordinators were instructed to succinctly thank the patient for his/her time and leave the room without delay. No personal heath identifiers were recorded on the log. These statements were recorded on the screening log and were later independently reviewed by two researchers who did not participate in the written informed consent process for any participant in this study. Both reviewers have experience in obtaining written informed consent for other studies of PE and VTE conducted by the Department of Emergency Medicine at Carolinas Medical Center and are familiar with the standard operating procedures for the conduct of clinical research studies used by this department. The reviewers classified each statement into the following, predefined, seven categories: “unable due to medical acuity,”“refused to provide a plasma sample,”“fatigue or subjective feeling of illness,”“concern for privacy of medical records,”“dislike or distrust of research,”“desired financial reimbursement,” and “other or no reason volunteered.”

Data Analysis

Data were compiled into spreadsheet format (Microsoft Office Excel 2003, Microsoft Corp., Seattle, WA) for analysis. All statistical analyses were performed using STATSDirect V3.3 software (Cheshire, UK). Overall outcome prevalence, frequencies of refusal or inability to consent, and population characteristics are reported as univariate data (proportions) with associated 95% confidence intervals (CIs) calculated using the Clopper-Pearson method. Significant differences between participants and nonparticipants are reported both as the 95% CI of the difference and by chi-squared analysis with the resulting odds ratio and associated two-sided p-values calculated using Fisher’s exact test.


Objective diagnostic testing for PE (D-dimer and/or CT angiography of the chest with venography) was ordered by the evaluating EP in 287 patients eligible for participation in this study during the study interval. Of these, 260 of 287 (91%; 95% CI = 87% to 94%) potential study subjects were identified and approached for enrollment. Written informed consent for participation was obtained in 183 of 287 (64%; 95% CI = 58% to 69%) eligible patients and 70% (95% CI = 64% to 76%) of those approached for consent. Forty participants (21%, 95% CI = 16% to 29%) required a separate venipuncture to collect the study blood sample. Five coordinators (two female and three male) obtained written informed consent for this study and more than 90% of patients were approached by two female coordinators fluent in English and Spanish. Additionally, all of the student assistants were female, and three of four were African American. Spontaneously stated reasons for refusal are summarized in Figure 1. There were no disagreements in the classification of reasons for refusal between two blinded reviewers.

Figure 1.

 Summary of informed consent process and reasons for refusal of consent. ED = emergency department.

During the study period, D-dimer testing was performed in 248 of the 287 patients eligible for this study (86%, 95% CI = 82% to 90%). Within the same population, the EP ordered a CT in 149 of 287 patients (52%, 95% CI = 46% to 58%), 39 of whom were not initially screened by D-dimer testing. Two ventilation–perfusion scans were performed in ED patients during the study period, and both patients met one or more exclusion criteria. No formal pulmonary angiograms were performed in ED patients during the study period.

Twenty-five (9%, 95% CI = 6% to 13%) potential study subjects had a VTE (PE and/or deep venous thrombosis) determined by CT during the study. The prevalence of VTE among those enrolled in the study was 11 of 183 (6%, 95% CI = 3% to 11%), compared with 14 of 104 (13%, 95% CI = 8% to 22%) among those who were missed, refused to consent, or were unable to provide consent. The difference in VTE prevalence was 7% (95% CI of the difference = 1% to 15%). Table 1 compares population characteristics (age, gender, race, and medical insurance status) between the two groups and shows a significantly greater proportion of whites and patients with private insurance in the enrolled group. Conversely, the proportions of African American patients and patients with Medicaid and uninsured or unknown insurance status are significantly higher among those not enrolled in this study. Table 2 reports these characteristics for patients with VTE and compares participants with VTE with nonparticipants with VTE.

Table 1. 
Comparison of the Population Characteristics and Insurer Status of Potential Study Subjects Who Were and Were Not Enrolled
CharacteristicEnrolled (n = 183)Not Enrolled (n = 104) 95% CIdiffRRp-Value
  1. 95% CIdiff = 95% confidence interval of the difference; N/A = not applicable; RR = relative risk

Age, yr (SD)48 (16)49 (16)N/AN/AN/A
Female gender, n (%)115 (63)63 (61)−9, 141.10.704
Race, n (%)
 African American73 (40)55 (53)−24, 00.590.036
 White (not Hispanic)101 (55)43 (41)1, 251.70.027
 Other or unknown9 (4)6 (6)−8, 40.840.786
Insurer status, n (%)
 Private93 (51)22 (21)18, 403.85< 0.0001
 Medicare45 (25)29 (28)−14, 70.840.576
 Medicaid29 (16)28 (27)−22, −10.510.031
 Self-pay or unknown16 (9)15 (24)−25, −70.270.0001
Table 2. 
Comparison Population Characteristics of Potential Study Subjects with Venous Thromboembolism Who Were and Were Not Enrolled
Age (yr)GenderRaceMedical Insurance
28MOther or unknownSelf-pay or unknown
31MAfrican AmericanPrivate
52FAfrican AmericanPrivate
Mean (SD) = 57 (18)
Not Enrolled
30FWhiteSelf-pay or unknown
52MAfrican AmericanPrivate
62FAfrican AmericanPrivate
Mean (SD) = 56 (15)


Informed consent bias in clinical research has been identified in other settings,1,2,5 but to our knowledge has not previously been measured in the ED setting. This study quantifies the influence of the informed consent process as a source of systematic bias resulting in a significantly lower estimated prevalence (difference of 7%) in the primary study outcome among study participants. This study also identifies possible reasons why patients refuse to participate in a minimal-risk research study conducted in an ED.

Many potential confounders could have affected each step of the informed consent process. We address several of these in this study. First, the informed consent process requires the availability of researchers to screen and enroll patients, and the results of the study are susceptible to the effects of missing eligible patients. This is a particular challenge in studies that seek to enroll a consecutive sample from an ED providing care 24 hours a day, 7 days a week. We are unaware of any accepted standard defining a “consecutive” sample in the context of the potential for error in the screening process. Item 5 of the STARD checklist suggests the population studied should represent the population of interest.8 To reduce the probability of bias with a feasible goal, we used the predetermined standard of > 85% of eligible patients approached for enrollment. Second, an individual’s willingness to, or ability to, provide written informed consent is not a potential confounder until written informed consent is required. Thus, we have chosen the setting of a minimal-risk, noninterventional study with minimal requirements of the participants, as opposed to an interventional study, for a focused examination of the written informed consent process itself. Finally, because only a small percentage of patients did not offer a reason for refusal, this suggests that we did not miss important beliefs held by our patient population.

As expected, medical acuity and refusal of the blood sample accounted for a significant number of failures to obtain consent. We anticipated that a small number of patients with severe PE would meet our definition of “medical acuity precluding enrollment.” However, many patients with PE are also relatively well-appearing and medically stable enough to provide written informed consent and undergo an interview and blood draw.

Unexpectedly, dislike or distrust for research and desire for financial reimbursement accounted for equal proportions of nonparticipants compared to medical acuity and refusal of the blood sample. Nonparticipants were also more likely to be African American and have Medicaid as their primary source of payment for medical care or lack insurance all together. Prior research has demonstrated that African Americans are less likely than whites to believe that their own physicians would fully explain research participation and more likely to believe that their own physician would expose them to unnecessary risks.10 Shavers et al. also found that African Americans believe that the burden of medical research is not shared equally among racial and ethnic groups.11 In the emergency care setting, these beliefs may be exacerbated.

For 2004–2005, 42% of North Carolina residents had either Medicare or Medicaid or were uninsured. Of these, approximately 30% are African American.12 In this study, the proportions of uninsured or Medicaid patients were higher among nonparticipants and were higher than the state averages. Insurer status has been shown to be a reliable surrogate for economic status.13 Thus, our data agree with prior findings suggesting that these populations may not have sufficient assurance of protection to be willing to participate in a minimal-risk study.10,11 Moreover, the involvement of African American researchers in the informed consent process in this study did not appear to diminish this bias toward nonparticipation among poor African American patients.

Interestingly, neither African Americans nor patients lacking private insurance were underrepresented among those with a positive outcome in either group (Table 2). It is possible that medical acuity may have been a stronger source of confounding than other potential factors among patients with a VTE.

This study was considered to be of minimal risk by our IRB, consistent with the definition used by other institutions. It is also a study of a disease that is considered to be potentially life-threatening and included patients with the potential to be impaired or unavailable for consent due to medical acuity. In this study, medical acuity was an important reason for why consent could not be obtained. The Common Rule suggests that this study could have satisfied all published federal requirements for waiver of, or exception from, informed consent.14–17 However, many IRBs interpret the rule more conservatively.18 This study reiterates previous recommendations for clarification of the recommendations for waiver of or exception from consent.15,17,19

Concern for privacy, which is the basis of concerns related to HIPAA enforcement, and a major factor associated with the development of the Common Rule, was the stated reason for refusal to consent in a minority of cases, suggesting that patients do not perceive this as a major obstacle to participating in research. Other work has also demonstrated that community members are accepting of minimal-risk research protocols that have no direct benefit to participants.20,21 Thus, the significant and increasing paperwork burden introduced by recent HIPAA-related policies is likely not justified in addressing patients’ concerns of privacy with regard to research.15,20,22


The study was conducted at a single center and the results could vary in other settings. An interaction effect due to the age, gender, race, or other characteristics of the coordinator, potential study subject, or encounter could not be evaluated on an individual level. We did not record the number of interactions during which a language interpreter was used. Finally, we are unable to determine the effect of age, gender, race, or insurer status on the EP’s decisions to initiate diagnostic testing for PE.


Our data show a lower estimated prevalence of VTE among patients who participated in the written, informed consent process than the estimated prevalence of VTE among patients who refused participation or were unable to participate in this study. The informed consent process may represent an important source of systematic bias, especially in this low-prevalence population. Furthermore, African Americans and uninsured patients, or patients with Medicaid as their primary source of payment for medical care, were less likely to participate in this minimal-risk study. These observations underscore the importance of reevaluating waiver of and exception from informed consent in minimal-risk studies conducted in the ED.