The authors have no conflicts of interest to report.
A poster was presented at the Society of General Internal Medicine 27th annual meeting, Chicago, IL, May 15, 2004.
Address correspondence and requests for reprints to Dr. Nápoles-Springer: University of California San Francisco, 3333 California Street, Suite 335, San Francisco, CA 94118-1944 (e-mail firstname.lastname@example.org).
Background: Limited evidence exists on the effectiveness of recruitment methods among diverse populations.
Objective: Describe response rates by recruitment stage, ethnic-language group, and type of initial contact letter (for African-American and Latino patients).
Design: Tracking of response status by recruitment stage and ethnic-language group and a randomized trial of ethnically tailored initial letters nested within a cross-sectional telephone survey on physician-patient communication.
Participants: Adult general medicine patients with ≥1 visit during the preceding year, stratified by 4 categories: African-American (N= 1,400), English-speaking Latino (N= 894), Spanish-speaking Latino (N= 965), and non-Latino white (N= 1,400).
Measurements and Results: Ethnically tailored initial letters referred to shortages of African-American (or Latino) physicians and the need to learn about the experiences of African-American (or Latino) patients communicating with physicians. Of 2,482 patients contacted, eligible, and able to participate (identified eligibles), 69.9% completed the survey. Thirty-nine percent of the sampling frame was unable to be contacted, with losses higher among non-Latino whites (46.5%) and African Americans (44.2%) than among English-speaking (32.3%) and Spanish-speaking Latinos (25.1%). For identified eligibles, response rates were highest among Spanish-speaking Latinos (75.2%), lowest for non-Latino whites (66.4%), and intermediate for African Americans (69.7%) and English-speaking Latinos (68.1%). There were no differences in overall response rates between patients receiving ethnically tailored letters (72.2%) and those receiving general letters (70.0%).
Conclusions: Household contact and individual response rates differed by ethnic-language group, highlighting the importance of tracking losses by stage and subpopulation. Careful attention to recruitment yielded acceptable response rates among all groups.
Inclusion of ethnic minorities in clinical research is vital to address ethnic disparities in health. Despite a federal mandate to include minorities and women in research, limited evidence exists on the effectiveness of recruitment methods among diverse populations.1,2 Most of the recruitment literature on ethnically diverse groups consists of overviews of historical, attitudinal, and structural barriers to participation and strategies for addressing these, or descriptive accounts of recruitment results.3 Rigorous tests of specific strategies are rare.4 Empirical evidence is needed to guide selection of recruitment and retention strategies among diverse populations. Estimates from our previous studies among diverse populations indicated a loss of 40% to 50% of our sampling frames due to the inability to locate respondents, either due to incorrect contact information or lack of a response to a letter or telephone call, which appears to be consistent with other population- or clinic-based studies.4,5 Thus, we devoted additional resources to designing, implementing, and tracking recruitment in a cross-sectional telephone survey of interpersonal processes of care among ethnically diverse adult general medicine patients. We utilized 5 strategies to enhance response rates. First, we designed and pretested initial contact letters to maximize their appeal and readability among potential participants identified through a medical records database. Second, we used a bilingual format with Latinos. Third, we developed ethnically tailored letters for African Americans and Latinos. Fourth, we enhanced the recruitment protocol for Latinos given their smaller numbers in the sampling frame. Finally, we monitored response rates at various stages of the recruitment process by ethnic and language group to identify points at which potential respondents were lost.
Within the survey study, we nested a randomized trial to examine the effects of ethnically tailored initial contact letters on response rates. We selected the first mailing as the intervention point because mailings may offer a low-cost, low-intensity method for recruiting ethnic minorities.6 Also, many Institutional Review Boards (IRB) require that initial contact with patients occur through the mail rather than by telephone or in person. Little empirical data exist on whether the content of an initial letter influences the rate of subsequent telephone contact. We hypothesized that raising awareness through the letter about the specific concerns of an ethnic group with respect to their interactions with physicians would increase their response rates to the telephone survey compared to a more general letter. Thus, the purpose of this study was to: 1) describe response rates by stage and ethnic-language group and 2) assess the effectiveness of an ethnically tailored initial contact letter on response rates to a telephone survey on physician-patient communication.
Based on previous models7,8 and research3,9–12 on recruitment and retention of ethnic minorities, we developed a framework of factors (e.g., sampling frame, family and community context, recruitment and study methods, personnel) that can affect response at various stages of the recruitment and retention processes (see Appendix available online at http://www.jgim.org; Figure 1). The ability to identify factors leading to differential response rates by stage and ethnicity can facilitate the adjustment of recruitment strategies to improve their effectiveness.12 In this study, we apply the framework to examine differential loss in the sampling frame by stage and ethnic-language group.
A randomized trial of the ethnically tailored letter was nested within the cross-sectional telephone survey that aimed to assess the quality of interpersonal processes of care occurring during the medical encounters of ethnically diverse patients. Efforts to contact patients who received an initial invitation letter to participate in the telephone survey were monitored and a final disposition was ascertained in all cases.
The sampling frame consisted of adults who had made at least 1 visit between April 30, 2000 and April 30, 2001 to the University of California San Francisco-Mt. Zion adult primary care clinics. The ethnic composition of these adult outpatient practices during 2001 was 10% African-American, 15% Latino, and 60% non-Latino white; the payer mix was approximately 40% full-capitation managed care, 30% Medicare, 25% Medicaid, and 5% uninsured. The sampling frame was derived from the medical records database and contained information on patients' ethnicity, language, gender, age, and health insurance. We stratified the sampling frame into 4 ethnic-language categories: African-American (N= 2,369), English-speaking Latino (N= 894), Spanish-speaking Latino (N= 965), and non-Latino white (N= 10,822). We selected a random sample of 1,400 African Americans and 1,400 non-Latino whites and all Latinos (as there were fewer in the sampling frame). Our goal was to recruit approximately 400 patients in each ethnic-language group. Thus, a total of 4,660 initial contact letters was mailed to patients.
As required by the IRB, patients were sent an initial contact letter explaining the purpose of the study, and how their name was obtained. The general initial contact letter requested participation in a telephone survey of the interpersonal processes occuring during medical visits. Letters contained the following elements: emphasized the need for information on patients' opinions on communication occurring during medical encounters; used concise and clear language; stressed that the study was being conducted by researchers and not their health care provider; provided reassurance that the responses of individuals would not be shared with their doctors; mentioned a $ 20 payment; used the logo of the academic institution; and were written at a seventh grade reading level.
Alternative designs of initial contact letters and envelopes were pretested among 36 African Americans and Latinos recruited from community settings and not included in the main study. Respondents were asked to compare an envelope with the university logo only, to one that also included a picture of a patient and physician with a caption that read, “Your opinions matter. Help us improve communication between doctors and patients. Please consider taking part in this study.” Pretesting revealed that 78% of respondents indicated they would be more likely to open and read a letter enclosed in the envelope with the drawing and caption, so this was the final general envelope used. For the general letter, a postage stamp with a picture of fruit was used.
The ethnically tailored letter was the same as the general letter except that it included two additional sentences and a modified sentence. In the opening paragraph, these sentences read, “As we all know there is a shortage of African-American (Latino) doctors. We are trying to learn from African-American (Latino) patients about their personal experiences in communicating with their doctors. If you take part in this study, you may help us develop better ways to train non-African-American (non-Latino) doctors so that patients are more satisfied with their care.” In the pretest, 81% of the African-American and Latino participants indicated they would be more likely to participate in the survey in response to the ethnically tailored letter than the general letter. We also created an ethnically tailored envelope by replacing “patients” on the general envelope with “African-American (or Latino) patients.” For Latino patients, the caption on the envelope was printed in both English and Spanish. For the ethnically tailored envelopes, we used a postage stamp of Roy Wilkins for African Americans or Frida Kahlo for Latinos.
Patients identified in the sampling frame as Latino or African American were randomly assigned within each ethnic-language group to receive either a generic or ethnically tailored initial letter. Random assignment occurred using a SAS program (SAS Institute, Cary, NC). White patients were not included in the randomized trial and received the general envelope and letter in English.
The computer-assisted telephone interview (CATI) survey was conducted by a professional health survey company between October 1, 2001 and January 31, 2002. A total of 17 trained and experienced interviewers conducted the surveys, including 7 who were bilingual in English and Spanish. Bilingual staff conducted all telephone calls to potential respondents identified as Latino.
Eligibility was confirmed and verbal informed consent obtained as approved by the IRB. The survey lasted 31 minutes on average (range 21 to 42 minutes), and contained questions on the quality of communication (general clarity, elicitation of and responsiveness to patient concerns, explanations), decision making (responsiveness to patient preferences, consideration of ability and desire to comply), and interpersonal style (friendliness, respectfulness, discrimination, cultural sensitivity, emotional support, empowerment) of their physicians.13 The protocol specified at least 15 telephone attempts to each patient, approximately 2 weeks after mailing of the initial letter. Calls were made to each person at varying times (including evenings) and days of the week (including weekends).
The enhanced recruitment protocol for Latinos specified additional follow-up efforts in cases where telephone numbers or addresses were incorrect or nonresponse continued after 15 telephone attempts. This protocol specified a review of the electronic medical record for updated contact information, a search of online telephone directories, review of telephone call logs, follow-up of telephone calls received, subsequent mailings to forwarding addresses for returned mail, one repeat mailing, and up to 30 telephone attempts.
A computer database was used to track the disposition of all patients in the sampling frame and all telephone calls made and received. The response status was tracked by stage of recruitment, ethnicity, and type of letter (for nonwhite patients). Response rates were calculated using the ratio of completed surveys to those patients in the ethnic-language group we were able to contact, who were eligible, and who were not too ill to participate, henceforth referred to as “identified eligibles.”
Chi-square analyses were used to examine demographic differences among participants and nonparticipants. Multivariate logistic regression was used to assess the independent effects of demographic variables on response status. For the randomized trial of the ethnically tailored mailing, within each minority ethnic-language group and for the total nonwhite sample, we compared the proportion that responded to the survey by letter type using χ2 tests.
The sample for the recruitment study consisted of 4,613, representing the total number of initial contact letters mailed minus 47 potential respondents (26 African Americans, 1 Spanish-speaking Latino, and 20 non-Latino whites) who were dropped from these analyses because telephone contact was never attempted by the survey company due to reaching the sample quota for ethnic-language strata (n= 46) or oversight (n= 1). The recruitment sample included 1,374 African Americans, 894 English-speaking Latinos, 965 Spanish-speaking Latinos, and 1,380 non-Latino whites.
Of 4,613 potential respondents, 65.9% of households were contacted (Table 1). This household contact rate varied by ethnic-language group, and was lowest for non-Latino whites (58.0%) and African Americans (61.1%) and highest for Spanish-speaking Latinos (79.0%). Once a household was reached, the rate of contact with the sampled patient was roughly similar across groups, ranging from 91.3% to 95.0%. Thus, a total of 39% of potential respondents was lost, 34% due to an inability to establish household contact, and an additional 5% due to an inability to contact the specified patient.
Table 1. Contact Rates Among Adult General Medicine Patients by Ethnic-Language Group
The denominator for the percent is the number of potential respondents remaining in the preceding step of recruitment for that ethnic-language group.
Invitation to participate
(contact letter mailed)
Establish contact with household
Establish contact with individual
With the time of the survey ranging from 5 to 21 months after the last known clinic visit, the proportion of the total sample lost due to an incorrect telephone number ranged from 13.6% for Spanish-speaking Latinos to 21.2% among African Americans (Table 2). Of those with presumably correct telephone numbers, whites and African Americans had substantially higher proportions of those never reached by telephone (29.2% and 23.0%) than English-speaking and Spanish-speaking Latinos (12.3% and 11.5%). As a proportion of the letters mailed, taking all sources of loss into account, response rates ranged from a low of 30.5% for non-Latino whites to a high of 48.8% for Spanish-speaking Latinos. As a proportion of identified eligibles, response rates exceeded 66% in all groups and reached 70% in the total sample.
Table 2. Final Disposition and Response Rates Among Adult General Medicine Patients by Ethnic-Language Group
English- speaking Latino
Spanish- speaking Latino
Contact letter mailed
Confirmed incorrect telephone number (includes disconnected)
Individual never reached (busy, no answer, answering machine, or not available)
Ineligible, too ill, or deceased
Proportion of identified
Women, those aged 40 to 74, those who were not privately insured, and Latinos were more likely to respond (Table 3). In multivariate analyses, women (OR, 1.42; 95% CI, 1.24 to 1.62), those aged 40 to 49 years (OR, 1.75; 95% CI, 1.34 to 2.28), aged 50 to 64 years (OR, 1.93; 95% CI, 1.49 to 2.50), and aged 65 to 74 years (OR, 1.60; 95% CI, 1.25 to 2.04), English-speaking Latinos (OR, 1.59; 95% CI, 1.33 to 1.91), and Spanish-speaking Latinos (OR, 2.10; 95% CI, 1.76 to 2.50) were more likely to complete the survey. Compared to those with no insurance, those who had private insurance were less likely to complete the survey (OR, 0.83; 95% CI, 0.71 to 0.97).
Table 3. Odds Ratio for Having a Completed Survey by Demographic Characteristics
Although the proportion that completed the survey was slightly higher among those receiving the tailored letter, there were no statistically significant differences by letter status within any of the nonwhite groups nor for the entire nonwhite sample (Table 4). In the total sample of minority patients, 39.8% of those receiving the general letter completed the survey compared to 41.5% of those receiving the tailored letter. Using the total number within each ethnic-language group receiving the same type of letter as the denominator, response rates were very similar for those receiving the general or tailored letter among African Americans (33.5% vs 35.5%), English-speaking Latinos (41.2% vs 41.4%), and Spanish-speaking Latinos (47.4% vs 50.2%). Response rates using identified eligibles as the denominator also did not differ by type of letter received.
Table 4. Completed Surveys Among African-American, and English-speaking and Spanish-speaking Latino Adult General Medicine Patients by Type of Letter Received
Differences in percentages between general and ethnically tailored letters were not statistically significant based on χ2 statistics (all P values>.3).
Response Rate (%)
Response Rate (%)
Response Rate (%)
Response Rate (%)
This is one of a few studies to report recruitment results by stage and ethnicity. Examining recruitment at various stages revealed interesting ethnic differences. We lost 34% of the potential sample due to an inability to contact the household and another 5% because we were unable to reach the targeted individual. This could be due to the lack of clinic procedures to regularly update patient contact information or the higher residential mobility of particular ethnic groups. Data from a national survey of Medicare managed care beneficiaries enrolled for at least 12 months suggested that incorrect contact information was more problematic among Latinos and African Americans than whites,14 while in our study this was true only for African Americans. In our study, once household contact was established, the ability to reach the sampled patient was similar across groups. However, non-Latino whites and African Americans were less likely to agree to complete the survey once contacted.
Latinos, especially those who were Spanish speaking, were most likely to complete the survey once contacted. The more intensive follow-up of this group may explain their higher contact rates; the use of bilingual materials and interviewers may also have minimized refusals. Higher response rates among Latinos may also be explained by less frequent invitations to participate in research compared to African Americans and whites, or by a cultural script named simpatía characterized by an interpersonal style that seeks to avoid conflict.15
Studies similar to ours have reported overall response rates, not rates within race or ethnic strata. Our overall response rate among identified eligibles of 70% is comparable to those obtained in recent surveys of Medicare managed care enrollees (75% to 80%)14 and the Behavioral Risk Factor Surveillance System (57% to 71%).16 Data on race-specific response rates among eligible individuals could be found for only a few studies, but these differed considerably from ours in study design. A case-control study that used population- and community-based methods to recruit controls for a lung cancer study requiring a home interview and blood specimen obtained response rates among eligible African Americans between 19%–82% and 25%–78% for eligible Latinos depending on the recruitment method.17 Another study requiring a one-time clinic visit obtained response rates among eligible Latinas of 79% and 55% using media and clinic registry recruitment strategies, respectively.18 Heterogeneity with respect to type of study and recruitment methods make generalizations difficult.
In our study, non-Latino whites had the lowest rates of household contact and of survey completion once contacted and eligible, as opposed to ethnic groups typically considered “hard-to-reach.” The attention paid to recruitment appears to have increased minority response rates to the degree that they approximated or exceeded that of whites. It is also possible that the research topic, physician-patient interactions, could have been especially salient among nonwhite patients, regardless of the reference to their specific ethnic group. Alternatively, these findings could be due to the receipt of more frequent invitations to participate, ethnic differences in household patterns related to responding to telephone calls, or other factors.
This study did not explore individual and family telephone and mail practices, attitudes about research and the study topic, and previous research experiences, which may have led to differences in response rates. Another study limitation is that Asians were not included in the study sample.
Reporting recruitment results by population subgroups is a necessary first step in addressing the limited access of diverse ethnic groups to research. Yet, even by this minimum reporting standard, many prevention and treatment trials are failing.19,20 For example, a review of 205 U.S. Phase III randomized cancer treatments trials found that none reported whether they had used race or ethnicity as a selection criterion. Similarly, a review of 47 randomized trials of statin drugs in adults found that only 8 (17%) reported the ethnic composition of their samples.19 With very few exceptions, even for major studies that include diverse ethnic groups, most report only enrollment overall, or at the most, their final enrollment by ethnicity. Surveillance and reporting of recruitment by stage and ethnicity (including response rates as a proportion of eligibles by ethnic group) provide important information that allows investigators to predict where in the recruitment process and among which subgroups losses are more likely to occur.
This study highlights the usefulness of nesting research on recruitment methods within larger studies. Doing so allows us to maximize resources and achieve methodological advances in the science of recruitment. Systematic assessment of the effectiveness of recruitment and retention methods will contribute to the elimination of disparities in research participation, and improve the validity of research results for an increasingly diverse population.2,9,10,21 Thus, further attention to the development and testing of recruitment and retention methods for use in underrepresented groups is warranted. Addressing social and ethnic disparities in health and health care will require greater sensitivity and resources to include diverse groups in research, establish the effectiveness of treatments, and identify successful interventions for subpopulations.
This research was supported by Agency for Healthcare Research and Quality grant R01 HS10599 and the Resource Center for Minority Aging Research program of the National Institute on Aging, the National Institute of Nursing Research, and the Office of Research on Minority Health, grant P30 AG15272.