Supported by the National Institute on Drug Abuse, RO1 Grant DA14343.
Tracking Inner City Substance Users from the Emergency Department: How Many Contacts Does It Take?
Article first published online: 7 FEB 2008
© 2008 by the Society for Academic Emergency Medicine
Academic Emergency Medicine
Volume 15, Issue 2, pages 136–143, February 2008
How to Cite
Cunningham, R., Walton, M. A., Tripathi, S. P., Outman, R., Murray, R. and Booth, B. M. (2008), Tracking Inner City Substance Users from the Emergency Department: How Many Contacts Does It Take?. Academic Emergency Medicine, 15: 136–143. doi: 10.1111/j.1553-2712.2008.00033.x
- Issue published online: 7 FEB 2008
- Article first published online: 7 FEB 2008
- Received July 12, 2007; revision received September 25, 2007; accepted September 25, 2007.
- emergency department;
- research methodology;
Background: Longitudinal studies of substance users report difficulty in locating and completing 12-month interviews, which may compromise study validity.
Objectives: This study examined rates and predictors of contact difficulty and in-person follow-up completion among patients presenting with cocaine-related chest pain to an inner-city emergency department (ED). The authors hypothesize that less staff effort in contacting patients and lower follow-up rates would bias subsequent substance use analysis by missing those with heavier substance misuse.
Methods: A total of 219 patients aged 19 to 60 years (65% males; 78% African American) with cocaine-related chest pain were interviewed in the ED and then in person at 3, 6, and 12 months. Demographics, substance use measures, and amount/type of research staff contacts (telephone, letters, home visits, and locating patient during return ED visits) were recorded. Poisson and negative binomial regression analyses were conducted to predict quantity of patient contacts for the 12-month follow-up.
Results: Interview completion rates at 3, 6, and 12 months were 78, 82, and 80%, respectively. Average contact attempts to obtain each interview were 10 at 3 months (range 3–44), 8 at 6 months (1–31), and 8 at 12 months (1–49); 13% of patients required a home visit to complete the 12-month interview. Participants requiring more contact attempts by staff were younger and reported more frequent binge drinking at baseline (p < 0.05), but were less likely to meet criteria for substance abuse or dependence (p < 0.5), or to report prior mental health treatment (p < 0.05). Comparisons of parallel regressions predicting contact difficulty based on the entire sample, the low-effort group, and the difficult-to-reach group showed variation in findings.
Conclusions: This study demonstrates that substantial staff effort is required to achieve adequate retention over 12 months of patients with substance misuse. Without these extensive efforts at follow-up, longitudinal analyses may be biased.
Aprimary challenge facing researchers conducting longitudinal studies with emergency department (ED) patients is the difficulty encountered in locating patients for follow-up assessments.1,2 While many patient populations are difficult to track longitudinally, patients with unstable employment, housing, and medical care present challenges frequently encountered by ED researchers. This is particularly true when conducting investigations with substance abuse populations, because the chaotic and sometimes transient lifestyle of these individuals can impede the researchers’ ability to maintain contact over an extended period of time.3 Although statistical approaches to compensate for random attrition are available,4–6 and have been described elsewhere for ED populations,2 the most desirable approach is to reduce attrition by implementing effective patient tracking and retention techniques.7,8
Follow-up rates from ED studies of patients with substance misuse vary greatly by study with attrition rates ranging from less than 10% to more than 50%.7–13 Low follow-up rates threaten study validity. The need to identify effective tracking methods arises from the knowledge that incomplete data can compromise the internal and external validity of a study, limiting the generalizability of study findings because those who were not assessed may have differed on the independent and dependent variables, thereby affecting the results and subsequent conclusions.14,15 Because some attrition is inevitable in longitudinal follow-up studies, understanding individual and contextual characteristics that predict contact difficulty (defined as either the amount of time or the number of attempts required to locate a participant and complete a follow-up assessment) has tremendous utility for planning longitudinal studies. Further, examination of contact difficulty provides insight into the problem with final analysis and the possible bias that attrition might cause in the interpretation of study findings.
This study adds to the ED literature by examining rates and predictors of contact difficulty and in-person follow-up completion among adult patients presenting with recent cocaine use to an inner-city ED. Although the population described is one of substance users, the concepts of longitudinal tracking of typically difficult-to-reach (DTR) ED patients in the inner city may have a broader utility. We also sought to determine whether less staff availability or effort in contacting patients and subsequent lower follow-up rates would bias findings by missing those with greater substance use during the 12-month follow-up. Specifically, demographic and substance use characteristics among those patients who were easier to contact and follow-up were compared to those who were more difficult to contact. In addition, we describe the contact efforts used by this project, so that recommendations can be provided to assist other investigators in planning longitudinal studies in the ED with transient or DTR populations.
This natural history study screened a consecutive cohort of patients presenting to an ED with recent cocaine use and the chief complaint of chest pain to participate in a longitudinal study. All research procedures were approved by the investigators’ institutional review boards (IRBs).
Study Setting and Population
This study was conducted at a Level 1 inner-city ED (see Booth et al.16 and Cunningham et al.17 for a more detailed research protocol description). This ED is located in a socioeconomically depressed region with high levels of poverty (ranking in the bottom 15 of 331 Metropolitan areas in the United States in unemployment rate).18 Standard of care at the study hospital requires that chest pain patients, aged 60 years and under, undergo urine screening for cocaine metabolites with their acute coronary syndrome workup.
Sample Recruitment. Research staff recruited patients in the ED between the hours of 8 am and 10 pm, 7 days per week, during the 2½-year recruitment period, from June 2002 through February 2005. Patients who came in after 10 pm were recruited in the observation unit by research staff each morning (patients were not routinely discharged from the chest pain observation unit before 8 am due to nurse and physician staffing). Consecutive patients, aged 18 to 60 years, were approached by research staff to participate in the screening. Patients who were not approached because the staff was busy with another participant, or who left before 8 am, were recorded as “missed” by research staff. Thirty-nine patients over the study period (2%) were missed. For the screening, research staff obtained written informed consent to view patient medical records to determine study eligibility, for which participants could choose a $1 gift. Eligibility for Phase II of the study included ages of 18 to 60 years; positive toxicologic urine screen for cocaine, or if urine screen results were incomplete or unavailable, physician documentation of the patient’s cocaine use. Patients were ineligible if they were pregnant, unable to provide informed consent (e.g., unconscious, incarcerated), or acutely suicidal or homicidal (requiring physical restraints or security monitoring during the ED visit). After signing a written consent form, participants in Phase II completed a 2-hour baseline interview during their visit (see “Measures”) and received a $25 gift card (e.g., Wal-Mart, Target).
Information gathered at baseline and during contact efforts was used for retaining and tracking participants in the research study over the 12-month follow-up period.
Tracking Information Gathered. At baseline, participants were asked to provide information that would allow study personnel to contact them for follow-up interviews to be conducted at 3, 6, and 12 months postbaseline. Specific information collected included date of birth, social security number, telephone numbers (work, cellular, and home), and living and mailing address. Study personnel also gathered the names, telephone numbers, and addresses of at least two contact persons (e.g., spouse, family, and friend) who would know the patient’s whereabouts over the next year. Participants also were asked to provide names of other individuals, including case workers, physicians, and social service agency workers, as well as locations of places where they often frequented (e.g., churches, shelters). Before discharge, participants were given an index card containing the project logo (CPR), baseline interviewers’ names, project telephone number, and a toll-free number that they could use to contact study personnel, as well as the interview dates and payment information for each follow-up interview. A review of the participant’s medical record, which included identifying information, assisted in confirming information provided by the participant.
During subsequent contacts with a participant, locator information was verified or updated. For participants who were not easily located by staff, hospital medical records were searched in addition to public databases, such as Department of Public Health death records, Internet people finder databases (e.g., Alumnifinder, Yahoo people search), and offender and prison Web sites.
Contact Efforts. Participants were called within 2 weeks after their baseline assessment. Contact information was confirmed and participants were reminded of the 3-month follow-up date. Approximately 4 weeks before each scheduled follow-up date, a letter was sent to participants informing them of their upcoming interview and interview payment. “Forwarding address correction requested” was stamped on the envelope so that the post office would notify the project if a letter had been forwarded to a new address. Cases were assigned to interviewers 2 weeks before scheduled due dates. Interviewers were expected to locate and interview participants within 2 weeks after the scheduled interview date. Once assigned a case, interviewers would call the participant’s contact numbers, typically between 8 am and 10 pm, 7 days per week, until they scheduled the interview, without harassing participants. Typically, interviewers would call throughout the day, but leave no more than one message per day. If an interviewer was not sure if a person lived at an address, a letter was sent by certified mail, and the return receipt was requested. In compliance with IRB requirements, if at any time participant asked not to be contacted, they were thanked for their participation and no further contact efforts made. When possible, scheduled participants were sent a reminder letter and called the night before the interview. If a participant failed to appear at the appointed time, they were called again and, if needed, sent a missed appointment letter.
For participants who did not have a scheduled interview 1 week before their due date, a second letter was sent and the assigned interviewer started to contact the designated participant’s contacts. A home visit was scheduled on occasions when an interview was not scheduled by the due date. Home visits would also occur for participants who did not respond to the initial letter and had no known phone number. A letter informing the participant of the home visit was sent out 1 week before the scheduled visit. These home visits were conducted with two interviewers (for safety) and took place during daylight hours. If the participant was not home, interviewers left friendly, handwritten notes on index cards, similar to the ones given to the participant at baseline. During visits to the participant’s residence, study personnel would attempt to contact neighbors to confirm if the participant resided at that address or if they knew a more current address. During winter months, letters (without revealing that study was related to substance use) were left at local shelters where participants were known to have previously stayed.
Follow-up Interview Protocols and Staff Motivation. At all times, confidentiality was assured with mail, messages on machines, and conversations with significant others. All project related stationery and business cards used a generic project name that was not related to substance use (CPR, Chest Pain Risk). During the consent process, participants were ensured of the confidentiality of their responses and information and consented to follow-up interviews. When leaving messages, the project name was used, but no mention of substance use or inclusion criteria was made or would be revealed. When so instructed by the participant, study personnel would not contact a person (e.g., spouse) even if previously permitted to do so.
A research assistant who lived in the local community where the majority of the subjects resided tracked the participants and conducted follow-up interviews. Participants were asked to come to the study office located at the participating medical center where the baseline interview occurred; most participants were comfortable returning to a known environment. If needed, transportation by taxi was provided, paid for by the project. Some participants preferred to meet at a convenient public location such as a fast-food restaurant. Finally, when these efforts were exhausted, participants were interviewed in their homes.
Remuneration methods used to enhance compliance with follow-up interviews included remuneration for time spent of $30 gift card for the 3-month interview, a $35 gift card for the 6-month interview, and a $45 gift card for the 12-month interview. During follow-up, participants were asked to provide a voluntary urine specimen, for which they were provided an extra remuneration of a $10 gift card. Participants were provided with a toll-free phone number to contact study offices and were given an incentive of $5 per interview if they telephoned the study office within 2 weeks of their scheduled interview date or agreed to have their follow-up interview take place at the medical center.
One problem frequently encountered by follow-up staff was decreasing motivation and increasing frustration over months of attempting to contact DTR participants. Several procedures were used to provide a team atmosphere that reduced interviewer burnout. First, interviewers were paid hourly for interviews and travel time and were reimbursed for mileage. Interviewers helped each other troubleshoot difficult cases and brainstorm case-finding alternative strategies at weekly 1-hour meetings. During these meetings, all active cases were discussed and interviewers were praised for their efforts, especially for the completion of interviews. Although interviewers were discouraged from trading cases, they frequently assisted each other in the location efforts.
The validity of self-report was enhanced via the use of trained interviewers who assured confidentiality and privacy during research interviews;19 in addition, urine drug screens were obtained.19–21 The Substance Abuse Outcomes Module (SAOM)22 is designed for evaluation of substance abuse treatment outcomes (http://www.netoutcomes.net/Netoutcomes/) and measures Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), substance use disorders23 and outcome domains. For the current study, the SAOM was used to measure basic demographic information including age, gender, race, marital status, education, and employment status, as well as information on lifetime and recent (past 3 months) substance use diagnoses, substance abuse, and mental health treatment history.24 The SAOM has undergone extensive reliability and validity examinations and demonstrates reasonable reliability (internal reliability, coefficient α = 0.58–0.90, test–retest reliability 0.56–0.99) and validity (concurrent validity generally 0.5–0.8, predictive validity 0.5–0.9).25 Concurrent validity for the SAOM was based on longer key instruments, such as a structured diagnostic interview for substance use disorders, the Composite International Diagnostic Interview-Substance (CIDI-SAM),26 and the Addiction Severity Index.27 The SAOM has shown a 90%–93% agreement with the CIDI-SAM on DSM-IV substance use diagnosis (present or absent).25 Self-reported use of substance abuse treatment services included lifetime and past-year use of formal specialty treatment services and/or informal services (e.g., self-help groups), as well as the past-year report of any mental health–related treatment or treatment for depression.
Tracking Measures. Correspondence and tracking efforts were recorded in contact logs and listed as one of five categories: 1) interviewer telephone calls—telephone calls made by any research staff member, including to a participant, a participant contact, and other locations (e.g., shelters, work [These calls often were made in the evening up to 10 pm.]); 2) participant-initiated calls—telephone calls received by staff members, including calls from the participant and participant contacts; 3) letters—letters sent by research staff, including initial contact letters, home visit letters, letters to shelters, and missed appointment letters; 4) return ED visits—any return visits to the ED of the medical center identified by interviewers where contact was made with participants; and 5) home visits—home visits made by research staff.
Descriptive information regarding interview completion rates, number of contact attempts, and types of contact attempts were computed. Logistic regression analyses were used to identify potential differences between study completers versus noncompleters. For analyses, the number of each of the five types of contact efforts was summed into a total count. For analysis, the total range (1–49) and frequency of contact attempts to complete the 12-month interview were examined and divided into approximately thirds. Those participants who staff reached with the least effort (LE; approximately two-thirds of all participants were reached with seven or fewer contacts, total) were deemed “least effort” (n = 145; 68%). Participants who required eight or more contact attempts to complete an interview were deemed “difficult to reach” (n = 74; 32%). Poisson and negative binomial regression analyses were conducted to predict number of 12-month contacts required, based on baseline characteristics (age, gender, race, marital status, hourly wage) and problem severity indicators of substance use from the SAOM that have been demonstrated to impact follow-up difficulty in prior studies,13,19,20 such as days of cocaine and binge drinking in the prior 4 weeks, substance abuse, and/or dependence diagnoses. In addition, lifetime substance use treatment and mental health treatment were included, because it was hypothesized that patients who were more familiar with detailed mental health and substance use questions and appointments may be more comfortable returning to complete a follow-up on these topics. Finally, negative binomial regression analyses were conducted to predict the number of 12-month contacts required for three separate groups (LE, DTR, and LE + DTR), based on baseline characteristics.
Among the 302 eligible individuals presenting to the ED during the recruitment period with recent cocaine use and chest pain, 73% agreed to participate (n = 219), 19% refused to participate, and 8% were missed by recruitment staff (e.g., occupied with other subjects, patient discharged before research staff evaluation). Demographic and psychosocial characteristics of the study group are presented in Table 1. All participants (via eligibility criteria) had recent cocaine use, and 48% met criteria for cocaine dependence. Sixty percent of the sample graduated from high school, 72% reported income of less than $20,000 per year, and 16% were married or living with a partner. Demographic characteristics of baseline participants (n = 219) and nonparticipants (n = 89) were essentially similar.
|Characteristics||n or Mean (SD)||%|
|Male||n = 142||65|
|Age, mean (SD)||38.6 (8.93)||—|
|African American||n = 171||78|
|Never married/single||n = 170||78|
|High school educated (yes)||n = 132||60|
|Currently employed (yes)||n = 90||41|
|<$10,000||n = 61||35|
|$10,000–19,999||n = 65||37|
|>$20,000||n = 50||28|
|Substance use diagnoses past 12 months|
|Any abuse/dependence diagnosis (yes)||n = 142||65|
|Cocaine abuse/dependence (yes)||n = 106||48|
|Other drug abuse/dependence (yes)||n = 13||6|
|Substance use frequency (past 4 weeks)|
|Days binge drinking||6.0 (9.46)||—|
|Days using marijuana||7.1 (9.86)||—|
|Days using crack/cocaine||7.51 (8.50)||—|
|Social support score||73.06 (20.85)||—|
|Psychological distress||71.9 (26.43)||—|
|Depression symptom severity||10.5 (7.92)||—|
|Lifetime mental health treatment (yes)||n = 84||38|
|Lifetime substance use treatment (yes)||n = 120||55|
Interview Completion Rates and Description
Interview completion rates were 77, 82, and 80% at 3, 6, and 12 months, respectively (see Table 2). Study protocol included urine drug screens at all follow-up interviews and more than 90% of participants completed urine drug screen (91, 91, and 94% at each follow-up interview, respectively). Contact attempts varied by type and effort including telephone, letters, in-person ED return visits, and in-person home visits; with 12% of the 3-month follow-up interviews requiring a home visit for completion. Figure 1 describes the number of contact attempts per participant by type of contact. Contact efforts were summed to create a total count of contacts. Participants who staff reached with the LE (two-thirds of all participants were reached with between one and seven contacts) were deemed LE. In contrast, DTR participants (eight or more contact attempts total) required approximately three times the interviewer calls and two times the number of home visits to successfully complete follow-up interviews (Figure 1). For those participants who did not complete a 3-month interview (n = 45), 53% (n = 24) did go on to complete a 6- or 12-month interview. However, in comparison, of those that did complete a 3-month interview (n = 170), 96% went on to complete a 6- or 12-month interview.
|Time of Follow-up (Months)||Completed Interview (%)||Home Visits (%)||Average Contact Attempts at Each Interview (Range)|
Comparisons of Study Dropouts with Participants Completing 12-month Follow-up
Logistic regression analyses comparing those who completed the 12-month follow-up (n = 174) with noncompleters (n = 38) on baseline factors (i.e., gender, age, hourly wage, marriage status [yes/no], race [African American/other], lifetime substance use treatment [yes/no], mental health treatment [yes/no], substance abuse dependence [yes/no], binge drinking, and cocaine use days) was significant. Seven participants were excluded from this analysis because they had died (n = 1) or were in jail (n = 6). Further examination of individual variable contribution showed that only two variables were significant: men (Wald statistic 7.8; p < 0.01) and those with greater hourly wages (Wald statistic 12.1; p < 0.01) were less likely to complete the 12-month interview.
Predicting Contact Difficulty at 12-month Follow-up
Separate Poisson and negative binomial regression analyses for the three groups (DTR, LE, and DTR + LE) were conducted to predict contact difficulty at the 12-month interview. While negative binomial regression was a better model for DTR + LE and only DTR samples, Poisson regression model was used for LE sample. Table 3 shows that in the LE sample, age and cocaine use were significant predictors, age and number of days cocaine used in the past 4 weeks were significant predictors, and the incidence rate ratio for a one-unit increase in age and cocaine use days decreased by exp(–0.014) = 0.986. Thus, younger participants and those who have fewer days of cocaine use required more contacts to get in touch with. In the DTR sample, age, hourly wage, and substance abuse or dependence diagnosis were significant predictors; younger participants and those who earn more hourly required more contacts to reach, while those who were diagnosed with substance abuse or dependence past 12 months required fewer contacts to reach. In the LE + DTR sample, age, binge drinking, substance abuse/dependence diagnosis, and mental health treatment were significant predictors; younger participants and participants with higher numbers of binge drinking days required more contacts and hence are DTR, while those participants who were diagnosed with substance abuse or dependence in the past 12 months and participants who reported past mental health treatment required fewer contacts to reach.
|Variable||LE Sample* (n = 145)||DTR Sample† (n = 74)||LE + DTR Sample† (n = 219)|
|Age||−0.014 (0.986)‡||−0.012 (0.988)§||−0.016 (0.984)‡|
|Gender (male)||0.006 (1.006)||0.110 (1.12)||0.075 (1.08)|
|Race (African American)||0.125 (1.13)||−0.104 (0.901)||−0.126 (0.88)|
|Marital status (single/never married)||0.195 (1.22)||0.011 (1.01)||0.033 (1.03)|
|Hourly wage||0.001 (1.00)||0.004 (1.004)¶||0.002 (1.002)|
|Days of cocaine use past 4 weeks||−0.014 (0.986)§||0.003 (1.003)||0.007 (1.007)|
|Binge drinking days past 4 weeks||0.006 (1.006)||0.007 (1.007)||0.013 (1.01)§|
|Any substance abuse/dependence diagnoses past 12 months (yes)||−0.071 (0.931)||−0.247 (0.781)§||−0.232 (0.793)§|
|Lifetime substance use treatment (yes)||0.184 (1.20)||0.130 (1.14)||−0.006 (0.994)|
|Mental health treatment (yes)||−0.110 (0.90)||0.101 (1.106)||−0.230 (0.794)§|
Substantial effort is needed to reinterview patients with substance use who are enrolled in inner-city ED-based studies. A review of the medical literature found a paucity of information regarding the efforts needed to retain study participants from nonmental health medical settings. Understanding the factors and resources needed to retain patients in longitudinal research from the ED may aid in the successful planning and execution of ED-based research projects. Although the extent of contact efforts needed to complete follow-up interviews in this study varied considerably, they tended to be greater at the 3-month follow-up than at either the 6- or the 12-month follow-up, demonstrating the importance of retaining participants at the initial follow-up data point. This finding is consistent with previous studies that have also found attrition rates to be highest at the first follow-up.28,29 Without the resources to make these extensive contact attempts, longitudinal analyses may be biased in directions that are not necessarily predictable. In our study, predictors of contact difficulty varied based on how the sample was divided (LE, DTR, or total).
More specifically, when examining the total sample (LE + DTR), ED participants requiring more contact attempts over the year were younger, engaged in more episodes of binge drinking, were less likely to have a substance abuse or dependence diagnosis in the past 12 months, and reported less mental health treatment than did participants who required fewer contact attempts. Those patients who are younger may be less established or settled in the community. In fact, in the LE and DTR subgroups, age consistently predicted contact difficulty. It may be that those with no mental health treatment in the past are less comfortable completing subsequent questionnaires on sensitive topics such as depression and substance use, or that those with prior mental health treatment, are better functioning psychosocially than those substance users who have not sought mental health treatment, and thus have more stable lives and are easier to find by staff. Our finding that patients requiring more contact attempts were significantly more likely to have engaged in binge drinking is in line with several previous studies in patients undergoing outpatient substance use treatment,29,30 but has not been demonstrated in patients in ED-based studies. Surprisingly, it was more difficult to interview participants at follow-up who did not meet substance abuse or dependence criteria; it may be that these participants are more highly functioning and employed, such that it may be more difficult to schedule interviews into their busy lives. Note that findings for models examining the DTR group found they made more money and were less likely to meet substance abuse or dependence criteria. A prior ED-based evaluation by Woolard et al.2 found that demographic characteristics (including gender and socioeconomic status) did not predict dropouts or those who were difficult to contact. Neuner et al.,13 in an ED study evaluating contact difficulty among primarily white patients in Germany with alcohol misuse, found that individuals with a greater number of alcohol-related problems required more contact attempts to complete follow-up interviews. Although the pattern clearly differs across studies, when taken together there are clear differences in the characteristics of those patients who are DTR, but not in predictable directions, suggesting the importance of tenacious tracking efforts.
In the meantime, as Figure 1 demonstrates, considerable staff resources and associated budgetary allotment are necessary to follow both the DTR patients and the 20% who were never located for follow-up. Following ED patients has different challenges than tracking patients enrolled in an outpatient clinic or substance abuse treatment setting, where they have repeat scheduled visits and have an established relationship with a provider or therapist. These challenges can be exacerbated in the inner-city ED where financial, insurance, and educational barriers to continuity of outpatient care often exceed those in more affluent settings. In addition, following patients with substance misuse or abuse has additional challenges. For instance, Scott31 notes that understanding the norms of the substance abuse population (e.g., frequent changes in living environment, transitory nature of relationships with many collateral contacts) is a necessary component in reducing contact difficulty. Contact difficulty is affected by the extent of location information gathered at baseline, financial constraints that limit the amount of time spent locating participants, failure to ensure participants’ confidentiality and establish rapport, assessments occurring at inconvenient times or locations, or lack of adequate reimbursements.2,31,32
Longitudinal follow-up of any population of ED patients presents challenges that are best identified early and planned for, most specifically in the grant planning stages, by including an adequate follow-up budget. We have found that estimating 5 hours of research staff time, per follow-up per subject, provides a rough estimate to ensure that there is adequate budget to complete the follow-up phase of a study protocol with urban ED patients. Although this initially may seem excessive when viewed in light of the budget portion allotted to intervention development and recruitment, the benefit justifies the cost, especially when considering the difficulty of analyzing data with poor follow-up rates. In addition, our observations and experience finds that the personal characteristics of research staff who are successful at tracking patients for follow-up interviews tend to differ from those who are the most successful at recruiting patients in the ED setting. Successful follow-up staff often are local residents, know the community well, are willing to go to participant’s houses and shelters, and view each patient follow-up in a manner similar to detective work. These skills and attributes often differ from the recruiting staff who may or may not be from the immediate area, are very comfortable in a medical setting with ill patients and medical staff, and who need to be able to comfortably approach ED patients and succeed at obtaining consent for study enrollment. Owing to these differing skills, in general the follow-up staff is hired separately from the recruiting staff and, while maintaining a spirit of teamwork with occasional overlap as practical logistics demand, for the most part their tasks are separate from that of recruiters.
Although it remains controversial how much attrition is acceptable without biasing a study, current standards in other settings suggest that the retention rate should be at least 80%. Several recent studies have challenged the view that retention rates of 70% may be acceptable, citing concerns that a 30% attrition rate has the potential to engender bias effects that are as large as the treatment effects under review (e.g., Foster and Bickman,33 Hedeker et al.,34 and Scott31). Scott31 ranked participants in two substance use studies according to the number of contacts required to complete an interview. When comparing the first 70% of participants (who therefore needed fewer contacts for completion) and the remaining 30% who required the most contacts, the data suggested that including only 70% led to biases that were severe enough to compromise the internal validity of the studies. This may suggest that follow-up rates closer to 80% than 70% are required to maintain study validity. Our findings mirror these conclusions in that the pattern of predictors of contact difficulty varied.
All data are based on self-report. However, the use of standardized measures, assurance of confidentiality (including a National Institute on Drug Abuse (NIDA) certificate of confidentiality), inclusion of urine drug screens, use of research staff, and lack of consequences for reports have been shown to increase the validity of data regarding both substance use and involvement with illegal activities.19,35 Data from this study are from an inner-city ED and may not generalize to other more affluent or suburban populations of substance users.
Our study examined rates and predictors of contact difficulty and in-person follow-up completion among substance users in an inner-city ED. The contact efforts used successfully by this project in a population that is typically difficult to follow may assist investigators in planning other longitudinal ED studies with substance users, as well as other DTR populations in urban settings. Substantial staff effort is required to successfully follow patients over 12 months. However, without these extensive efforts, longitudinal analyses may be biased. Although statistical approaches are available to correct biases,2 it is always preferable to prevent attrition through implementation of tracking efforts. These findings have implications for future ED-based longitudinal research.
The authors acknowledge Harvey Siegel for his contributions to the project, Pat Bergeron for her assistance with manuscript preparation, and Hurley Medical Center.
- 10An approach to the problem of recruitment and retention rates for longitudinal research. Behav Assess. 1987; 9:169–87., .
- 18Michigan Department of Community Health. Census 2000. 2002.
- 19Self-report issues in alcohol abuse: State of the art and future directions. Behav Assess. 1990; 12:91–106., .
- 23American Psychiatric Association. DSM-IV Draft Criteria. Washington, DC: American Psychiatric Press, 1993.
- 24The Drinker Inventory of Consequences (DrInc): An instrument for assessing adverse consequences of alcohol abuse. Project MATCH Monograph Series, Vol. 4. DHHS Publication No. 95–3911. Rockville, MD: National Institute on Alcohol Abuse and Alcoholism, 1995., , .
- 34Sample size estimation for longitudinal designs with attrition: comparing time-related contrasts between two groups. J Educ Behav Stat. 1999; 24(1):70–93., , .