How reminders given to patients and physicians affected pap smear use in a health maintenance organization

Results of a randomized controlled trial

Authors


  • Presented in part at the Society of General Internal Medicine, Washington, DC, April 1994.

Abstract

BACKGROUND

Despite its effectiveness as a method of controlling cervical carcinoma, the use of Pap smear testing remains incomplete, and its promotion in the primary care setting provides an important opportunity for intervention.

METHODS

The authors conducted a randomized controlled trial that involved three sites of a health maintenance organization (HMO) serving an urban minority population. Their aim was to evaluate the impact of reminders given to patients and physicians on site visitation by patients and Pap smear use. Eligible women (n = 5801) were randomly assigned to 1 of 4 intervention combinations (in which reminders were given to either the patient or the physician, to both, or to neither). If they were ineligible for patient reminder intervention, patients were randomized only to physician reminder intervention (the presence or absence of it). The letter of reminder mailed to the patient invited women due for Pap smears to visit the HMO site, and the reminder for physicians was a medical record notice that a Pap smear was due. Logistic and survival analyses were used to investigate the correlation of intervention status with visitation, interval of time to a visit, and Pap smear use.

RESULTS

In the primary intent-to-treat analysis, there was no significant effect of either patient or physician reminder interventions on rates of visitation or Pap smear completion. The secondary efficacy analyses demonstrated no overall effect of either patient or physician reminders, but effects among subgroups of women at individual HMO sites were noted. At Site 3, there was an apparent increase in time to the next visit among the subgroup of women with a chronic illness (16 weeks with intervention vs. 9 weeks without). With the physician reminder, the odds that a Pap smear would be given during the study year were increased among women without a previous Pap smear at Site 1 (adjusted odds ratio = 1.39) and those with a chronic illness at Site 2 (adjusted odds ratio = 3.38).

CONCLUSIONS

Reminders given to patients and physicians had a limited impact on visitation by patients to the HMO sites or Pap smear completion. Although some subgroups of women may benefit, the authors also observed a possibly unfavorable impact among other subgroups. These results emphasize the importance of identifying more effective interventions, targeting them to women most likely to benefit, and not overlooking the possibility that preventive intervention will have an unanticipated adverse effect. Cancer 1998;82:2391-2400. © 1998 American Cancer Society.

Although there is debate concerning the appropriate interval between Pap smear examinations, there is essentially universal endorsement of the procedure as the primary method of controlling cervical carcinoma.1 Nevertheless, it is estimated that in 1997 14,500 women in the U.S. will be diagnosed with invasive cervical carcinoma and that 4800 will die of the disease.2 Furthermore, rates of invasive cervical carcinoma and related deaths are not uniform in the U.S. population.3 Minority women and those of lower socioeconomic status remain at higher risk; this risk is related to multiple factors, including higher rates of disease incidence, more advanced stage disease at detection, and lower rates of stage specific relative survival.3-5 One factor that may contribute to these differences is variation in patterns of Pap smear testing. Recent studies have indicated that, although over two-thirds of women in the U.S. reported having undergone Pap smear testing within the preceding 3 years, women with lower incomes, lower levels of education, and without health insurance-particularly minority women with these characteristics-were less likely to report completion of a recent Pap smear.6-8 Efforts to improve further the control of cervical carcinoma should therefore be directed toward increasing Pap smear testing among women at risk for under-utilization.

The primary care setting provides an important opportunity to increase Pap smear use. Within primary care settings, multiple intervention strategies have been examined, and among the most effective are procedure reminders for providers.9-14 At three institutions serving lower-income African American residents of Detroit, we previously demonstrated the effectiveness of computerized physician reminders in increasing the use of mammography.15, 16 Mailings to enrolled patients reminding them to undergo a medical procedure can potentially act synergistically with reminders given to physicians, by prompting visits or influencing services requested at a visit.11, 17-23 We designed a randomized controlled trial to examine the individual and joint effectiveness of coordinated patient and physician Pap smear reminders at three practice sites of a single health maintenance organization (HMO) serving inner city residents of Detroit, among whom HMO coverage had eliminated copayment and deductible costs as barriers to these health services.

METHODS

The study design was a 1-year randomized trial to determine the joint and individual effectiveness of patient and physician reminders in promoting Pap smear use among women younger than 40 years who were enrolled in each of 3 sites of a large, multisite HMO in Detroit, Michigan. Two interventions were evaluated: a reminder for patients (a mailed letter) inviting women due for Pap smears to visit the HMO to obtain the procedure, and a reminder for physicians (a prompt placed in the medical record) encouraging performance of the procedure on visitors due for Pap smears. Randomization to the physician and patient reminder interventions were carried out separately because the delivery of the patient reminder intervention had to be staggered to avoid any potential overloading of the clinical service. The intervention trial was conducted during the period March 1993 to April 1994. Approval for the study was obtained from the Institutional Review Board of Wayne State University and the participating HMO.

Settings

The three study sites are separate practice locations of an HMO serving a population mostly composed of Detroit residents who are eligible for Medicaid. Two of these three sites had also previously participated in a related trial that examined the effect of patient and physician reminders on the use of mammography among women age 40 years or older (the study populations in the two trials did not overlap).16 During the study year, all 20 physicians providing primary care at the study sites participated in the project. The specialties represented include family medicine (2 physicians), internal medicine (9), and gynecology (9). At each of the study sites, gynecologists could perform a Pap smear at the time of a visit by an eligible woman. However, at the time of this study, time and facility restraints in the HMO precluded the direct provision of Pap smear examination by family physicians and internists; thus, their patients required referral to a gynecologist for Pap smear completion.

Sample Selection

Women were eligible for inclusion if they were ages 18-40 years and had visited the HMO during the year preceding the study year. Of 10,509 potentially eligible women, we excluded 4708 whose last known Pap smear results were abnormal or insufficient for cytologic diagnosis. These women were not considered candidates for screening, and existing HMO procedures targeted them for prompt diagnostic evaluation.

A two-stage randomization procedure was used to assign women first to physician reminder intervention and then to patient reminder intervention. At the beginning of the study period, the 5801 women who were eligible for participation were randomly assigned using a site specific, stratified randomization procedure to receive or not receive the physician reminder intervention (a medical record reminder). Strata were defined by age, previous Pap smear use, and number of HMO visits in the preceding year. To avoid overloading the clinics, a separate randomization was carried out to assign women to patient reminder intervention. Women who remained eligible were selected and randomized to patient reminder intervention on a weekly basis in groups of 156. Women were not eligible for patient reminder intervention if they had either discontinued their enrollment in the HMO (n = 1235) or had had a Pap smear (n = 393) after randomization for physician reminder intervention (because the patient reminder intervention would no longer be appropriate). By the end of the study year, 3848 (66%) of the 5801 women randomized to physician reminder intervention had been randomized to patient reminder intervention.

Description of the Intervention

The computer-based reminder system generated Pap smear reminders for both patients and physicians. The reminders were generated off-site. The patient reminder letter was mailed to patients, and the physician reminder was placed in medical records by the research team. Both the patient reminder and the physician reminder were triggered by the patient's Pap smear due date. The Pap smear due date was 1 year after the date of the last normal Pap smear, because this was the policy of the HMO. Women not known to have had a Pap smear previously were considered due on the first day of the study.

The patient reminder was a personalized letter signed by the medical director of the HMO rather than by the woman's personal physician (this was because the women visited multiple physicians). The letter provided a brief rationale concerning Pap smear testing and identified the woman as due for the procedure. A brochure from the National Cancer Institute with information about a pelvic examination and the Pap smear procedure was included with the letter. Nine percent of mailed reminders were undeliverable (10% for Site 1, 8% for Site 2, and 9% for Site 3).

The physician reminder was a brightly colored single-page reminder that included patient specific Pap smear information. This notice was generated and prominently placed at the front of the patient's medical chart approximately 2 months before her Pap smear due date. The physician reminder was removed from the chart by project personnel once documentation of a completed Pap smear was obtained; otherwise, it remained in the medical record until the end of the study year. Ninety-two percent of the medical record reminders inserted into charts were documented as being at the front of the chart and available to the physician at the time of a patient visit, based on successful retrieval of the physician reminder from the medical record by research staff at the conclusion of the study.

Evaluation and Analysis

Electronic administrative records were used to determine demographic and clinical characteristics. Chronic illness was coded as present if either diabetes or hypertension was noted. Paper copies of all Pap smear results were received from each study site for the baseline and study year, and these reports were used to classify the occurrence and results of Pap smear examination. Three outcome measures were assessed: 1) occurrence of a primary care visit during the study year, 2) time to the first primary care visit after the patient reminder intervention date, and 3) completion of a Pap smear during the study year. Time to the first primary care visit was treated as a censored observation when women had not visited within 13 months of the last day of intervention.

We defined one primary and two secondary evaluation objectives for this trial. The primary objective was to evaluate the overall effectiveness of the four combinations of patient and physician reminder interventions during the year of intervention (patient reminder, physician reminder, both, or neither). In this intent-to-treat analysis, we assessed rates of visitation by patients and Pap smear completion among all women eligible for assignment to both patient and physician reminder intervention (n = 3848). The two secondary evaluation objectives focused on the assessment of efficacy of the individual physician and patient reminder interventions. These analyses were limited to randomized women who could be exposed to the intervention intended for them. Therefore, the efficacy evaluation of the patient reminder intervention excluded the 140 randomized women who discontinued and never regained HMO coverage after patient reminder randomization, because letters could not be sent to them. The evaluation of the physician reminder intervention excluded the 2055 women who did not visit a primary care provider during the study year, because these women could not be exposed to a medical record-based reminder intervention. We also examined the efficacy of the patient and physician reminders among subgroups of women, as discussed below.

Statistical Methods

Differences between groups were identified using chi-square tests. Logistic regression analysis was used for the intent-to-treat analysis of the effects of joint patient and physician reminders.24 The model included two main categories, intervention group (patient reminder, physician reminder, both, or neither) and HMO site.

Analyses of the efficacy of the physician and patient reminder interventions were conducted by site to examine site specific differences. For the physician reminder, logistic regression analysis was used to examine the effect of intervention on Pap smear completion. The initial multivariate model included main effects for each patient characteristic and intervention group plus all two-way interaction terms between intervention group and patient characteristics. The Hosmer-Lemeshow test was used to examine lack of fit of each logistic regression model, and the final models included all main effects plus all significant two-way interactions. PROC LOGISTIC25 was used to fit all logistic regression models.

The efficacy of the patient reminder intervention was evaluated by examining the time to the first primary care visit after intervention. The Kaplan-Meier method26 and a multivariate accelerated failure time model were used.26 All independent variables and their interactions with a given intervention group were included in the initial model, and the final model excluded all interactions that were not significant. PROC LIFEREG in SAS was used to fit all accelerated failure time models.25

The sample size calculations were based on the assumption that, in the absence of any intervention, the baseline rate of Pap smear completion would be between 25% and 60%. Site specific sample sizes were designed to provide at least 80% power to identify a difference of 15% in the rate of visitation or Pap smear completion between at least 2 intervention groups. Given these assumptions, and with a significance level of 0.05, a sample of 160 women was required for each intervention group at each site.

RESULTS

Patient Characteristics

Characteristics of the study women eligible for assignment to the physician and patient reminder interventions at each site are presented in Table 1. The vast majority of study women (87%) were eligible for Medicaid coverage. Ninety-five percent were African American, based on review of the medical records of 527 women eligible for the study. The study women visited a primary care site frequently; however, despite their frequent visits, relatively few had been diagnosed with a chronic illness. Although eligibility requirements differed for the patient and physician reminder evaluations, characteristics of women included in each evaluation were similar, and there were no significant differences among women assigned to receive or not receive physician or patient reminder intervention at any study site (data not shown).

Table 1. Characteristics (%) of Women Eligible for Inclusion in the Patient Reminder, Combined, and Physician Reminder Evaluation Groups
 Patient reminder and combined groupsaPhysician reminder groups
Site 1Site 2Site 3Site 1Site 2Site 3
  • a

    The patient reminder evaluation excluded 140 women who were eligible for the combined evaluation.

  • b

    During the baseline period.

Total no. of patients110013021446105814301258
Age (yrs)      
  18-19877987
  20-24242624262827
  25-29272725272724
  30-34242328222226
  35-39171717171617
Insurance      
  Commercial6118101310
  Entitlement948992908790
Chronic illnessb      
  Hypertension9461057
  Diabetes mellitus322422
Sexually transmitted diseaseb      
  Yes293728314331
  No716372695769
Primary care visitsb      
  1432629402125
  2-3313635323435
  4+263835284441
Gynecology visitb      
  Yes705855726559
  No304245283541
Previous Pap smearb      
  Yes292532342937
  No717568667163

Combined Reminder Evaluation: The Effects of Intervention among the Four Patient-physician Intervention Groups in the Intent-to-Treat Analysis

Table 2 presents the primary care visit and Pap smear completion rates observed during the study year among women assigned to patient and physician intervention groups as well as the site-adjusted odds ratios (and 95% confidence intervals) describing the association of intervention group status with visitation to HMO sites by patients and Pap smear completion. Results are presented for the three study sites combined because there were no important differences among them. Approximately three-quarters of these women visited a primary care physician during the study year. In site-adjusted logistic regression analysis, there was no significant association of intervention as a main effect with visitation (χ2 = 5.18, df = 3, P = 0.159).

Table 2. Relationship of Intervention Group with Visitation to an HMO Site and Pap Smear Completion
 nEffect on visitationEffect on Pap smear completion
Visit (%)ORa95% CIPap smear (%)ORa95% CI
  • HMO: health maintenance organization; OR: odds ratio; CI: confidence interval; N/A: not available.

  • a

    Odds ratios are adjusted for site and intervention status.

Intervention group       
  Neither96475ReferenceN/A28ReferenceN/A
  Patient only964750.980.80-1.21291.070.88-1.30
  Physician only960771.070.87-1.32291.050.86-1.28
  Both patient and physician960791.230.99-1.52321.231.01-1.50

Approximately 30% of randomized women completed Pap smears during the study year, and the unadjusted Pap smear rates did not significantly differ among the four intervention groups (χ2 = 4.90, df = 3, P =0.179). Controlling for site, neither of the independent effects for the patient or physician reminders were significant. The effect of the combined patient and physician intervention was marginally significant (site-adjusted odds ratio = 1.23; 95% confidence interval = 1.01-1.50).

Patient Reminder Evaluation: The Effect of Patient Reminder Intervention on Time to a Visit

At Sites 1 and 2, the median time to the first postintervention visit was 17 weeks for women in either intervention group, and the effect of the patient letter intervention on time to visit was not significant (Site 1: χ2 = 0.78, 1 df, P = 0.38; Site 2: χ2 = 0.16, 1 df, P = 0.69). At Site 3 the effect of the patient reminder differed significantly between those women with and those without a chronic illness (P = 0.0124). Figure 1 (3K) shows the correlation of patient reminder intervention with time to the first postintervention visit for women assigned to receive or not receive the patient letter intervention, stratified by the presence or absence of a chronic illness. Among women without a chronic illness, the median time to a visit was 16 weeks; this did not differ between the two intervention groups (Fig. 1 (3K)). However, among women with a chronic illness, the median time to a visit was 9 weeks without the patient reminder intervention and 16 weeks with patient reminder intervention. This increase in time to next visit among women with a chronic illness who were assigned to receive the patient reminder intervention was statistically significant (adjusted coefficient = 0.77; 95% confidence interval = 0.13-1.41). Factors other than intervention status that were associated with a reduced time to the first postintervention primary care visit at one or more sites were younger age, commercial medical insurance, chronic illness, a previous history of a sexually transmitted disease, previous Pap smear, and a more recent visit in the year preceding the study (data not shown).

Figure 1.

Relationship of patient reminder intervention with time to first postintervention visit are shown for Site 3, according to whether patients received reminders and/or had a chronic illness.

Physician Reminder Evaluation: The Effect of the Physician Reminder on Pap Smear Use among Visitors

Table 3 presents the adjusted odds ratios describing the association of patient characteristics and physician reminder intervention with Pap smear completion. Overall, approximately 44% of women in the study who visited an HMO site during the study year had a Pap smear. There was no overall effect of the physician reminder as a main effect on Pap smear completion at any of the sites (46% vs. 44% for physician reminder vs. no physician reminder at Site 1, 46% vs. 44% at Site 2, and 44% versus 41% at Site 3). However, there was evidence of a physician reminder effect that varied among specific subgroups of visitors to Sites 1 and 2.

Table 3. Adjusted Odds Ratios (and 95% Confidence Intervals) Describing the Correlation of Physician and Other Patient Characteristics with Pap Smear Completion, by Site
VariablesSite 1Site 2Site 3
(n = 1058) OR (95% CI)(n = 1430) OR (95% CI)(n = 1258) OR (95% CI)
  • OR: odds ratios; CI: confidence interval; N/A: not applicable as a result of significant or nonsignificant interactions.

  • a

    During the baseline period.

Physician reminder   
  YesN/AN/A1.15  (0.91-1.45)
  (ref) NoN/AN/A1.00
  Chronic illness   
    YesN/A3.38  (1.32-8.63)N/A
    NoN/A1.02  (0.81-1.27)N/A
  Previous Pap smear   
    Absent1.39  (1.02-1.89)N/AN/A
    Normal0.69  (0.45-1.05)N/AN/A
Age (yrs)   
  18-241.24  (0.83-1.86)1.49  (1.05-2.10)1.30  (0.88-1.92)
  25-290.99  (0.66-1.50)1.35  (0.94-1.93)1.25  (0.83-1.87)
  30-341.14  (0.76-1.73)1.09  (0.75-1.57)1.06  (0.72-1.56)
  (ref) 35-391.001.001.00
Insurance   
  Commerical1.02  (0.65-1.58)1.02  (0.73-1.41)1.53  (1.03-2.26)
  (ref) Entitlement1.001.001.00
Primary care visitsa   
  (ref) 11.001.001.00
  2-30.89  (0.65-1.21)0.96  (0.70-1.31)0.94  (0.68-1.29)
  4+0.97  (0.69-1.36)1.13  (0.82-1.56)0.97  (0.69-1.35)
Gynecology visita   
  Yes1.35  (0.97-1.88)1.02  (0.78-1.33)1.57  (1.17-2.10)
  (ref) No1.001.001.00
Previous Pap smeara   
  (ref) AbsentN/A1.001.00
  NormalN/A1.36  (1.05-1.76)1.43  (1.08-1.88)
Chronic illnessa   
  Yes1.08  (0.71-1.63)N/A0.54  (0.33-0.90)
  (ref) No1.00N/A1.00
Sexually transmitted diseasea   
  Yes0.67  (0.50-0.89)1.19  (0.94-1.50)1.09  (0.83-1.42)
  (ref) No1.001.001.00
Patient reminder   
  Yes1.11  (0.84-1.47)1.05  (0.81-1.37)1.21  (0.94-1.56)
  (ref) No1.001.001.00
  Not randomized3.02  (2.10-4.34)3.29  (2.49-4.34)2.64  (1.84-3.79)

At Site 1, the effect of the physician reminder intervention was significant among women not known to have had a previous Pap smear (adjusted odds ratio = 1.39; 95% confidence interval = 1.02-1.89). By contrast, among women with a normal previous Pap smear, the odds of a study year Pap smear were reduced in the physician reminder intervention group, but this effect was not significant (adjusted odds ratio = 0.69; 95% confidence interval 0.45-1.05). At Site 2, the pattern of intervention effect differed among women with and without a chronic illness. Among Site 2 women with a chronic illness, the odds for completion of a study year Pap smear more than tripled in association with assignment to the physician reminder intervention (adjusted odds ratio = 3.38; P = 0.015). However, there was no physician reminder effect on Pap smear completion among women without a chronic illness. Factors other than physician reminder intervention that were independently associated with Pap smear completion among visitors to one or more sites included the presence of a previously normal Pap smear, younger age, commercial insurance, a gynecologic visit during the baseline period, absence of a chronic illness, and absence of a history of a sexually transmitted disease. The increased likelihood of a study year Pap smear among women not eligible for randomization to patient reminder intervention was an artifact, because prerandomization Pap smear completion was the reason for their exclusion.

DISCUSSION

The determinants of Pap smear screening in the primary care setting are complex, but its accomplishment generally requires a visit by a woman to a health care provider who recommends and/or performs the procedure. We therefore designed a two-part intervention to promote this process; our strategy first promoted visitation by women due for Pap smears and then prompted physician recommendation of the procedure. We targeted a predominately Medicaid- eligible population of minority women previously characterized by lower rates of Pap smear use and conducted this study within an HMO that gave women access to primary care and to Pap smear examination with no out-of-pocket expense, thus reducing two major potential barriers to procedure accomplishment. During the study year, over three-quarters of women enrolled in the HMO visited a physician, but fewer than one-third of enrollees (and 44% of visitors) obtained a Pap smear, and we observed no consistent impact of the reminders on HMO site visitation or Pap smear use. Because the determinants of procedure accomplishment reflect the interaction of the patient, the physician, and the health care system, the role of each must be considered in interpreting these results.

We hypothesized that a reminder letter sent to women at the time that they became due or overdue for a Pap smear could affect the likelihood of their obtaining a Pap smear by promoting visits among women who might not otherwise do so, by reducing the time to a next visit, or by indirectly affecting physician behavior at a visit. We found no evidence to support either the "conversion" of nonvisitors to visitors or an alteration of visitation-related behavior. We did, however, note an unanticipated effect on time to visitation among women with a chronic illness at one of the sites. At this site we observed an apparent delay in the time until the next visit among women with either hypertension or diabetes who received a letter reminding them to obtain a Pap smear. One possible explanation for this potentially adverse effect is that women with a chronic illness who otherwise visit regularly might choose to delay their next visit when they become aware that their arrival could trigger an uncomfortable or otherwise "unwanted" Pap smear examination. Although this is a post-hoc observation that applies to only one of three study sites, it does reinforce the caution that even seemingly "low risk" interventions, such as reminders that patients are due for medical procedures, can have unanticipated effects. It also emphasizes the importance of controlled trials in assessing not only the anticipated benefits but also the potential harm (and costs) of preventive intervention.

Patient reminders have had varying success in other settings, and we are uncertain as to why the patient reminder letter was less effective than had previously been observed.11, 17-19, 21-23 Because over two-thirds of eligible women visited during the study year and did so relatively frequently, there may have been little opportunity for visitation rates or timing to be affected. Differences in study populations were also relevant; we excluded women with previously abnormal Pap smears, and the lower-income, minority women we targeted may have differed from other groups in their patterns of response to the reminder letter. Although we were confident that the patient letters were delivered (only 9% were returned), we had no information concerning patients' comprehension of or immediate responses to the letter. We can only speculate as to whether a letter signed by a personal physician, with different content, or containing some other behavioral inducement would have been more effective than the reminder letter we employed.20, 27 In addition, we did not assess the potential efficacy of other potential modes of communication (for example, telephone vs. mailed reminders). It should also be noted that our measurement of the outcome associated with the letter reminder (namely, HMO site visitation) did not assess the potential impact of this intervention on intermediate outcomes, such as advancement in "stage of readiness," as had previously been assessed for patient reminders promoting mammography.28

Once a primary care visit takes place, the behavior of the primary care provider becomes an important determinant of Pap smear use. As with the patient reminders, the explanation for the observed absence of a consistent effect of the physician reminder on Pap smear use is uncertain. We and others have previously demonstrated a significant effect of physician reminders in primary care settings.14, 15 However, our previous experience with mammography may not apply to Pap smears, because mammography targets women older than those in the current study (all of whom were younger than 40 years), and there are other important differences between the two procedures, including patient acceptance, discomfort, and the logistics of completion. Of particular relevance was the requirement by the study HMO that primary care physicians refer visitors due for Pap smears to a gynecologist for the procedure. While this was necessitated by then-current HMO facility and time requirements, this introduction of repeat HMO visits might substantially attenuate a reminder effect. Exposure of physicians to the intended intervention appeared relatively complete; over 90% of medical record reminders were confirmed to be located in the medical chart at the time of or after patient visits.

A limited number of factors were associated with failure to complete Pap smear examination during the study year. These included presence of a chronic illness and lack of a previous Pap smear. It is noteworthy that, to some extent, the physician intervention appeared to target women with these factors. The effect of the physician reminder was significant among women with a chronic illness at Site 1 and among those without a previous Pap smear at Site 3. Although they were preliminary and post-hoc, these observations support further exploration of the value of intervention targeted to risk groups.

We also noted that intervention can have unanticipated effects. For example, among women at Site 1 with a previous normal Pap smear, assignment to physician reminder intervention was associated with a reduced likelihood that they would have a Pap smear during the study year. Similar to our observation of an apparent delay in time to a next visit among women with a chronic illness who received a reminder in the mail, this observation of a potentially "negative" effect of a physician reminder serves as a cautionary note in designing similar interventions. In the case of the physician reminder, physicians who believe that Pap smears need be obtained on a less frequent basis than the study HMO's annual guideline may have simply viewed the reminder as an indicator that the woman was "up-to-date" rather than as a prompt to perform the procedure.

A number of limitations to this study should be acknowledged. First, limited information was available concerning physician and patient characteristics, including Pap smear-related beliefs and attitudes, perceived barriers to Pap smear use, or attitudes concerning the reminders. We believe it unlikely that Pap smears were completed outside the study settings, in view of the access requirements that are inherent to HMO enrollment. Also, the observed rates of Pap smear completion should not be considered indicative of the pattern of overall use of the procedure at the participating HMO, because we excluded women whose last Pap smear had been abnormal. These women may have been more likely than the study women to have a Pap smear. Furthermore, our data did not allow identification of women who had previously undergone hysterectomy and/or other women for whom Pap smear may not have been indicated. Finally, some study physicians (and their patients) may disagree with an HMO policy that recommends an annual, as opposed to a less frequent, Pap smear (as has been recommended for lower risk women by some authorities).1

Despite considerable effort, our results demonstrated no consistent or substantial impact of a reminder-based intervention strategy on Pap smear utilization in an HMO in which financial barriers to access had been lessened for an economically disadvantaged urban population of women. In our intent-to-treat analysis, we noted an increase of approximately 25% in the odds of a visit or obtainment of a Pap smear during the study year among women assigned to the combined reminder strategy. However, even among the women randomized to combined reminders, the Pap smear completion rate did not exceed 32% (vs. 28% for those exposed to neither reminder); thus, any intervention impact was modest and of borderline statistical significance. At the same time, we did observe that it may be possible to identify subgroups of women more likely to benefit from these interventions and thus to target the intervention more efficiently. For example, physician reminders may prove most important for women among whom preventive procedures compete with other health care needs (similar to those with a chronic illness, as we observed at one study site). Patient reminder letters could focus on women who do not otherwise visit regularly. Furthermore, other approaches that could complement reminder-based interventions could also be integrated into the primary care setting. For example, in settings similar to the study HMO, where primary care physicians cannot perform Pap smears, a strategy that either equips the primary care provider to do so or promotes visits directly to a gynecologist or other health provider who can provide this service could remove an important logistic barrier.29, 30 If our preventive objectives are to be accomplished more completely, further efforts to promote preventive services in the primary care setting must identify effective behavioral and logistic approaches that are complementary to reminder systems.31, 32

Ancillary