SEARCH

SEARCH BY CITATION

Keywords:

  • family health history;
  • clinical decision support;
  • performance measure;
  • prenatal care;
  • genetic screening

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information

“The Pregnancy and Health Profile,” (PHP) is a free genetic risk assessment software tool for primary prenatal providers that collects patient-entered family (FHH), personal, and obstetrical health history, performs risk assessment, and presents the provider with clinical decision support during the prenatal encounter. The tool is freely available for download at www.hughesriskapps.net. We evaluated the implementation of PHP in four geographically diverse clinical sites. Retrospective chart reviews were conducted for patients seen prior to the study period and for patients who used the PHP to collect data on documentation of FHH, discussion of cystic fibrosis (CF) and hemoglobinopathy (HB) carrier screening, and CF and HB interventions (tests, referrals). Five hundred pre-implementation phase and 618 implementation phase charts were reviewed. Documentation of a 3-generation FHH or pedigree improved at three sites; patient race/ethnicity at three sites, father of the baby (FOB) race/ethnicity at all sites, and ancestry for the patient and FOB at three sites (P < 0.001–0001). CF counseling improved for implementation phase patients at one site (8% vs. 48%, P < 0.0001) and CF screening/referrals at two (2% vs. 14%, P < 0.0001; 6% vs. 14%; P = 0.05). Counseling and intervention rates did not increase for HB. This preliminary study suggests that the PHP can improve documentation of FHH, race, and ancestry, as well as the compliance with current CF counseling and intervention guidelines in some prenatal clinics. Future evaluation of the PHP should include testing in a larger number of clinical environments, assessment of additional performance measures, and evaluation of the system's overall clinical utility. © 2014 Wiley Periodicals, Inc.


INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information

While evidence of the utility of family health history (FHH) screening and genetic testing in primary care has increased in recent years, providers still face difficult clinical decisions in the absence of definitive evidence for the majority of genomic applications [Khoury et al., 2010]. Professional society guidelines can provide useful direction, but the provider may be faced with a laundry list of societies that advise on a particular condition which, at times, provide conflicting information. For example, evaluation of a family at risk of Fragile X syndrome is addressed by five professional societies representing different clinical perspectives and with some differences in recommended FHH assessment and management [Filipek et al., 2000; McConkie-Rosell et al., 2005; Sherman et al., 2005; Moeschler et al., 2006; ACOG, 2010]. Even when professional society guidelines exist and are in agreement there are often significant gaps between recommended practice and clinical practice. In prenatal care, offering at-risk pregnant women and their partners carrier screening for cystic fibrosis has been a nationally-recognized recommendation since 1997 [NIH, 1999], but a significant number of clinicians' practices are still not consistent with the guidelines [Darcy et al., 2011]. Similarly, FHH has long been recognized as an effective and cost-efficient method to screen for disease, but it is not consistently utilized in clinical practice [Levy et al., 2009].

In prenatal care, offering at-risk pregnant women and their partners carrier screening for cystic fibrosis has been a nationally-recognized recommendation since 1997, but a significant number of clinicians' practices are still not consistent with the guidelines.

Clinical decision support (CDS) can address gaps between guidelines and practice by presenting the provider with suggested evidence- or guidelines-based clinical actions within the clinical workflow. Health IT applications and electronic health records (EHRs) provide an opportunity for effective delivery of CDS within existing workflows, which may reduce time and access barriers and increase compliance with guidelines [Welch and Kawamoto, 2013]. Systems for collection and assessment of FHH in adult clinics have shown improved FHH documentation and increased access to targeted services, such as to specialists or genetic testing [Ozanne et al., 2009; Rubinstein et al., 2011; Orlando et al., 2014]. To date, outcomes assessment of electronic FHH collection and CDS has not been performed in prenatal care. Prenatal care faces particular challenges, such as time constraints during which intervention can occur (e.g., the pregnancy or a single trimester), management of the triad of pregnancy health, child health, and woman's health, postpartum, and continuity of care considerations between mother and child.

The Pregnancy and Health Profile (PHP): A Screening and Risk Assessment Tool collects patient-entered FHH, personal health history, and obstetric history. The system then performs risk assessment, and presents the provider with CDS during (or potentially prior to) the first clinical encounter [Edelman et al., 2013; Lin et al., 2013; NCHPEG, 2013]. The goal of the PHP is to improve identification of women and babies at increased risk of genetic disease as well as other prenatal risk factors and to increase the rate at which these families are offered appropriate services and screening. Previous analysis demonstrated high patient satisfaction and usability regarding the PHP [Edelman et al., 2013]. Provider responses to PHP and the components and resources required for implementation have also been reported [Edelman et al., 2013].

In this article, we describe the effect of implementation of the PHP on adherence to select genetic guidelines in prenatal care. We assessed six measures of guideline adherence in prenatal care (Table I): documentation of (1) a 3-generation FHH; (2) race, ethnicity, and ancestry; (3) discussion of cystic fibrosis carrier screening and (4) hemoglobinopathies among at risk individuals; and (5) tests ordered or referrals for cystic fibrosis and (6) tests ordered or referrals for hemoglobinopathies.

Table I. Genetic Performance Measures
Performance measureImplementation phase interventiona
  1. a

    See www.nchpeg.org/index.php?option=com_content&view=article&id=410&Itemid=277 for additional information about the intervention and CDS.

  2. b

    These examples are for the most comment risk scenarios. Targeted messages for other risk scenarios, such as a positive family history of disease or previous genetic testing for the disease, were applied when appropriate.

% of patients with documented 3-generation FHHPedigree + tabular FHH list
% of patients and partners with documented race, ethnicity, and ancestry dataDetailed race, ethnicity, ancestry data pre-populated into report + informing CDS
% of patients with documented discussion, counseling, or education about cystic fibrosis carrier screeningCDS Exampleb: Offer carrier screening for cystic fibrosis. If both the father of the pregnancy and the patient are identified as cystic fibrosis carriers, refer to genetic counseling to review prenatal testing and reproductive options.
% of patients with a cystic fibrosis discussion for hemoglobinopathy who received a recommendation for carrier screening or a genetics referral 
% of patients at increased risk based on ethnicity with documented discussion, counseling, or education about hemoglobinopathy carrier screeningCDS Exampleb: Check CBC (MCV value). Order hemoglobin electrophoresis. Measure patient's serum ferritin levels to exclude iron deficiency anemia. If both parents are determined to be carriers, refer to genetic counseling.
% of patients at increased risk for hemoglobinopathy who received a recommendation for carrier screening or a genetics referral 

METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information

Implementation

The PHP was implemented into four clinical sites: (1) Mountain Area Health Education Center, an academic community-based obstetrics and gynecology residency program in Asheville, NC; (2) Maine Dartmouth Family Medicine Residency Program, a rural academic family medicine program in Fairfield and Augusta, ME; (3) Montefiore Medical Group–Comprehensive Family Care Center, an urban federally qualified health center with an academic affiliation in the Bronx, NY; and (4) Clearvista Women's Care, Community Health Network, an obstetric practice that is part of a community hospital network in the suburbs of Indianapolis, IN. These sites were a convenience sample that were selected to provide geographic and demographic diversity [Edelman et al., 2013].

To prepare for implementation, at each site project and site staff conducted a needs assessment to determine patient flow and processes, space, staff resources, and provider educational needs and preferences. A customized installation, implementation, and training plan was developed for each site. The CDS and PHP itself was not customized among sites; it was a consistent intervention. Provider training included 1–2 h group training covering the value of FHH in prenatal care, the components of the tool, the proposed implementation plan, and workflow challenges and resolutions. Ongoing physician support was provided by site coordinators as needed. While participating providers were aware of the overall evaluation goals and methods, providers did not know the specific performance measures assessed in this study.

There were additional preparations for implementation at ME, including participation in a formative evaluation of a PHP prototype among a subset of providers, with feedback informing revisions that resulted in the implementation version of PHP. Additionally, staff and residents performed a baseline chart audit assessing practice performance in FHH collection and genetic screening as part of initial needs assessment. These data were presented to ME providers prior to implementation.

The approach to integration of the tool into clinical flow, patient and provider eligibility, and customized implementation has been previously described [Edelman et al., 2013]. In brief, patients entered their personal history and FHH for themselves and father of the baby (FOB) in the PHP via a tablet computer prior to seeing the clinician at the first prenatal visit. Women who could not read English were excluded. At one site (NY), informed consent was obtained and patients had the option to decline use of the tool. Providers received the PHP report, an adaptation of the ACOG Antepartum Record, populated with the patient-entered data and with CDS for genetic and FHH conditions and obstetric risks. At two sites (ME, NC), the PHP report used by providers was filed with paper records while staff at the remaining two sites transcribed relevant data from the PHP report into the EHR (NY) or scanned the report into the EHR (IN). IRB approval or exemption was received through each site's Institutional Review Board.

Data Collection

After the study period, charts from consecutive patients seen immediately prior to the study period (pre-implementation phase) and charts from patients who used the tool (implementation phase) were reviewed to compare documentation of FHH and discussion about cystic fibrosis (CF) and hemoglobinopathy (HB) risks (Table I; see Supplement for data collection instrument). We selected these outcomes based on predicted ability to detect differences in provider behavior between pre-implementation and implementation, strong professional society support, and the relatively high incidence of CF and HB compared to other genetic conditions.

The 3-generation FHH measure was defined as documentation of at least one member of three generations (e.g., the patient, her children, and her parents) [ACOG, 2011b]. Race categories included African American or Black; Asian or Pacific Islander; Caribbean or West Indian; Caucasian or White; and Native American. Ethnicity was defined as Hispanic or Latina [OMB 1997]. Ancestry was defined as a country or region of origin (e.g., Turkish) or Jewish ancestry and included in the PHP if recommended by professional society guidelines [ACOG, 2005, 2007, 2009, 2011a]; patients could also select “None” or “Prefer not to answer.” All patients were considered to be candidates for CF screening [ACOG, 2011a] and patients or parents of African, Asian, Mediterranean, Middle Eastern, and Hispanic ancestries were considered to be at increased risk of HB [ACOG, 2007].

Clinicians and administrators also participated in online or paper-based surveys and interviews to provide feedback on the usability, utility, and implementation process of the PHP [Edelman et al., 2013].

Data Analysis

All data were entered into Excel and analyzed using Excel and web-based statistics calculators. For each performance measure, Fisher's exact test was used to compare the proportion of pre-implementation patients whose provider followed the professional guideline to the proportion of patients during the implementation phase whose provider followed the guideline. Based on the wide variation in adherence to professional guidelines by site, pre- and implementation analyses were done separately for each site. P values were considered significant at the P < 0.05 level. In order to examine the potential effect of the nested design (multiple patients per provider, with varying distributions) on results, the performance measure for CF was examined by provider (for both pre-implementation and implementation phases), with no evidence that the design affected results.

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information

Across the four sites, 500 pre-implementation charts were reviewed; with a range of 86–214 per site, depending on resources and logistics. Six hundred eighteen patients used the tool during the study period and all were reviewed for performance measure analysis (Fig. 1). Differences in race/ethnicity between pre-implementation and implementation phase patients were expected since the tool was designed to improve collection and documentation (Table II). Administrators from each respective site reported no system or population changes during the pre-implementation time period compared to the pilot study and believe the two patient populations are consistent for this analysis.

image

Figure 1. Patient study flow.

* At some clinics, the tool could only be used when the study coordinator was onsite. Logistical conflicts include sick days, holidays, and other scheduling conflicts.

Download figure to PowerPoint

Table II. Patient Demographics, Pre-Implementation (Phase 1), and Implementation (Phase 2)
 NCMENYIN
 Phase 1Phase 2Phase 1Phase 2Phase 1Phase 2Phase 1Phase 2
  1. Percentages may not equal 100: Racial categories are mutually exclusive but ethnicity and race are not mutually exclusive. If patients did not have either race or ethnicity documented, they were included in the “no documentation” category below.

# providers2028523751077
# patients100254214968641100228
Ethnicity% (n)% (n)% (n)% (n)% (n)% (n)% (n)% (n)
Hispanic or Latina3.0 (3)4.7 (12)2.3 (5)5.2 (5)20.9 (18)61.0 (25)0.0 (0)2.6 (6)
Race% (n)% (n)% (n)% (n)% (n)% (n)% (n)% (n)
Caucasian/White83.0 (83)83.9 (213)69.6 (149)87.5 (84)3.5 (3)12.2 (5)80.0 (80)80.7 (184)
African-American/Black12.0 (12)9.4 (24)2.8 (6)0.0 (0)14.0 (12)29.3 (12)6.0 (6)11.0 (25)
Asian/Pacific Islander0.0 (0)0.8 (2)0.5 (1)2.1 (2)1.2 (1)0.0 (0)2.0 (2)4.4 (10)
Native American0.0 (0)0.4 (1)0.9 (2)3.1 (3)0.0 (0)2.4 (1)0.0 (0)0.0 (0)
Caribbean/West Indian0.0 (0)0.0 (0)0.0 (0)0.0 (0)0.0 (0)17.1 (7)0.0 (0)0.0 (0)
Multi-racial1.0 (1)3.1 (8)0.0 (0)2.1 (2)0.0 (0)0.0 (0)2.0 (2)1.3 (3)
No documentation of race/ethnicity4.0 (4)0.8 (2)26.2 (56)3.1 (3)60.5 (52)24 (1)10.0 (10)1.8 (4)

Implementation phase charts had an average of 7.6 (site range 6.9–8.3) genetic CDS considerations and 2.8 (2.3–3.3) non-genetic, obstetric CDS considerations triggered via the PHP. Table III lists the most common CDS considerations across sites.

Table III. Top 10 Genetic and Obstetric Conditions Triggered in CDS Across Four Sites (n = 618 Participants)
Risk assessment for genetic condition% (n)Risk assessment for obstetric conditions/risk factors% (n)
Aneuploidy99.7 (616)Works with children43.9 (271)
Cystic fibrosis96.6 (597)Tuna consumption33.2 (205)
Diabetes59.9 (370)Second-hand smoke32.7 (202)
Hypertension57.9 (358)Acne22.8 (141)
Depression52.9 (327)Exercise, lack of22.8 (141)
Thalassemia42.4 (262)Not taking prenatal vitamins19.7 (122)
Sickle cell disease or trait40.5 (250)Smoker16.7 (103)
Cardiovascular disease40.3 (249)Frequent urinary tract infections12.9 (80)
Recurrent pregnancy loss29.8 (184)Raw meat consumption11.5 (71)
Bipolar disorder20.6 (127)Infertility10.7 (66)

Family History

FHH documentation improved at three out of four sites. Documentation of a 3-generation FHH improved with the tool in two sites (NC, NY; Table IV). At ME, documentation of a 3-generation pedigree was collected instead; one out of 214 pre-implementation charts had a pedigree, and 98% of charts had a pedigree in the implementation phase. IN had an existing appointment template that included prompts to ask about the health of the patient's parents and children, so this level of documentation was present for almost all patients in the pre-implementation phase. Staff at IN reported in interviews that the level of detail collected in the FHH increased when using PHP, consistent with chart findings from the other sites.

Table IV. Family History and Race, Ethnicity, and Ancestry Performance Measures, Comparison Between Pre-Implementation (Phase 1), and Implementation (Phase 2)
% of charts with documentedNCMENYIN
Phase 1 % (n)Phase 2 % (n)P-valuePhase 1 % (n)Phase 2 % (n)P-valuePhase 1 % (n)Phase 2 % (n)P-valuePhase 1 % (n)Phase 2 % (n)P-value
  1. NA, not available or not applicable; NS, not significant.

# patients100254 21496 8641 100228 
3-generation FHH64.0 (64)98.4 (250)<0.001NA97.9 (94)NA0.0 (0)97.5 (39)<0.00199.0 (99)99.1 (226)NS
Race/ethnicity of patient96.0 (96)99.2 (252)NS73.8 (158)96.9 (93)<0.000139.5 (34)97.5 (39)<0.00190.0 (90)98.2 (224)<0.001
Race/ethnicity of FOB3.0 (3)97.6 (248)<0.0012.8 (6)95.8 (92)<0.000133.7 (29)97.5 (39)<0.0010.0 (0)98.2 (224)<0.001
Country of origin of pt. grandparents/ancestors0.0 (0)16.5 (42)<0.0013.3 (7)34.4 (33)<0.00015.8 (5)12.5 (5)NS0.0 (0)23.2 (53)<0.001
Country of origin of FOB grandparents/ancestors0.0 (0)11.0 (28)<0.0010.5 (1)30.2 (29)<0.00013.5 (3)7.5 (3)NS0.0 (0)18.9 (43)<0.001

In three sites (ME, NY, IN), documentation of patient race/ethnicity improved with use of the tool, and documentation of FOB race/ethnicity improved in all sites (Table IV). FOB information was rarely documented in the pre-implementation charts. Patient and FOB ancestry documentation also improved in three (NC, ME, IN; Table IV). Across all sites, 86% of patients in the implementation phase responded to ancestry questions and ancestry data was provided for 77% of FOBs. Clinically significant ancestry data (those countries associated with condition risk) was provided for 22% of patients and 17% of FOBs (Table IV).

Implementation phase charts had an average of 5.5 (SD 3.2) health conditions identified in each family (Table V). Data are not available for the number of health conditions identified in the pre-implementation phase.

Table V. Top 10 Clinical Conditions Reported in Personal and/or Family Histories Across Four Sites (n = 609 patients who reported any family history information)
Personal and family history conditions% (n)
  1. a

    Collected from patient medical history only.

Diabetes60.8 (370)
History of hospitalization/surgerya58.5 (356)
Hypertension57.5 (350)
Depression53.7 (327)
Cancer46.1 (281)
Cardiovascular disease35.0 (213)
Acnea29.4 (179)
Bipolar disorder25.3 (127)
Recurrent pregnancy loss (2+)18.7 (114)
Other mental health condition16.6 (101)

Cystic Fibrosis Carrier Screening

The frequency of discussion, counseling, or education about CF carrier screening improved in one site (ME; Table VI). Two sites (NC, IN) offered CF screening to the majority of patients in the pre-implementation phase (88–84%) and this stayed the same during implementation (82–90%). In NY, CF screening was less common in pre-implementation (5.8%) and did not increase during implementation (2.5%). These trends did not change when we examined the subset of patients who have a higher likelihood of being a CF carrier (white, European, and Jewish race/ancestry) or among those with a positive family history. Three couples reported a positive FHH of CF pre-implementation (all at NC) and 12 couples during implementation (5 NC, 1 ME, 3 NY, 3 IN). Of these, NC, ME, and IN charts had documented counseling for all positive cases, and NY did not have documented counseling for any positive cases.

Table VI. Cystic Fibrosis Performance Measures, Comparison Between Pre-Implementation (Phase 1) and Implementation (Phase 2)
 NCMENYIN
 Phase 1 % (n)Phase 2 % (n)P-valuePhase 1 % (n)Phase 2 % (n)P-valuePhase 1 % (n)Phase 2 % (n)P-valuePhase 1 % (n)Phase 2 % (n)P-value
  1. Sections A and B shows the percentage of charts with a documentation of discussion or counseling about cystic fibrosis (CF) carrier screening, among (A) all patients, and (B) couples where one or both partners are of a high risk ethnicity. High risk ethnicity was defined as white, Caucasian, European ancestries, or Ashkenazi Jewish ancestry. Section C presents the percentage of patients, among those that received a discussion of CF and regardless of ethnicity, that received a recommendation for carrier screening or a referral to genetics regarding CF risk. NA, not available or not applicable; NS, not significant.

  2. a

    ∼1.7% of patients across all sites women had CF carrier testing previously; these women were included in this table and coded as having received counseling and testing.

# patients100254 21496 8640 100228 
% couples with high risk ethnicity80.0 (80)87.4 (222)NS69.6 (149)89.6 (86)<0.00014.7 (4)15.0 (6)NS80.0 (80)83.8 (191)NS
(A) All pts: (%) discussiona of CF carrier testing88.0 (88)81.9 (208)NS8.4 (18)47.9 (46)<0.00015.8 (5)2.5 (1)NS84.0 (84)90.4 (206)NS
(B) High risk ethnicity: (%) discussiona of CF carrier testing85.0 (68)80.6 (179)NS3.3 (7)45.3 (39)<0.00010.0 (0)0.0 (0)NS90.0 (72)92.1 (176)NS
(C) All pts with a CF discussiona: (%) with test or referral recommendation30.7 (27)30.3 (63)NS2.2 (4)13.7 (13)<0.000160.0 (3)100.0 (1)NS6.0 (5)13.6 (28)0.05

We examined provider recommendations for CF screening or a genetics referral, among patients who had a discussion about screening (Table VI). In two sites (ME, IN), the frequency of patients receiving CF screening/referrals increased in the implementation phase. Similar to the CF counseling rates, a consistent proportion of patients receive a screening test or referral at NC (∼30% of those with a discussion) and a lower proportion of patients received a screening test or referral at NY.

Hemoglobinopathy Screening

More implementation phase patients were identified to be at increased risk of a hemoglobinopathy based on ethnicity compared to pre-implementation phase patients (NC, 39.8% vs. 16.0%, P < 0.0001; NY 95.0% vs. 38.4%, P < 0.0001; IN 39.5% vs. 8%, P < 0.0001; ME 33.3% implementation phase, pre-implementation unknown because of different ethnicity categorization). However, similar rates of counseling were seen among those who were identified to be at increased risk. Counseling rates were relatively low pre-implementation and did not change with tool use (NC, 31.3% pre-implementation vs. 25.7% implementation; NY, 9.1% vs. 2.6%; IN, 25.0% vs. 10%; ME, 0% implementation phase). Analysis of couples with African American race or ancestry, a subset of the total population of increased risk couples, also showed similar counseling rates between pre-implementation and implementation phases (NC, 30.8% pre-implementation vs. 36.2% implementation; NY, 8.3% vs. 0.0%; IN, 33.3% vs. 19.4%; ME, 0.0% vs. 0.0%). We originally aimed to analyze the Asian American subset, but did not have sufficient numbers of this demographic group (Table II). No sites had an increase in the proportion of at-risk patients receiving a HB test or referral and in NC, the frequency decreased from pre-implementation to implementation phase (81% vs. 33%, P < 0.01).

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information

To our knowledge, this study presents the first evaluation of the clinical utility of a comprehensive prenatal genetic clinical decision support system in real world clinical practice. We found areas of practice where performance improved during the PHP implementation period, and areas where practice did not change, suggesting factors beyond the PHP influencing genomic risk documentation and counseling. Additionally, for a given performance measure, we observed improvement at some sites and not others.

To our knowledge, this study presents the first evaluation of the clinical utility of a comprehensive prenatal genetic clinical decision support system in real world clinical practice. We found areas of practice where performance improved during the PHP implementation period, and areas where practice did not change, suggesting factors beyond the PHP influencing genomic risk documentation and counseling.

FHH documentation had significant improvement in three sites, with all sites showing documentation of FHH in over 97% of charts. These findings are consistent with other studies that demonstrate electronic FHH tools can improve FHH documentation. Baer et al. [2013] evaluated the implementation of a web-based risk appraisal tool that collected FHH about specific cancers and cardiovascular conditions and made the data available in the EHR with CDS for screening. Intervention patients were more likely to have documentation of cancer FHH compared to controls (2% vs. 0.6%; OR 4.3). Scheuner et al. [2014] developed a cancer genetics toolkit that included an EHR reminder to collect FHH; documentation improved by 10% from a baseline of 26.6% [Scheuner et al., 2014]. Our definition of a 3-generation FHH (documentation of at least one individual in three generations) in this evaluation was very conservative and did not describe the horizontal depth of the FHH (inclusion of siblings, aunts, uncles), or additional detail generally recommended for comprehensive collection and risk assessment (e.g., affected and unaffected relatives, age of onset of disease). Powell and colleagues recently published proposed quality indicators necessary for accurate primary care FHH risk assessment. Powell et al. [2013] determined that while almost all of the 390 charts analyzed included some mention of FHH, only 4% included sufficient FHH data to support comprehensive risk assessment and recommended development of electronic CDS tools that can efficiently and effectively aid the provider to collect FHH and assess risk. These quality indicators are more detailed than the FHH and race and ancestry performance measures used in this study and could be considered in future evaluations of the PHP. Other studies demonstrate 27–40% of primary care charts with FHH documented at baseline [Acheson et al., 2000; Scheuner et al., 2014]. The PHP's ability to improve upon FHH documentation in the prenatal setting overcomes a significant barrier to FHH implementation—collecting clinically useful FHH data—and can enable providers to detect inherited risks, as recommended by ACOG [2011b].

To our knowledge, ours is the first study that systematically compared race, ethnicity, and ancestry documentation pre-implementation to implementation. We demonstrated the PHP improved documentation of patient and FOB race and ancestry and identified more patients at higher ethnicity-based risk for conditions such as hemoglobinopathies in the majority of sites. This ethnicity-based screening can add value to the prenatal clinic where screening members of certain populations for a specific genetic condition, such as Tay–Sachs disease, sickle-cell disease, and thalassemia, has become standard of care in clinical practice [ACOG, 2005, 2007]. Furthermore, such systematic documentation of race, ethnicity, and ancestry data has potential utility in other areas of medicine such as pharmacogenomics where a specific genomic assessment may be offered to patients of certain ancestries [Yip et al., 2012].

Our data suggest that the PHP may be particularly useful in identifying women of Mediterranean and Middle Eastern ancestries to be candidates for HB screening, as there was limited collection of such ancestry data during the pre-implementation phase and these women made up a large proportion of the high risk ethnicity group in the implementation phase. Additionally, while not statistically significant, sites appeared to have screening discussions with women of African American ancestry more frequently than other at-risk ethnicities, suggesting additional opportunity for HB counseling with couples of non-African ethnicity.

Although improving FHH collection is important, the provider must use this information clinically to realize the benefits of FHH. We found limited evidence that the PHP affected the discussions that providers have with patients about carrier screening. With the exception of CF counseling at ME, rates of discussion of CF and HB did not change between the pre-implementation and implementation phases. For those practices that already had an effective CF education and counseling program in place at baseline, such as NC and IN, we would not expect much improvement with implementation of the PHP. However, we did hypothesize that sites with low baseline counseling rates, such as ME and NY for CF and all sites for HB, would see improvement during the implementation phase. The limited observed change in this study seems to be consistent with the few similar studies identified in the literature. In the Baer et al. study, there was no change in the proportion of patients in the intervention group that received a discussion of FHH with the provider (70.9%) compared to the control group (73.8%) [Baer et al., 2013]. Shaikh et al. studied the impact of an automatic body mass index calculation in the EHR on provider counseling about nutrition and physical activity. They found that FHH assessment did increase from 7% to 62%, but found no change in the rate of patient counseling for FHH risks or nutritional assessment. They concluded that passive changes, such as an electronic alert in the EHR, are insufficient to impact practice change [Shaikh et al., 2010].

A possible explanation for the limited change we observed in counseling for carrier screening and offering of services (testing or referrals) is that there may have been competing, higher priority risks for the patient (e.g., acute illness, substance abuse) that were not assessed as part of this study, but required the provider to focus encounter time and management on non-genetic issues. It is also possible that providers disagreed with the guidelines regarding carrier screening presented to them in CDS. In an evaluation of patient and provider feedback of the PHP, providers had variable reactions to the value of the PHP and helpfulness of CDS, with some providers voicing concerns about the number of CDS conditions presented and the design of the CDS presentation [Edelman et al., 2013]. Any technical issues experienced in the delivery of the PHP report or CDS could have impacted providers' acceptance and trust, and in turn compliance, with the system. Providers may have limited awareness, understanding, or knowledge of the performance measures assessed in this study [Darcy et al., 2011] and the PHP intervention may not have been not sufficient to overcome these barriers. The guidelines themselves on which the performance measures were based may not provide complete guidance to clinicians, which could contribute to the limited practice changes observed in this study. Future studies could also expand the assessment of provider-patient counseling to include follow-up interviews with patients and providers or even direct observation to determine if more subtle, or non-documented, changes happened within the encounter.

At ME, we did observe improvement in cystic fibrosis counseling and services offered between pre-implementation and implementation (8% vs. 48%), which is aligned with ACOG recommendations [ACOG, 2011a]. More frequent engagement and the series of interventions experienced by ME providers may account in part for the observed improvement. ME was the first site we recruited to participate in the pilot and at that early stage, a subset of their providers participated in formative evaluation pilot testing that informed improvements to the tool. These providers had previous exposure to a version of the PHP and also had an opportunity to engage in constructive feedback in the development period. As part of initial awareness and training, ME providers conducted a chart review to assess baseline performance in FHH documentation and genetic screening. These data were presented back to the entire ME staff and providers could examine how their practice compared to professional society guidelines. These additional opportunities of engagement and intervention, along with a protracted period of exposure to PHP, created a unique implementation compared to the other three sites, and may have impacted the degree to which providers accepted the PHP tool and its CDS messages. Adult learning theory and implementation science support multiple complimentary interventions as a mechanism to maximize practice change, as do findings from FHH studies that have utilized a multi-pronged approach to implementation [Orlando et al., 2011; Scheuner et al., 2014].

The design of this study imposed some limitations, as previously discussed [Edelman et al., 2013]. In addition to previously reported limitations of the study design, this paper described evaluation of six performance measures previously untested in clinical settings. These performance measures were created: based on clinician and investigator consideration of (1) the frequency of the condition and/or patients appropriate for screening in the general population, and (2) provider awareness and acceptance of professional guidelines related to the measure. The measures have not, however, been studied in other settings, and we have no data about how provider performance on these six measures correlates to or predicts performance in other areas of genetic practice.

We recruited a convenience sample of clinics that had the resources and interest to collaborate in evaluation of the PHP. The populations were not randomized and the study observations may also include effects of temporal confounders and other factors in the clinical environment that were not measured. The findings are not generalizable; they are related to the specific population of women who read and speak English at one of the four sites. This study also makes an assumption that documentation of the first prenatal visit in the medical record accurately reflects the activities in the clinical encounter. While this is a reasonable assumption, we expect that there were situations in which the documentation did not fully address the patient's care. Site staff conducting the chart review may have also missed documentation, resulting in underreporting of activities in either phase. Finally, while the PHP underwent validation testing in a small sample prior to implementation and continues to undergo quality control assessment, the validity of the PHP has not been formally tested in a randomized trial.

While this study evaluated one kind of outcome of a clinical intervention, it is well recognized that successful implementation of any new system or guideline in clinical practice requires a coordinated design and multi-faceted evaluation that addresses process, participant, and clinical outcomes [Feldstein and Glasgow, 2008]. Implementation of genomic applications and tools should follow frameworks and best practices established in other areas of medicine while also allowing for opportunities to develop and validate instruments and measures effective in evaluating such genomic implementation activities. The work presented in this paper, along with previously published findings on PHP implementation [Edelman et al., 2013], can be used as an example to other groups or institutions seeking to implement PHP or similar tools. In our experience, engaging the clinical users in cycles of planning, development, and improvement, is important for successful implementation; as is providing customizable implementation plans and clinical, technical, and administrative support during implementation.

While this study evaluated one kind of outcome of a clinical intervention, it is well recognized that successful implementation of any new system or guideline in clinical practice requires a coordinated design and multi-faceted evaluation that addresses process, participant, and clinical outcomes.

We have demonstrated the PHP can improve documentation of FHH and race and ancestry, identification of couples at increased risk of disease based on race and ancestry, and the frequency of cystic fibrosis counseling and interventions in some prenatal clinics. This study did not find significant improvements in counseling, screening, or referrals for hemoglobinopathies after implementing PHP. Furthermore, we have explored the development of six proposed performance measures as a mechanism for assessing genomic implementation in prenatal medicine. Future evaluation of the PHP should include validation and assessment of these and additional performance measures and evaluation of the downstream impact on services utilized, system cost, and health outcomes in large populations. Future genomic implementation efforts of the PHP or related programs should include engagement of providers in intervention design and data collection, delivery of multiple interventions to support key messages and increase impact, and additional study of other factors that can influence implementation in the clinical system.

ACKNOWLEDGMENTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information

The Pregnancy and Health Profile was developed by the National Coalition for Health Professional Education in Genetics, Genetic Alliance, HughesRiskApps.net, the March of Dimes, and the Health Resources and Services Administration (HRSA) and is funded by Grant #U33MC12786 from HRSA. The views expressed in this publication are solely the opinions of the authors and do not necessarily reflect the official policies of the U.S. Department of Health and Human Services or the Health Resources and Services Administration, nor does mention of the department or agency names imply endorsement by the U.S. government. We are grateful to the women, providers, and clinic staff that participated in this project, especially the study coordinators and investigators from our collaborating clinical sites for their effort and continued partnership. We also acknowledge the project's national advisory committee for members' advice and time, and Joseph McInerney, Michele Lloyd Puryear, and Penny Kyler for their leadership and vision at early stages of this project.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information
  • Acheson LS, Wiesner GL, Zyzanski SJ, Goodwin MA, Stange KC. 2000. Family history-taking in community family practice: Implications for genetic screening. Genet Med 2:180185.
  • ACOG. 2005. ACOG committee opinion. Number 318, October 2005. Screening for Tay–Sachs disease. Obstet Gynecol 106:893894.
  • ACOG. 2007. ACOG Practice Bulletin No. 78: Hemoglobinopathies in pregnancy. Obstet Gynecol 109:229237.
  • ACOG. 2009. ACOG Committee Opinion No. 442: Preconception and prenatal carrier screening for genetic diseases in individuals of Eastern European Jewish descent. Obstet Gynecol 114:950953.
  • ACOG. 2010. ACOG Committee Opinion No. 469: Carrier screening for fragile X syndrome. Obstet Gynecol 116:10081010.
  • ACOG. 2011a. ACOG Committee Opinion No. 486: Update on carrier screening for cystic fibrosis. Obstet Gynecol 117:10281031.
  • ACOG. 2011b. Committee Opinion No. 478: Family history as a risk assessment tool. Obstet Gynecol 117:747750.
  • Baer HJ, Schneider LI, Colditz GA, Dart H, Andry A, Williams DH, Orav EJ, Haas JS, Getty G, Whittemore E, Bates DW. 2013. Use of a web-based risk appraisal tool for assessing family history and lifestyle factors in primary care. J Gen Intern Med 28:817824.
  • Darcy D, Tian L, Taylor J, Schrijver I. 2011. Cystic fibrosis carrier screening in obstetric clinical practice: Knowledge, practices, and barriers, a decade after publication of screening guidelines. Genet Test Mol Biomarkers 15:517523.
  • Edelman EA, Lin BK, Doksum T, Drohan B, Edelson V, Dolan SM, Hughes KS, O'Leary J, Vasquez L, Copeland S, Galvin S, DeGroat N, S P, Feero WG, Adams C, Scott JA. 2013. Evaluation of a novel electronic genetic screening and clinical decision support tool in prenatal clinical settings. Matern Child Health J epub ahead of print.
  • Feldstein AC, Glasgow RE. 2008. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf 34:228243.
  • Filipek PA, Accardo PJ, Ashwal S, Baranek GT, Cook EH Jr, Dawson G, Gordon B, Gravel JS, Johnson CP, Kallen RJ, Levy SE, Minshew NJ, Ozonoff S, Prizant BM, Rapin I, Rogers SJ, Stone WL, Teplin SW, Tuchman RF, Volkmar FR. 2000. Practice parameter: Screening and diagnosis of autism: Report of the Quality Standards Subcommittee of the American Academy of Neurology and the Child Neurology Society. Neurology 55:468479.
  • Khoury MJ, Coates RJ, Evans JP. 2010. Evidence-based classification of recommendations on use of genomic tests in clinical practice: Dealing with insufficient evidence. Genet Med 12:680683.
  • Levy DE, Garber JE, Shields AE. 2009. Guidelines for genetic risk assessment of hereditary breast and ovarian cancer: Early disagreements and low utilization. J Gen Intern Med 24:822828.
  • Lin BK, Edelman EA, McInerney JD, O'Leary J, Edelson V, Hughes KS, Drohan B, Kyler P, Puryear ML, Dolan SM. 2013. Personalizing prenatal care using family health history: Identifying and integrating a panel of conditions into a novel screening tool. Personalized Med 10:307318.
  • McConkie-Rosell A, Finucane B, Cronister A, Abrams L, Bennett RL, Pettersen BJ. 2005. Genetic counseling for fragile x syndrome: Updated recommendations of the national society of genetic counselors. J Genet Couns 14:249270.
  • Moeschler JB, Shevell M, American Academy of Pediatrics Committee on Genetics. 2006. Clinical genetic evaluation of the child with mental retardation or developmental delays. Pediatrics 117:23042316.
  • NCHPEG. 2013. The pregnancy & health profile: A screening & risk assessment tool. Lutherville, MD: NCHPEG. http://www.nchpeg.org/index.php?option=com_content&view=article&id=410&Itemid=277
  • NIH. 1999. Genetic testing for cystic fibrosis. National Institutes of Health Consensus Development Conference Statement on genetic testing for cystic fibrosis. Arch Intern Med 159:15291539.
  • OMB. 1997. Revisions to the standards for the classification of federal data on race and ethnicity. Federal Register 62:9.
  • Orlando LA, Hauser ER, Christianson C, Powell KP, Buchanan AH, Chesnut B, Agbaje AB, Henrich VC, Ginsburg G. 2011. Protocol for implementation of family health history collection and decision support into primary care using a computerized family health history system. BMC Health Serv Res 11:264.
  • Orlando LA, Wu RR, Himmel T, Buchanan AH, Powell KP, Hauser E, Henrich VC, Ginsburg G. 2014. Health services impact of implementing family health history risk stratification in primary care. Am J Med Genet C.
  • Ozanne EM, Loberg A, Hughes S, Lawrence C, Drohan B, Semine A, Jellinek M, Cronin C, Milham F, Dowd D, Block C, Lockhart D, Sharko J, Grinstein G, Hughes KS. 2009. Identification and management of women at high risk for hereditary breast/ovarian cancer syndrome. Breast J 15:155162.
  • Powell KP, Christianson CA, Hahn SE, Dave G, Evans LR, Blanton SH, Hauser E, Agbaje AB, Orlando LA, Ginsburg GS, Henrich VC. 2013. Collection of family health history for assessment of chronic disease risk in primary care. N C J Med 74:7.
  • Rubinstein WS, Acheson LS, O'Neill SM, Ruffin MTt, Wang C, Beaumont JL, Rothrock N. 2011. Clinical utility of family history for cancer screening and referral in primary care: A report from the Family Healthware Impact Trial. Genet Med 13:956965.
  • Scheuner MT, Hamilton AB, Peredo J, Sale TJ, Austin C, Gilman SC, Bowen MS, Goldzweig CL, Lee M, Mittman BS, Yano EM. 2014. A cancer genetics toolkit improves access to genetic services through documentation and use of the family history by primary-care clinicians. Genet Med 16:6069.
  • Shaikh U, Nelson R, Tancredi D, Byrd RS. 2010. Presentation of body mass index within an electronic health record to improve weight assessment and counselling in children and adolescents. Informatics in Primary Care 18:235244.
  • Sherman S, Pletcher BA, Driscoll DA. 2005. Fragile X syndrome: Diagnostic and carrier testing. Genet Med 7:584587.
  • Welch BM, Kawamoto K. 2013. Clinical decision support for genetically guided personalized medicine: A systematic review. JAMIA 20:388400.
  • Yip VL, Marson AG, Jorgensen AL, Pirmohamed M, Alfirevic A. 2012. HLA genotype and carbamazepine-induced cutaneous adverse drug reactions: A systematic review. Clinical Pharmacol Ther 92:757765.

Supporting Information

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. METHODS
  5. RESULTS
  6. DISCUSSION
  7. ACKNOWLEDGMENTS
  8. REFERENCES
  9. Supporting Information

Additional supporting information may be found in the online version of this article at the publisher's web-site.

FilenameFormatSizeDescription
ajmgc31389-sm-0001-SuppData-S1.docx166KSupporting Information.

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.