Evaluating the impact of point- of-care decision support tools in improving diagnosis of obese children in primary care

Authors

  • Christine R. Ayash,

    Corresponding author
    1. Obesity Prevention Program, Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts, USA
    • Obesity Prevention Program, Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts, USA

    Search for more papers by this author
  • Steven R. Simon,

    1. Division of General Medicine and Primary Care, Section of General Internal Medicine, VA Boston Healthcare System, Brigham and Women's Hospital, Boston, Massachusetts, USA
    Search for more papers by this author
  • Richard Marshall,

    1. Department of Pediatrics, Harvard Vanguard Medical Associates, Boston, Massachusetts, USA
    Search for more papers by this author
  • Jill Kasper,

    1. Department of Pediatrics, Cambridge Health Alliance, Cambridge, Massachusetts, USA
    Search for more papers by this author
  • Virginia Chomitz,

    1. Institute for Community Health, Cambridge Health Alliance, Cambridge, Massachusetts, USA
    2. Institute for Community Health, Harvard Medical School, Boston, Massachusetts, USA
    Search for more papers by this author
  • Karen Hacker,

    1. Institute for Community Health, Cambridge Health Alliance, Cambridge, Massachusetts, USA
    Search for more papers by this author
  • Ken P. Kleinman,

    1. Obesity Prevention Program, Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts, USA
    Search for more papers by this author
  • Elsie M. Taveras

    1. Department of Ambulatory Care and Prevention, Harvard Medical School, Bosto, Massachusetts, USA
    Search for more papers by this author

  • Funding agencies: This research was supported in part by grants from The American Diabetes Association (Award #1-09-CR-49) and The American Recovery and Reinvestment Act (Award #R18 AE000026). Disclosure: The authors declared no conflicts of interest.

Abstract

Objective:

The purpose of this quasi-experimental study was to examine the effect of a computerized point-of-care alert with clinical decision support on the rates of diagnosis of childhood obesity in a multisite group practice in Massachusetts; Cambridge Health Alliance (CHA) which implemented an alert, relative to a separate group practice, Harvard Vanguard Medical Associates (HVMA), that did not.

Design and Methods:

Height and weight data from 19,466 children of 2-18 years with 34,908 well-child care visits in CHA and 123,446 children with 282,271 visits in HVMA between 2006 and 2008 were collected. The alert and decision support tool was activated for CHA patients with an age- and sex-specific body mass index of ≥95th percentile. The main outcome measure was documentation of an International Classification of Diseases, Ninth Revision [ICD-9] code for obesity before and after implementation of the alert at CHA in 2007.

Results:

Among obese children, the adjusted rate of an ICD-9 diagnosis of obesity increased from 2006-2007 to 2008 significantly more at CHA than at HVMA (P < 0.001 for time-by-provider group interaction). In 2006-2007, the rate of ICD-9 diagnosis of obesity was significantly lower at CHA than at HVMA (adjusted odds ratio [OR]: 0.57; 95% confidence interval [CI]: 0.52-0.62); but by 2008 was significantly higher at CHA than HVMA (adjusted OR: 1.25; 95% CI: 1.14-1.38).

Conclusion:

A point-of-care alert was effective in improving obesity diagnosis in a multisite group practice, relative to a separate group practice that did not adopt an alert. Clinical decision support tools could help improve obesity diagnosis in pediatric primary care.

Introduction

Childhood obesity is highly prevalent (1-3) and is associated with both short- and long-term adverse outcomes (4-10). In recognition of the public health importance of childhood obesity, the American Medical Association (AMA), the Health Resources and Services Administration (HRSA), and the Centers for Disease Control and Prevention (CDC) released Expert Committee Recommendations on the Assessment, Prevention, and Treatment of Child and Adolescent Overweight and Obesity in 2007, a revision of guidelines originally issued in 1998 (11). More recently in 2010, the United States Preventive Service Task Force recommended screening for obesity in children and adolescents as young as 6 years of age, further highlighting the wide-reaching importance of this clinical and public health issue (12). Despite the availability of these recommendations for nearly a decade and more recent recommendations for screening of obesity, the healthcare system has been slow to adopt recommended practices (13-16).

Clinical decision support tools in electronic health records (EHRs) offer potential for improving diagnosis and management of overweight and obese children and accelerating clinicians' adoption of childhood obesity evidence-based recommendations (17). Studies in adults have demonstrated the effectiveness of clinical decision support in outpatient settings (18). Point-of-care alerts and reminders have been useful in adult patients, for example, in improving the safety and quality of medication prescribing (19, 20). In pediatric outpatient settings, decision support has already been shown to improve prescribing patterns (21), increase immunization rates (22), and improve delivery of preventive asthma care (23). Only one study reported the implementation of a clinical decision support tool for pediatric obesity management and described increased trends in obesity counseling as a result, demonstrating the potential effectiveness of an alert and decision support in pediatric obesity management (24). However, few studies have examined the effects of such EHR tools in identification and management of obese patients (25).

The purpose of this study was to take advantage of a natural experiment, using a quasi-experimental design, to examine the predicted probability of an obesity diagnosis by pediatricians in two healthcare systems—one of which implemented a computerized point-of-care alert and clinical decision support for pediatric obesity screening and management in 2007.

Methods

Study design

We used data from 2006 to 2008 to evaluate the changes in the predicted probability of obesity diagnosis coding of overweight and obese children by pediatricians in two health systems: Cambridge Health Alliance (CHA) and Harvard Vanguard Medical Associates (HVMA) provider groups, both in eastern Massachusetts. We assessed obesity diagnosis coding before and after a point-of-care alert with decision support system (EPIC® Obesity Smart Set) was implemented at CHA to help providers identify and manage obese patients. We used a quasi-experimental approach (Figure 1) to estimate the changes in diagnosis coding of overweight and obese children by provider group (CHA vs. HVMA), controlling for patients' characteristics.

Figure 1.

Quasi-experimental study design. This figure shows the quasi-experimental study design, demonstrating the change in rate of obesity diagnosis coding before and after the implementation of the alert and clinical decision tool. The start-point of the EHR-based intervention is represented by the vertical gray line.

Setting and intervention

CHA is an academic public healthcare system that serves seven major communities in north of Boston. In 2006 and 2007, CHA ambulatory sites provided care for more than 20,000 children who were 2-18 years old in more than 100,000 visits. HVMA is a nonprofit, academically affiliated multispecialty group practice providing care to approximately 100,000 2- to 18-year old children at 17 sites across eastern Massachusetts. Human subjects committees of Harvard Vanguard Medical Associates and Cambridge Health Alliance approved the study protocols.

HVMA and CHA utilize an electronic medical record system (EpicCare; Epic systems Corporation, Verona, WI) for all outpatient encounters, including for well-child care visits. HVMA and CHA have used EpicCare, for all clinical encounters since 2000 and 2001, respectively. Beginning in December 2007, a point-of-care alert (Figure 2) was implemented at CHA that alerted pediatricians to patients with an age- and sex-specific body mass index (BMI) of ≥95th percentile, defined as obese. For children 2-18 years old with a BMI of ≥95th percentile, the alert led clinicians to complete a SmartSet®; a structured progress note included: (1) documentation and coding of a BMI percentile and diagnosis of obesity (ICD-9 Diagnosis Code V85.5), (2) documentation of counseling on nutrition (ICD-9 V65.3) and physical activity (ICD-9 V65.41), (3) prompts to order fasting laboratory testing if appropriate, (4) recommendations for a return visit or referrals, and (5) parent educational materials.

Figure 2.

Example of the point-of-care alert for a patient with a BMI of ≥95th percentile developed and tested at Cambridge Health Alliance. This figure shows the point-of-care alert designed in the electronic medical record system at Cambridge Health Alliance. The highlighted portion of the screen is the alert notifying the clinician that their patient's BMI percentile is ≥95th percentile. Below the alert message, clinicians can access the obesity-related SmartSetR, a structured progress note that included documentation and coding of a BMI percentile and diagnosis of obesity, documentation of counseling on nutrition and physical activity, prompts to order fasting laboratory testing, if appropriate, recommendations for a return visit or referrals, and parent educational materials. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The standard workflow for well-child care visits involves the medical assistant measuring the height and weight of the child, entering those data into the EMR, flagging the visit as well-child care, and notifying the clinician that the patient is in the examination room. Clinicians routinely access the EMR in the examination room during the visit; when the child's record is opened, clinicians receive the pop-up alert (i.e., a new window “in front of” the screen on which they were working) with brief educational content provided by the SmartSet to prompt clinicians to document BMI percentile, document a diagnosis of obesity, measure and document blood pressure, discuss and document counseling on nutrition and physical activity, and provide educational materials to help manage childhood obesity.

Compared with CHA, HVMA did not have a point-of-care alert to notify clinicians of patients with a BMI of ≥95th percentile or other tailored electronic decision support tools for the management of pediatric obesity during the study period. Usual obesity screening at HVMA consists of measuring children's height and weight, calculating BMI, and plotting the data on a growth chart to monitor and assess a child's growth and development. Decisions about management of obesity are left to the discretion of each clinician but include referrals to internal or outside weight-management programs.

Data sources and measures

We used electronic data from CHA and HVMA from January 1, 2006 to December 31, 2008 for all children of 2-18 years who were seen for well-child care visits and had their height and weight recorded. The rationale for selecting this particular time period was to assess diagnosis rates for overweight and obese children before and after the release of Expert Committee Guidelines of Assessment, Prevention, and Treatment of Child and Adolescent Overweight and Obesity, by CDC, AMA, and HRSA in June 2007 (11). In addition, this period included 24 months before the alert was implemented at CHA and 12 months after implementation.

We extracted anthropometric and demographic data from a total of 19,466 CHA patients with 34,908 well-child care visits (mean visits per patient = 2) and a total of 123,446 HVMA patients with 282,271 well-child care visits (mean visits per patient = 2) from 2006 to 2008 for analysis. In addition, all International Classification of Diseases, Ninth Revision (ICD-9) diagnosis codes were extracted for each patient visit, including overweight/obesity diagnosis codes 278.00, 278.01, 278.02, 783.1, and 277.7.

Patients' characteristics of interest included age, gender, race and ethnicity, height, and weight. We calculated BMI and age- and sex-specific BMI percentiles and z-scores using US national reference data (26).

The outcome of interest was the presence of an ICD-9 code for overweight and obese weight categories. The rationale for combining the overweight and obese diagnostic codes was that the definitions of obesity and overweight categories used in national guidelines changed in 2007, midway through our study period. In 2007, although the categories did not change, changes in terminology were proposed. BMI-for-age from 85th to 95th percentile was defined as “overweight” (instead of “at risk of overweight”) and BMI-for-age at or above 95th percentile was defined as “obesity” (instead of “overweight”). When reporting trends over time, we applied the new terminology for consistency. In addition, combining the overweight and obese diagnostic codes provides a more sensitive method for capturing the effect of the alert on diagnostic coding.

The dichotomous outcome indicated a diagnosis of overweight or obese weight categories based on the ICD-9 codes entered by clinicians during the visits (278.00 obesity, unspecified; 278.01 morbid obesity, 277.7 dysmetabolic syndrome, 278.02 overweight, 783.1 weight gain abnormal, V85.53 85-95th BMI percentile, or V85.54 ≥95th BMI percentile). We chose to include multiple options for overweight and obesity diagnostic coding because although the decision support tool provided several preselected choices for diagnosis coding, it was possible for clinicians to use the decision support tool but they have selected other codes which were more comfortable to use. Our approach allowed for maximum flexibility in assessing our outcomes.

Statistical analysis

We first examined univariate distributions for continuous and categorical variables and compared the results across pediatric provider groups (CHA vs. HVMA) using t-tests and χ2-square tests, respectively. Using multivariable logistic regression, we compared the probability of obesity diagnosis coding by provider group before and after the implementation of the decision alert system at CHA at the end of 2007, and adjusted these comparisons for patient gender, age, and race/ethnicity.

Specifically, we fit models predicting the presence of an appropriate diagnosis code for overweight or obese patients as a function of provider group (CHA vs. HVMA), an indicator for year 2008 (after the implementation of the EHR intervention at CHA), interaction between provider group and year 2008, a continuous time variable in years centered around 2006 (range, 0-2), and gender, age, race/ethnicity centered around their means. Thus, these models estimated adjusted differential changes from 2006-2007 to 2008 in predicted probability of appropriate obesity diagnosis coding for CHA relative to HVMA.

We calculated the predicted probability for each comparison group and year from model coefficients and plotted these probabilities to facilitate the interpretation of the results. In a sensitivity analysis, we allowed for trends from 2006 to 2007 to differ by provider group by introducing an interaction term between year and provider group CHA. In controlling for this difference in trend before 2008, the differential increase in mean adjusted probability of diagnosis at CHA was slightly diminished but still large and significant. In addition, we compared monthly time trends in predicted probability of obesity diagnosis between CHA and HVMA before the EHR intervention. Predicted probability of obesity diagnosis increased slightly but significantly more at CHA relative HVMA over this period (P = 0.051).

We used generalized estimating equations to account for correlated data arising from repeated measures (multiple visits per patient). All covariates were centered about their means for the analysis so that predictions from the model for each comparison group and each year could be interpreted as the predicted probability of obesity diagnosis for the average child. We used SAS version 9.1 (Cary, NC) for all analyses.

Results

Sample characteristics, by provider group, are summarized in Table 1. Overweight and obese patients from Cambridge Health Alliance were younger, more likely to be black or Hispanic, and more likely to be obese versus overweight than HVMA patients. Among overweight and obese patients, the proportion of visits with a documented diagnosis of overweight or obesity was <20% for both CHA and HVMA provider groups.

Table 1. Characteristics of overweight and obese children seen for well-child care visits in CHA and HVMA in 2006 through 2008a
CharacteristicCHA (N = 13,770 visits)HVMA (N = 81,367 visits)
  • a

    Total sample size was 34,908 well-child care visits in CHA and 282,271 visits in HVMA. Characteristics presented in this table are of the subsample of visits of children who were overweight or obese.

Age in years, mean (sd)9.4 (4.6)10.3 (4.4)
Weight category (%)
85–95th BMI percentile (overweight)46.154.7
>95th BMI percentile (obese)53.945.2
BMI percentile, mean (sd)94.5 (4.3)93.6 (4.3)
BMI z-score, mean (sd)1.78 (0.54)1.65 (0.44)
Percent of visits with overweight/obesity diagnosis
No83.582.6
Yes16.517.4
Female (%)46.946.2
Race/ethnicity (%)
White36.452.7
Black21.610.0
Hispanic21.05.0
Other19.13.3
Missing1.8429.0

Among children identified as obese (BMI ≥95th percentile)

During the 2006-2007 period prior to the implementation of the point-of-care alert and decision support EHR tool, obese children at CHA were significantly less likely to receive a diagnosis code of overweight or obese than obese children at HVMA (adjusted OR: 0.57; 95% CI: 0.52-0.62; P < 0.001) (Table 2). From 2006-2007 to 2008, however, predicted probability of obesity diagnosis coding increased substantially more at CHA than HVMA. Consequently, obese children at CHA were more likely to receive a diagnosis code of overweight or obesity than obese children at HVMA in 2008 (OR: 1.25; 95% CI: 1.14-1.38; P < 0.001; estimate not shown in tables), constituting a significant differential increase for obesity diagnosis at CHA (ratio of OR: 2.16; 95% CI: 1.92-2.44; P < 0.001) (Table 2 and Figures 3 and 4)5.

Figure 3.

Unadjusted predicted probability of diagnosis of obesity among children of 2–18 by month from January 2006 to December 2008 by provider group. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 4.

Adjusted predicted probability of diagnosis of obesity among children of 2–18 years from 2006 to 2008, by Provider Group. The mean adjusted probability of diagnosis is plotted by site and year. The mean adjusted probabilities were calculated from model coefficients after fitting separate logistic regression models for each comparison group and year.

Figure 5.

Adjusted Predicted Probability of Diagnosis of Overweight Among Children Age 2-18 Years from 2006 to 2008, by Provider Group. The mean adjusted probability of diagnosis is plotted by site and year. The mean adjusted probabilities were calculated from model coefficients after fitting separate logistic regression models for each comparison group and year.

Table 2. Multivariate adjusted rates of diagnosis of obesity among children of 2–18 years with a BMI of ≥95th percentile at CHA and HVMA from 2006 to 2008
Model variableOR95% CI
  • a

    Covariates were centered about their means.

Provider group
HVMA1.00 (ref)
sCHA0.570.52, 0.62
Year
2006–20071.00 (ref)
20080.830.75, 0.92
Year (continuous variable: annual change from 2006 to 2007 before the intervention for CHA and HVMA)0.161.10, 1.25
Interaction term of year (2008) by provider group (CHA)2.161.92, 2.44
Gender
Male1.00 (ref)
Female1.251.19, 1.33
Race/ethnicitya
White1.00 (ref)
Black1.421.32, 1.53
Hispanic1.341.23, 1.48
Other1.421.28, 1.59
Race/ethnicitya
White1.00 (ref)
Black1.421.32, 1.53
Hispanic1.341.23, 1.48
Other1.421.28, 1.59
Age (years)a
2–50.34(0.31, 0.37)
6–110.77(0.73, 0.82)
12–181.00 (ref)

From 2006 to 2007, increases in the predicted probability of obesity diagnosis coding were slightly greater for CHA than HVMA (+4.6% points for HVMA and +5.9% points for CHA; P = 0.04 for differential increase) (Figure 3). When we controlled for this difference in trend before 2008, the differential increase in the predicted probability at CHA was still significant.

In adjusted analysis of obese children across both group practices, black and Hispanic children were more likely than white children, older children were more likely than younger children, and female children were more likely than male children to receive an obesity diagnosis code for their visit.

Among children identified as overweight (BMI between the 85th and the <95th percentiles)

During the 2006-2007 period prior to the implementation of the alert and decision support tool, overweight children at CHA were significantly less likely to receive a diagnosis code of overweight or obesity than overweight children at HVMA (OR: 0.48; 95% CI: 0.39-0.60; P < 0.001) (Table 3). Increases in the predicted probability of an overweight diagnosis from 2006-2007 to 2008 did not differ for children at CHA relative to HVMA (OR: 1.13; 95% CI: 0.84-1.50; P = 0.42) (Table 3 and Figure 4). Overweight children in both practice groups were much less likely to receive an overweight-related diagnosis, as compared with obese children (Figures 3 and 4).

Table 3. Multivariate adjusted rate of diagnosis of overweight among children of 2–18 years with a BMI of ≥85th percentile and <95th percentile at CHA and HVMA from 2006 to 2008
Model variableOR95% CI
  • a

    Covariates were centered about their means.

Provider group
HVMA1.00 (ref)
CHA0.480.39, 0.60
Year
2006–20071.00 (ref)
20080.690.57, 0.85
Year (continuous variable: annual change from 2006 to 2007 before the intervention for CHA and HVMA)1.621.42, 1.85
Interaction term of year (2008) by provider group (CHA)1.130.84, 1.50
Gender
Male1.00 (ref)
Femalea1.36(1.24, 1.51)
Race/ethnicitya
White1.00 (ref)
Black1.471.31, 1.65
Hispanic1.431.19, 1.71
Other1.150.94, 1.41
Age (years)a
2–50.42(0.36, 0.48)
6–110.79(0.71, 0.88)
12–181.00 (ref)

Comment

In this study of more than 140,000 children with more than 300,000 weight, height, and BMI assessments from two healthcare systems in Massachusetts, we found that a computerized point-of-care alert and decision support tool for pediatric obesity screening and management resulted in increased predicted probability of obesity diagnosis coding, an important first step in obesity management. After implementation of a point-of care alert, we found substantially higher predicted probability of obesity diagnosis coding relative to a health system that did not implement a decision support tool for pediatric obesity. We found no evidence of an increase in coding for overweight children which the alert and decision support did not target.

Our findings are consistent with the previous studies, showing that EHR-based decision support can be effective in encouraging the adoption of evidence-based recommendations (24). Decision support tools have been increasingly used and show promise in pediatric settings. Several studies have shown that computerized decision support tools improve pediatricians' performance in prescribing patterns (21), immunization rates (22), and asthma care (23, 27). To our knowledge, our study is the first to examine differences in diagnosis of obesity after the implementation of an electronic decision support tool. Our study not only examined differences across heathcare sites (CHA vs. HVMA) but also assessed differential improvements in diagnostic coding associated with the implementation of a quality improvement initiative, specifically focused on improving documentation and monitoring of obese children. Although other studies have been able to examine trends of documentation rates of overweight and obesity (28), our natural experiment enabled us to compare these rates across two sites that were similar in EHR functionality at baseline but differed in functionality after an intervention. The additional electronic clinical decision support functions at CHA may have contributed to rapid improvements in documentation in obesity that resulted in performance that surpassed HVMA's documentation rate after the intervention.

Diagnosis and disclosure are the first key steps toward appropriate management and counseling for children with obesity. Identification of obesity by pediatricians, based on the diagnostic codes, has been associated with the provision of other important diagnostic and treatment practices (13, 29, 30). Identification can lead to increased communication with families and communication of obesity status has been associated with a positive impact on attempted weight loss and dietary behaviors (31).

There is a growing consensus that EHRs may provide a powerful platform for improving clinical care and patient outcomes (32, 33), but that adoption of EHRs alone may be insufficient. Provider groups need to implement highly functional EHRs, consistent with “meaningful use” criteria established by the Centers for Medicare and Medicaid Services, for these benefits to be realized. The EHR enhancement that CHA implemented in this study meets the core objectives and goals of meaningful use by activating pediatric clinicians to achieve improvements in recording and maintaining patient vital signs and chart changes (height, weight, blood pressure, BMI, and growth charts for children) and problem lists of current and active diagnoses. Thus, our findings support the notion that EHRs should be fully functional, complete with computerized clinical decision-support systems tailored to the patient population to address health problems effectively.

There were several limitations to our study. First, we had limited information on patient socioeconomic status and no health insurance information from CHA participants and therefore could not control for differences in these characteristics between HVMA and CHA patients. Second, it is possible that concurrent quality improvement efforts regarding pediatric obesity in CHA could have explained our results. For example, the roll out of the decision support tools at CHA did include clinician training on use of the tools and it is possible that this training only could have changed clinical practice. Third, it is also possible that the increase in diagnosis rates that we attributed to the EHR intervention may have been explained in part by the response of CHA providers to national guidelines released during the study period. However, in a separate study of in-depth interviews, clinicians in both CHA and HVMA reported that they were not familiar with the national guidelines (34). Fourth, our findings may only be generalizable to ambulatory health centers that use an EHR that features electronic decision support tools. Although most practices nationwide currently lack such systems, EHRs adoption rates are expected to climb in the forthcoming years. Thus, the electronic tools we studied are likely to generalize more and more pediatric settings in the future. Finally, we primarily focused our analysis on change in predicted probability of obesity diagnosis. We relied largely on ICD-9 codes and did not conduct medical record reviews to determine whether obesity was documented elsewhere in the medical chart. Our approach may have underestimated the true predicted probability of obesity diagnosis. However, in a study by Benson et al. (27) that included a chart review, the authors found that the predicted probability of obesity diagnosis remained low, demonstrating that overweight in children is greatly under-diagnosed even when other forms of documentation were considered.

Our study suggests that the use of alerts with decision support tools offers potential for accelerating the adoption of childhood obesity evidence-informed recommendations and can help improve the quality of obesity diagnosis in pediatric primary care. As appropriate diagnosis is the first step required for appropriate pediatric obesity assessment and management, the development and testing of electronic decision support tools in pediatric primary care may help support patients and their clinical teams in care improvement and obesity management.

Acknowledgements

The authors are grateful for Alice Knowles, manager of data services at the Institute for Community Health, for her work in programming and extracting data for this study analysis.

Ancillary