The obesity epidemic and changes in self-report biases in BMI

Authors

  • Aiko Hattori,

    Corresponding author
    1. Carolina Population Center, University of North Carolina at Chapel Hill, CB# 8120, Chapel Hill, NC 27516-2524, USA. At the time of the study, the University of California, Los Angeles
    • Carolina Population Center, University of North Carolina at Chapel Hill, CB# 8120, Chapel Hill, NC 27516-2524, USA
    Search for more papers by this author
  • Roland Sturm

    1. RAND Corporation, Santa Monica, CA 90407, USA
    Search for more papers by this author

  • Disclosure: The authors declare no conflict of interest.

  • Funding agencies: This project was supported by grant number T32 HS000046 from the Agency for Healthcare Research and Quality (AH), grant R01HD057193 from NICHD (RS), and grant 1R03CA173040-01 from the National Cancer Institute (RS).

Abstract

Objective:

To assess time trends in measurement error of BMI and the sensitivity/specificity of classifying weight status in the United States by analyzing the difference in BMI between self-reported and measured height and weight.

Design and Methods:

Data from 18,394 respondents aged 20-89 years from the National Health and Nutrition Examination Survey (NHANES) from 1999 through 2008 were analyzed. Multiple linear regression and logistic regression models estimated trends in reporting bias and misclassification of weight status by BMI categories, sex, age, and racial/ethnic groups, adjusting for the sampling design.

Results:

We find no evidence that there are time trends in the accuracy of self-report by BMI categories, sex, age, or racial/ethnic groups. The well-known downward bias in self-report has remained stable over the last decade; approximately one in six to seven obese individuals were misclassified as nonobese due to underestimation of BMI.

Conclusion:

Increases in obesity rates based on self-reported height and weight are likely to reflect actual weight increases and are not inflated by changes in reporting accuracy.

Introduction

Self-reported height and weight have been widely used in surveys to estimate body mass index (BMI). However, self-reported height and weight are subject to measurement error comprised of bias and random error (1). Social desirability bias and recall error are well-known reasons for measurement error (2-4). Overall, measurement error results in underestimation of BMI that differs by weight status and sociodemographics (3, 5, 6).

Increased media coverage of obesity in the past decade may have changed people's awareness of their weight and accuracy of self-report (7). If the relationship between self-report and actual BMI changes, trend estimates or comparisons of surveys at different time points will conflate true changes in BMI with changes in reporting accuracy. In particular, increased accuracy in self-report (e.g., due to more frequent monitoring of weight) would diminish the traditional downward bias in self-reported BMI and could lead to an inflated estimate of actual obesity trends. Two previous studies considered trends in measurement error among Americans, but did not include detailed trend analyses of recent years during which obesity became much more prominent in media and policy (5, 8).

This article investigates whether the relationship between self-report and objective measurement has changed in the United States between 1999 and 2008 by analyzing both bias and random error in the difference between actual BMI and BMI calculated from self-reported height and weight.

Design and Methods

We analyzed combined data from the five continuous 2-year surveys of National Health and Nutrition Examination Survey (NHANES) from 1999 through 2008, a multistage survey of a representative sample of the noninstitutionalized US civilian population. Information on self-reported height and weight was collected during an in-person interview followed by physical examinations shortly afterwards measuring height and weight. The five surveys collected data on a total of 24,693 respondents aged 20-89 years; study details are available elsewhere (9). The current analyses are limited to 18,394 adults aged 20-89 years self-classified as non-Hispanic Whites, non-Hispanic Blacks, or Mexican Americans, excluding 1,078 pregnant women and 3,066 respondents with either “flagged” or missing anthropometric measures. BMI with flag was significantly higher as expected (e.g., for wearing clothes), while self-report and measured BMI did not differ by the missingness status. For sensitivity analysis, we imputed missing values using the MI procedure in STATA, and the qualitative conclusions remained the same.

We considered the difference between the two BMI values (hereafter referred to as “BMI difference”) for each respondent by subtracting “self-reported BMI” (calculated from self-reported height and weight) from “measured BMI” (calculated from measured height and weight). We performed the same analysis separately for height and weight. We examined changes in the mean of the BMI difference (to measure systematic bias) and its variance (to measure accuracy). There were 376 subjects with a BMI difference more than 3 SD-range of the mean. A sensitivity analysis removing those outliers did not affect our conclusions.

Subgroup characteristics of interest included: BMI categories (“nonobese”, defined as a measured BMI < 30, and “obese,” defined as a measured BMI ≥ 30), sex, age groups (20-39 years, 40-59 years, and 60-89 years), racial/ethnic groups (non-Hispanic Whites, non-Hispanic Blacks, and Mexican Americans). Trends in BMI difference were examined through simple and multiple linear regressions; time was measured in survey year (standardized to 0 at year 1999-2000). Sensitivity and specificity of obesity based on self-reported BMI was obtained using measured BMI as the standard. Trends in misclassification were examined through logistic regression. All results were considered statistically significant at P<0.05 unless indicated otherwise. We performed all analyses in STATA version 12.0 (StataCorp LP, College Station, TX) and accounted for the sampling design.

Results

Between 1999 and 2008, self-reported BMI consistently underestimated measured BMI by an average of 0.53 U. Subgroup analyses regressing BMI difference and the squared residual on survey year did not indicate significant linear time trends in the bias or the degree of random error for any of the subgroups (Table 1).

Table 1. Mean and SD of BMI difference by survey year, BMI categories, and socio-demographic characteristicsa
CharacteristicNMean and SD of BMI differenceTest for trendsb
1999–20002001–20022003–20042005–20062007–2008P value
  • a

    Standard deviations (SD) are presented in parentheses. Observations are weighted to be nationally representative of the U.S. population.

  • b

    Test results of trends in BMI difference are based on simple liner regression models regressing BMI difference on survey year. Test results of trends in the standard deviation are based on simple linear regression models regressing squared residuals on survey year.

Overall18,3940.550.490.460.560.570.467
 (1.847)(1.984)(1.664)(1.770)(1.734)0.132
Sex
 Male9,3730.310.220.310.420.360.052
(1.528)(2.014)(1.494)(1.718)(1.595)0.635
 Female9,0210.800.750.610.690.770.601
(2.088)(1.917)(1.803)(1.811)(1.837)0.082
Race/ethnicity
 Non-Hispanic White10,0000.600.490.460.560.590.683
(1.590)(1.844)(1.515)(1.629)(1.570)0.426
 Non-Hispanic Black4,3120.440.530.510.560.520.522
(2.497)(2.197)(2.210)(2.293)(2.175)0.239
 Mexican American4,0820.250.360.340.510.420.115
(2.975)(2.888)(2.064)(2.157)(2.283)0.050
Age
 20–396,2320.460.390.430.420.480.745
(1.813)(2.331)(1.703)(1.821)(1.801)0.259
 40–596,2180.480.430.410.590.530.130
(1.979)(1.797)(1.709)(1.838)(1.701)0.261
 60–895,9440.850.810.620.740.820.511
(1.645)(1.513)(1.486)(1.505)(1.641)0.941
BMI categories
 BMI < 3012,1100.250.180.150.210.210.688
(1.475)(1.685)(1.248)(1.301)(1.287)0.052
 BMI ≥ 306,2841.261.181.111.231.270.690
(2.364)(2.387)(2.163)(2.278)(2.207)0.322

We examined differential trends in the bias by the subgroups through multiple linear regression of BMI difference on survey year, BMI categories, sex, age and racial/ethnic groups, and their two-way interactions with survey year. Survey year or its interactions with each of the subgroup characteristics were not significantly associated with BMI difference, suggesting that there were no meaningful time trends or differential trends by the subgroups. We also assessed quadratic and cubic polynomial models with respect to survey year to test for nonlinear trends and none of the coefficients were significant. BMI categories, sex, and age groups were independently associated with BMI difference. Underestimation of BMI was significantly greater among: obese individuals (compared with nonobese individuals) by 0.96 U (SE = 0.066; P < 0.001), females (compared with males) by 0.42 U (SE = 0.048; P < 0.001), non-Hispanic Whites (compared with Mexican Americans) by 0.22 U (SE = 0.094; P = 0.023) and individuals aged 60-89 years (compared with 20-39 years and 40-59 years) by 0.25 and 0.34 U (SE = 0.074; P = 0.001 and SE = 0.063; P < 0.001).

We examined differential trends in the degree of random error through multiple regression of the squared residual (actual BMI difference minus predicted BMI difference from the model) on the same set of independent variables. Again, there were no significant trends, neither in the direct effect of year nor in interactions with any subgroup characteristics. Reporting accuracy varies across the subgroups (but does not change over time). The squared residual was significantly greater among: obese individuals (compared with non-obese individuals) by 3.00 squared-units (SE = 0.593; P < 0.001), and individuals aged 40-59 years (compared with 60-89 years) by 1.02 squared-units (SE = 0.431; P = 0.020) and non-Hispanic Black and Mexican Americans (compared with non-Hispanic Whites) by 5.13 squared-units (SE = 1.727; P = 0.003) and by 2.07 squared-units (SE = 0.707; P = 0.004), respectively.

We repeated the analysis using height and weight as outcome variables. We did not find changes over time in accuracy, or differential changes by subgroups (Tables 2 and 3).

Table 2. Mean and SD of height difference (measured—self-reported height in centimeters) by survey year, BMI categories, and socio-demographic characteristicsa
CharacteristicNSurvey yearTest for trendsb
1999–20002001–20022003–20042005–20062007–2008P value
  • a

    Standard deviations (SD) are presented in parentheses. Observations are weighted to be nationally representative of the U.S. population.

  • b

    Test results of trends in height difference (measured - self-reported height) are based on simple liner regression models regressing height difference on survey year. Test results of trends in the standard deviation are based on simple linear regression models regressing squared residuals on survey year.

Overall18,394−0.87−0.77−0.88−0.92−0.960.189
(3.036)(3.111)(2.691)(2.663)(2.789)0.091
Sex
 Male9,373−1.19−1.05−1.25−1.28−1.310.167
(2.820)(3.430)(2.724)(2.764)(2.836)0.430
 Female9,021−0.56−0.48−0.53−0.57−0.620.451
(3.205)(2.728)(2.610)(2.510)(2.700)0.059
Race/ethnicity
 Non-Hispanic White10,000−0.90−0.74−0.88−0.93−0.980.235
(2.670)(2.873)(2.445)(2.453)(2.517)0.246
 Non-Hispanic Black4,312−0.73−1.01−0.92−0.86−0.830.969
(3.764)(3.412)(3.167)(2.888)(3.098)0.049
 Mexican American4,082−0.75−0.67−0.81−0.97−0.970.189
(5.041)(4.690)(3.955)(4.017)(4.196)0.146
Age
 20–396,232−0.56−0.52−0.84−0.75−0.780.041
(2.896)(3.689)(2.779)(2.805)(3.059)0.549
 40–596,218−0.61−0.48−0.55−0.67−0.750.120
(3.212)(2.472)(2.500)(2.418)(2.452)0.060
 60–895,944−1.91−1.86−1.61−1.73−1.750.249
(2.767)(2.803)(2.748)(2.719)(2.766)0.840
BMI categories
 BMI < 3012,110−0.77−0.66−0.77−0.83−0.820.318
(3.028)(3.291)(2.614)(2.616)(2.681)0.056
 BMI > = 306,284−1.12−1.01−1.11−1.11−1.240.280
(3.041)(2.650)(2.831)(2.740)(2.966)0.963
Table 3. Mean and SD of weight difference (measured - self-reported weight in kilograms) by survey year, BMI categories, and socio-demographic characteristicsa
CharacteristicNSurvey yearTest for trendsb
1999–20002001–20022003–20042005–20062007–2008P value
  • a

    Standard deviations (SD) are presented in parentheses. Observations are weighted to be nationally representative of the US population.

  • b

    Test results of trends in weight difference (measured - self-reported weight) are based on simple liner regression models regressing weight difference on survey year. Test results of trends in the standard deviation are based on simple linear regression models regressing squared residuals on survey year.

Overall18,3940.760.650.440.700.700.855
(4.215)(4.445)(4.010)(4.619)(4.236)0.724
Sex
 Male9,373−0.15−0.28−0.280.06−0.150.358
(3.671)(4.500)(3.923)(4.959)(4.085)0.134
 Female9,0211.661.571.141.331.520.228
(4.515)(4.191)(3.971)(4.165)(4.220)0.529
Race/ethnicity
 Non-Hispanic White10,0000.830.690.450.710.750.710
(3.819)(4.239)(3.640)(4.408)(3.959)0.593
 Non-Hispanic Black4,3120.580.440.530.740.650.537
(6.057)(5.359)(5.727)(5.841)(5.558)0.692
 Mexican American4,0820.190.530.150.500.280.871
(4.710)(4.962)(4.322)(4.518)(4.384)0.309
Age
 20–396,2320.820.680.420.490.660.287
(4.464)(4.911)(4.150)(4.846)(4.428)0.870
 40–596,2180.770.670.571.010.760.427
(4.160)(4.415)(4.249)(4.914)(4.361)0.413
 60–895,9440.610.520.220.440.630.913
(3.785)(3.396)(3.176)(3.408)(3.527)0.584
BMI categories
 BMI < 3012,1100.08−0.04−0.26−0.13−0.100.135
(3.153)(3.226)(3.041)(3.343)(3.179)0.584
 BMI ≥ 306,2842.322.181.902.272.240.953
(5.704)(6.108)(5.216)(6.073)(5.432)0.678

We compared the sensitivity and specificity of obesity (BMI ≥ 30) to examine the net impact of the measurement error on classification of relative weight status. Because of the downward bias in self-reported BMI, the sensitivity (i.e., the percentage of obese individuals correctly identified as obese by self-reported BMI) was consistently lower than the specificity (i.e., the percentage of non-obese individuals correctly identified as nonobese by self-reported BMI) as a whole and among any subgroups (full results not shown but available from the corresponding author upon request). An estimated 15.5% of obese individuals and 2.4% of nonobese individuals were misclassified as nonobese and obese, respectively, resulting in underestimation of obesity prevalence. Logistic regression analyses did not indicate significant time trends in misclassification (correctly vs. incorrectly classified) for any of the subgroups.

We tested for differential trends in misclassification by the subgroups through multiple logistic regression of the misclassification status on the aforementioned set of independent variables. Survey year or its interactions with each of the subgroup characteristics were not statistically significant. The odds of misclassification among obese individuals was 7.2 times (SE = 1.15; P < 0.001) that among nonobese individuals. Among Mexican Americans, the odds was 2.1 times (SE = 0.03; P < 0.001) that among non-Hispanic Whites. Other subgroup characteristics were not associated with misclassification.

Discussion

The study examined whether the relationship between self-report and objective measurement of relative weight status changed between 1999 and 2008, a plausible scenario given the increased media coverage of obesity. There is no evidence that people's responding behaviors have changed. We find no changes over time in accuracy of self-reported BMI and misclassification of weight status nor any differential changes by subgroups. The downward bias in classifying weight status has remained stable over the last decade; approximately one in six to seven obese individuals were misclassified as nonobese due to underestimation of BMI.

There could be a secondary effect on the accuracy of overall trends to the extent that the US population demographics change. The increasing proportion of Hispanics and older people (10) may add to misclassification of weight status, given the larger random error and the greater downward bias in self-report among those groups. If the population were to continue to gain weight (11) and to have the same level of bias, it could lead to underestimated trends when using self-report. Severe obesity has been increasing even more rapidly than moderate obesity (12), so we would expect that if there is any bias in using self-report for trend estimates, it would be in the direction of underestimating increases. Nevertheless, the lack of time trends in self-report BMI bias among obese individuals, despite the increase in severe obesity and expected increase in reporting bias, may suggest that differentials in self-report biases among obese individuals have diminished over the period. A study reports improved accuracy of self-report BMI among obese individuals between 1988-1994 and 2005-2008, using NHANES data, and relates it to increased acceptance of obesity status (8).

The study findings may be considered as a lower bound of bias and random error associated with self-reported anthropometric measures. In NHANES, self-report may be more accurate because the survey was conducted in-person and it was followed by an examination (3, 13, 14). Self-report biases may be larger in telephone surveys (e.g., the Behavioral Risk Factor Surveillance System), or in-person interviews without physical examinations (e.g., the National Health Interview Survey) (3, 13, 15).

Acknowledgements

The study presented here was conducted in collaboration among all authors. AH analyzed the data and prepared the manuscript. RS discussed study design and analysis, revised the manuscript, and supervised the study. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality, NIH or the National Cancer Institute.

Ancillary