This article is a U.S. Government work and is in the public domain in the U.S.A.
Special Issue Paper
Quantifying bias in a health survey: Modeling total survey error in the National Immunization Survey†
Article first published online: 4 FEB 2011
This article is a U.S. Government work and is in the public domain in the U.S.A. Published in 2011 by John Wiley & Sons, Ltd.
Statistics in Medicine
Special Issue: 13th Biennial CDC & ATSDR Symposium on Statistical Methods Info-Fusion: Utilization of Multi-Source Data
Volume 30, Issue 5, pages 505–514, 28 February 2011
How to Cite
Molinari, N. M., Wolter, K. M., Skalland, B., Montgomery, R., Khare, M., Smith, P. J., Barron, M. L., Copeland, K., Santos, K. and Singleton, J. A. (2011), Quantifying bias in a health survey: Modeling total survey error in the National Immunization Survey. Statist. Med., 30: 505–514. doi: 10.1002/sim.3911
- Issue published online: 10 FEB 2011
- Article first published online: 4 FEB 2011
- Manuscript Received: 9 FEB 2010
- Manuscript Accepted: 9 FEB 2010
- total survey error;
- Monte Carlo simulation;
- nonresponse bias;
- noncoverage bias;
- survey methodology
Random-digit-dial telephone surveys are experiencing both declining response rates and increasing under-coverage due to the prevalence of households that substitute a wireless telephone for their residential landline telephone. These changes increase the potential for bias in survey estimates and heighten the need for survey researchers to evaluate the sources and magnitudes of potential bias.
We apply a Monte Carlo simulation-based approach to assess bias in the NIS, a land-line telephone survey of 19–35 month-old children used to obtain national vaccination coverage estimates. We develop a model describing the survey stages at which component nonsampling error may be introduced due to nonresponse and under-coverage. We use that model and components of error estimated in special studies to quantify the extent to which noncoverage and nonresponse may bias the vaccination coverage estimates obtained from the NIS and present a distribution of the total survey error.
Results indicated that the total error followed a normal distribution with mean of 1.72 per cent(95 per cent CI: 1.71, 1.74 per cent) and final adjusted survey weights corrected for this error. Although small, the largest contributor to error in terms of magnitude was nonresponse of immunization providers. The total error was most sensitive to declines in coverage due to cell phone only households. These results indicate that, while response rates and coverage may be declining, total survey error is quite small. Since response rates have historically been used to proxy for total survey error, the finding that these rates do not accurately reflect bias is important for evaluation of survey data. Published in 2011 by John Wiley & Sons, Ltd.