Generalizing Observational Study Results: Applying Propensity Score Methods to Complex Surveys
Article first published online: 16 JUL 2013
© Health Research and Educational Trust
Health Services Research
Volume 49, Issue 1, pages 284–303, February 2014
How to Cite
DuGoff, E. H., Schuler, M. and Stuart, E. A. (2014), Generalizing Observational Study Results: Applying Propensity Score Methods to Complex Surveys. Health Services Research, 49: 284–303. doi: 10.1111/1475-6773.12090
- Issue published online: 17 JAN 2014
- Article first published online: 16 JUL 2013
- Manuscript Accepted: 11 MAY 2013
- National Institute of Mental Health. Grant Number: K25 MH083846
- Survey research;
- primary care;
- health care costs
To provide a tutorial for using propensity score methods with complex survey data.
Simulated data and the 2008 Medical Expenditure Panel Survey.
Using simulation, we compared the following methods for estimating the treatment effect: a naïve estimate (ignoring both survey weights and propensity scores), survey weighting, propensity score methods (nearest neighbor matching, weighting, and subclassification), and propensity score methods in combination with survey weighting. Methods are compared in terms of bias and 95 percent confidence interval coverage. In Example 2, we used these methods to estimate the effect on health care spending of having a generalist versus a specialist as a usual source of care.
In general, combining a propensity score method and survey weighting is necessary to achieve unbiased treatment effect estimates that are generalizable to the original survey target population.
Propensity score methods are an essential tool for addressing confounding in observational studies. Ignoring survey weights may lead to results that are not generalizable to the survey target population. This paper clarifies the appropriate inferences for different propensity score methods and suggests guidelines for selecting an appropriate propensity score method based on a researcher's goal.