Propensity score balance measures in pharmacoepidemiology: a simulation study^†

ABSTRACT

Background

Conditional on the propensity score (PS), treated and untreated subjects have similar distribution of observed baseline characteristics when the PS model is appropriately specified. The performance of several PS balance measures in assessing the balance of covariates achieved by a specific PS model and selecting the optimal PS model was evaluated in simulation studies. However, these studies involved only normally distributed covariates. Comparisons in binary or mixed covariate distributions with rare outcomes, typical of pharmacoepidemiologic settings, are scarce.

Methods

Monte Carlo simulations were performed to examine the performance of different balance measures in terms of selecting an optimal PS model, thus reduction in bias. The balance of covariates between treatment groups was assessed using the absolute standardized difference, the Kolmogorov–Smirnov distance, the Lévy distance, and the overlapping coefficient. Spearman's correlation coefficient (r) between each of these balance measures and bias were calculated.

Results

In large sample sizes (n ≥ 1000), all balance measures were similarly correlated with bias (r ranging between 0.50 and 0.68) irrespective of the treatment effect's strength and frequency of the outcome. In smaller sample sizes with mixed binary and continuous covariate distributions, these correlations were low for all balance measures (r ranging between 0.11 and 0.43), except for the absolute standardized difference (r = 0.51).

Conclusions

The absolute standardized difference, which is an easy‐to‐calculate balance measure, displayed consistently better performance across different simulation scenarios. Therefore, it should be the balance measure of choice for measuring and reporting the amount of balance reached, and for selecting the final PS model. Copyright © 2014 John Wiley & Sons, Ltd.